'rdIntersectSegmentCylinder' is a new function that will be used for intersection tests on cylindric clip volumes.
'rdIntersectSegmentAABB' was already present in the library, but used and reimplemented across multiple source files (pending deduplication).
Instead of a box query, just take the position in the tile grid the land-side off-mesh vert is on. This vastly improves the query reliability for off-mesh links with a larger radius to the point it never fails if it is in a valid location. The box query isn't needed here as the off-mesh vert has to be on a tile, else the polygon box query will fail (unless its bounding box does happen to overlap with the point, but in this case the off-mesh connection is placed improperly causing the only factor to have its link established being luck).
The same constants are used between Recast and Detour, but Recast had to declare and match the constant locally as it couldn't access the one from the Detour library. This patch avoids duplicate definitions and potential issues during refactors.
The algorithm has been moved to the dtNavMesh class, and now features a parameter based design allowing code that utilizes the algorithm to define engine-specific tests such as the raycasting, traverse masking, etc. The code can now be hooked up to any level editor without changing a line of code inside the Recast & Detour library. All parameters have been documented thoroughly. The code now also properly returns errors if out of memory, or if invalid parameters were provided. This commit does not affect the results of the traverse link generation, the navmesh remains identical to the one generated on the previous design.
pmin and pmax pointed to these stack arrays, but these stack arrays were inside the loop while pmin and pmax, along with their usages were outside the scope of the loop causing undefined behavior when they pointed to the currently out-of-scope stack variables. Moved the arrays outside the loop so they remain valid up to the last point they are needed.
dtMeshHeader::bvNodeCount is always set to polycount*2, but when we have null bv nodes, then these will also be used during dtNavMeshQuery::queryPolygonsInTile, which can yield incorrect results when the null nodes overlaps query box. Also added an additional optimization to only store non-null BV nodes which can save a significant amount of memory during the runtime. After this bug fix and optimization, the resulting mesh still works as intended in-game.
Fully unlinked can be removed entirely, semi unlinked needs a rebuild to remove unlinked polygons (still work in progress). Added ability to automatically remove fully unlinked tiles.
This function was removed in commit 041d02cd654d94bf4b059c212cb6a63a39e5324e but it should've been kept in as it was still used for dtNavMeshQuery::findRandomPointAroundCircle. Only the quantizer should be removed.
Let recast compute this instead of detour. We need this before the detour navmesh is getting build as we have to determine whether a polygon is too small before that, and flag it as such.
Now also gets stored in the project file. Also reordered the way each field gets stored so its easier to tweak or add/remove outside the editor by external tools.
Handle it in the same loop as standard traverse links, off-mesh links also have a link determining its traverse type after it has been reverse engineered and fixed. The output and behavior is now correct.
We would only clear polygons that weren't marked as unlinked, this was introduced when the prune tool was refactored to use the unlinked polygroup tag. The side effect is, when you remove a tile, and cause a polygon on a neighbor tile to not link to anything, it will be marked as unlinked which is the correct behavior, but when you readd the tile, the entire island will be marked as unlinked.
The function dtCreateDisjointPolyGroups got fixed later by not taking traverse links into account, since polygon islands linked together with traverse links are still disjoint. This was also the time the check had to be removed when clearing all labels but this didn't happen.
Removing and readding tiles now work properly after this patch.
