Detour_LevelInit resets the global NavMesh query, but if code is going to utilize the global Detour query for an existing query on a subsequent frame without attaching a NavMesh in the mean time, the code will crash. The previously attached NavMesh type has to be reattached after the hotswap.
Slight optimization to skip unsupported ones out before computing anything, and also improves the generation results as we try to get the best one out of the supported ones, instead of taking one from supported and unsupported and then later check if its unsupported. (if the best type is 9, and 9 isn't supported while 8 is, and 8 would've been the candidate after 9, then this patch will still give the link a chance for 8 while the previous implementation would've skipped it entirely and returned DT_NULL_TRAVERSE_TYPE, ultimately resulting in no link whatsoever.
The algorithm has been moved to the dtNavMesh class, and now features a parameter based design allowing code that utilizes the algorithm to define engine-specific tests such as the raycasting, traverse masking, etc. The code can now be hooked up to any level editor without changing a line of code inside the Recast & Detour library. All parameters have been documented thoroughly. The code now also properly returns errors if out of memory, or if invalid parameters were provided. This commit does not affect the results of the traverse link generation, the navmesh remains identical to the one generated on the previous design.
pmin and pmax pointed to these stack arrays, but these stack arrays were inside the loop while pmin and pmax, along with their usages were outside the scope of the loop causing undefined behavior when they pointed to the currently out-of-scope stack variables. Moved the arrays outside the loop so they remain valid up to the last point they are needed.
dtMeshHeader::bvNodeCount is always set to polycount*2, but when we have null bv nodes, then these will also be used during dtNavMeshQuery::queryPolygonsInTile, which can yield incorrect results when the null nodes overlaps query box. Also added an additional optimization to only store non-null BV nodes which can save a significant amount of memory during the runtime. After this bug fix and optimization, the resulting mesh still works as intended in-game.
Fully unlinked can be removed entirely, semi unlinked needs a rebuild to remove unlinked polygons (still work in progress). Added ability to automatically remove fully unlinked tiles.
This function was removed in commit 041d02cd654d94bf4b059c212cb6a63a39e5324e but it should've been kept in as it was still used for dtNavMeshQuery::findRandomPointAroundCircle. Only the quantizer should be removed.
Titanfall 2 navmeshes flag their polygons as follows. The pattern was always that if a polygon connects to a polygon on a neighboring tile, it should be flagged as EDITOR_POLYFLAGS_HAS_NEIGHBOUR, and if the polygon's surface area isn't higher than 120 (NAVMESH_SMALL_POLYGON_THRESHOLD) it should be flagged as EDITOR_POLYFLAGS_TOO_SMALL.
Let recast compute this instead of detour. We need this before the detour navmesh is getting build as we have to determine whether a polygon is too small before that, and flag it as such.
Instead of using just the polygon edges, use the detail edges as well. Ray cast errors have been fully eliminated; nothing ever clips through geometry anymore. The results are also a lot more detailed now as we can connect various sub edges together. The resulting navmesh also performs better in-game, the npc's are now a lot more fluent with jumps and climbs, they rarely clip though geometry, if they do its typically an error with the input geometry used to build the navmesh.
There was also a bug in GetBestTraverseType; the exclusive check (where at least one overlap had to occur) returned true if both overlap tests failed. This has been fixed by flipping the check and now checking if at least one overlaps when an exclusive test was specified.
Use rdClassifyPointOutsideBounds to mark neighbor tile so it only gets marked if we mark a tile, and a polygon outside the marked tile. Also added an optimization so the same tile doesn't get rendered twice (side == 0xff).
Now also gets stored in the project file. Also reordered the way each field gets stored so its easier to tweak or add/remove outside the editor by external tools.
Handle it in the same loop as standard traverse links, off-mesh links also have a link determining its traverse type after it has been reverse engineered and fixed. The output and behavior is now correct.
We would only clear polygons that weren't marked as unlinked, this was introduced when the prune tool was refactored to use the unlinked polygroup tag. The side effect is, when you remove a tile, and cause a polygon on a neighbor tile to not link to anything, it will be marked as unlinked which is the correct behavior, but when you readd the tile, the entire island will be marked as unlinked.
The function dtCreateDisjointPolyGroups got fixed later by not taking traverse links into account, since polygon islands linked together with traverse links are still disjoint. This was also the time the check had to be removed when clearing all labels but this didn't happen.
Removing and readding tiles now work properly after this patch.
From now on, internal links are connected first before external neighbor links. This now also happens in the same pass which improves performance (this yields better results and behavior after the last few and current changes).
Also added a fine tunable test dictating whether links should only connect when edges overlap, or when one of the edges overlap (can be tweaked by setting an elevation trigger, by skipping out on lower elevations if this deems necessary, but typically doesn't).
The slope angle option was originally in the table but removed. After reconsideration this has been brought back in as you cannot define min/max slopes with 3d distances and elevation differences. The distance either needs to be 2d (which will break the max 2550.f distance per link rule), or an additional slope check has to be used to define this properly.
After a lot of fine tuning, the results appear to be almost identical to the original navmeshes shipped with Respawn games. The algorithm is also a lot faster now (went from 30 seconds to 8 seconds on the staging area with much better and more accurate results).
The direction is represented over the XY plane, previously XZ on Recast's original coordinate system. There were multiple inconsistencies in Recast where the XZ plane got represented as XY and visa versa.
