Recast: fix incorrect quantization for poly surface area

Factor should be 100, as 512x512 tiles with only 4 verts in Titanfall navmeshes are 5242, ours are twice as large +1. Our tiles are also 512x512 for small to medium navmeshes, and this constant results in nearly identical values, it was still off with 1 (5243 vs 5242).

Probing another poly in a Titanfall 2 navmesh, resulted in a polygon surface area of 1152.0, quantized by 100 is 11.52, the value stored in the Titanfall 2 navmesh was 11 so the value needs to be floored.

The values are now identical between custom and original navmeshes.
This commit is contained in:
Kawe Mazidjatari 2024-07-12 22:29:16 +02:00
parent f5834c6077
commit 7c53cdca2b
2 changed files with 2 additions and 2 deletions

View File

@ -83,7 +83,7 @@ static const int DT_MIN_POLY_GROUP_COUNT = 3;
static const int DT_MAX_TRAVERSAL_TABLES = 5;
/// The cached poly surface area quantization factor.
static const float DT_POLY_AREA_QUANT_FACTOR = 50.f;
static const float DT_POLY_AREA_QUANT_FACTOR = 100.f;
/// @{
/// @name Tile Serialization Constants

View File

@ -814,7 +814,7 @@ bool dtCreateNavMeshData(dtNavMeshCreateParams* params, unsigned char** outData,
p->vertCount++;
}
dtVscale(p->center, p->center, 1 / (float)(p->vertCount));
p->surfaceArea = (unsigned short)dtMathRoundf(dtCalcPolySurfaceArea(p,navVerts) / DT_POLY_AREA_QUANT_FACTOR);
p->surfaceArea = (unsigned short)dtMathFloorf(dtCalcPolySurfaceArea(p,navVerts) / DT_POLY_AREA_QUANT_FACTOR);
src += nvp*2;
}