580 Commits

Author SHA1 Message Date
Kawe Mazidjatari
a546836a10 RTech: small cleanup for PakDecoder_s
Removed unused union
2024-04-05 18:40:36 +02:00
Kawe Mazidjatari
80fecfa228 Tier1: create simple depth manager class
Will be used to manage depths of nested arrays
2024-04-05 18:35:06 +02:00
Kawe Mazidjatari
25e5bb667d Tier0: add new thread tools
- Add ThreadInMainOrServerFrameThread()
- Add ThreadCouldDoServerWork()
- Add ThreadJoinServerJob()
- Properly implement ThreadInServerFrameThread()
- Export all these functions for external DLL usage (plugins, etc)
2024-04-05 18:35:06 +02:00
Kawe Mazidjatari
fa421f259c Tier0: JobThreads improvements
- Add JobUserData_s for JT functions (used as arguments for JT callbacks).
- Add JT_GetCurrentJob().
- Add JT_WaitForJobAndOnlyHelpWithJobTypes().
2024-04-05 18:35:05 +02:00
Kawe Mazidjatari
1b039d52be VScript: remove ScriptLanguage_t
Doesn't exist in r5 and its unused in the SDK. The game only supports squirrel.
2024-04-05 18:35:05 +02:00
Kawe Mazidjatari
b1b80d3be0 Tier0: fix DEBUG compile error
Only occurred in DEBUG as Assert otherwise gets compiled out, and CThreadSpinRWLock::m_iWriteDepth only exists when compiled with REENTRANT_THREAD_SPIN_RW_LOCK. The game does not feature this so this is not defined. Moved to the REENTRANT_THREAD_SPIN_RW_LOCK block to only check on this assert when this feature is enabled.
2024-04-05 18:33:09 +02:00
Kawe Mazidjatari
d7478d3228 Tier2: WebSocket system improvements
Naming improvements, additional new comments/documentation and a few assertions. No logic has been changed.
2024-04-05 18:33:09 +02:00
Kawe Mazidjatari
e02f8f36fe Tier2: create modular WebSocket system
This code was actually part of the LiveAPI system, but there were many opportunities to make this particular code modular, so it has been decoupled and moved to Tier2. The LiveAPI system will soon use this class instead. The implementation has also been improved by adding dedicated routines for updating socket parameters, disconnecting/reconnecting and destroying sockets.

This commit also removes legacy workaround code in UtlVector which was used for before we had early enough access to the game's memalloc singleton. This code was no longer used.

This commit also implements the CUtlStringList class, which is now used for the new websocket class to split each socket connection up by a comma delimiter.
2024-04-05 18:31:58 +02:00
Kawe Mazidjatari
9d7eb7b07b Pylon: rename field "playerCount" to "numPlayers"
Force consistency between code structure.
2024-04-05 18:28:26 +02:00
Kawe Mazidjatari
8c593f666f Miles: add global type and prevent non-existent banks from loading
This patch adds miles globals and additional code to prevent banks from loading that don't exist. It falls back to "english" if requested language isn't installed.
2024-04-05 18:28:26 +02:00
Kawe Mazidjatari
f67d8e87b1 Tier1: use template index type for memory allocation
For CUtl* classes: use the index type rather than just ssize_t, this will throw compile warnings for code that constructs a CUtl* object allocating more memory than the index type allows (e.g. allocating UINT16_MAX while the index type has been set to UINT8_MAX).
2024-04-05 18:27:09 +02:00
Kawe Mazidjatari
582ec3791e Tier1: fully implement CUtlSymbol(*)
The class has been modified to match the implementation of the engine, the only modifications done were changing size types, so they compile to the correct size based on the platform (in case of the GameSDK project, this will be 64 bits).
2024-04-05 18:26:04 +02:00
Kawe Mazidjatari
615598c1b9 Tier0: implement CThreadSpinRWLock
Implementation matches that of the game executable, this will be used for CUtlStringTableMT.
2024-04-05 18:26:04 +02:00
Kawe Mazidjatari
44d663bf1c ImGui: cleanup and improvement to surface implementation
* Properly run fade in-out animations
* Make common methods/members shared
* Remove extraneous SDK thread
2024-04-05 18:22:56 +02:00
Kawe Mazidjatari
23a3c8fc00 Engine: make task queue singleton static
Compile it as static object & renamed to more correct names. Also made the integral type for frame delay an unsigned int instead
2024-04-05 18:19:39 +02:00
Kawe Mazidjatari
6cf88dc16c Engine: render ImGui in main thread and fix many threading bugs
ImGui drawing code now takes place in the main thread, a snapshot of the render data is created in CMaterialSystem::SwapBuffers(), and is being rendered in the render thread right before SpinPresent().

The reason why this was necessary, is because ConVar::GetString() isn't thread safe if its not marked FCVAR_MATERIAL_SYSTEM_THREAD or FCVAR_ACCESSIBLE_FROM_THREADS, and we used it for the console suggestions window, which iterates over every ConVar, accessible from threads or not.
2024-04-05 18:17:12 +02:00
Kawe Mazidjatari
a89ab882af Tier1: fix ConCommand::Dispatch()
Fix incorrect function prototype and add additional logic implemented in the engine. The code was crashing as the first parameter is the command target, but appears unused, the second parameter was actually the parameter that contained a pointer to the CCommand object which we needed.
2024-04-05 18:13:32 +02:00
Kawe Mazidjatari
609d705a0c Tier1: static construction of ConVar objects during link time
Fully implemented ConVar class so we could statically construct all SDK convars, this avoids a level of indirection, and allows for creating ConVar's everywhere in the project.

This patch also removed the settings tab of the ImGui server browser, as it has threading issues, while it technically never caused a crash yet, it has been removed as there was no point keeping it vs the work required to make it thread save (it only managed 2 convars which are perfectly manageable through cfg's or the in-game console).

Also temporarily disabled the creation of ConVar's in the mod system due to a memory leak, we would allocate and register a convar based on details parsed out of a mod file definition, but never unregister and free it.
2024-04-05 18:13:32 +02:00
Kawe Mazidjatari
d6b14fa12c Statically construct all ConCommand
Properly implement the ConCommandBase and ConCommand classes so we could statically construct all ConCommand objects in the global scope of each translation unit, this way we don't need to put them in a global file and deal with preprocessor directives to compile then in/out for certain projects.
2024-04-05 18:13:31 +02:00
Kawe Mazidjatari
6e7bfab4bd RTech: major cleanup of pak globals and improvement
Add missing types to fully mapped out PakGlobals_s struct, which is a 13MiB+ structure! This covers the vast majority of the pakfile system which is why we had to change a bunch of stuff for this patch. This patch also comes with:

- Reversed 'JobFifoLock_s' structure
- Reversed 'PakTracker_s' structure
- Reversed 'PakAssetTracker_s' structure

Many globals have been dropped as they were covered by the large PakGlobals_s singleton.

The pak decoder logic has been changed up as well, we now use a decode mode enumerant which will make it easier to add in more decoders for the pak files in the future.
2024-04-05 18:13:31 +02:00
Kawe Mazidjatari
2baa80d253 Engine: improve CClient::Authenticate performance
- Removed 2 std::string copy constructions
- Removed 32 sprintf calls per token auth request.
- Fixed a bug where we format the NucleusID as s64 instead of u64.
- Added additional hardening for when token/sessionId stitching fails, this will now always reject the connection.
- Improved the macro to make sure we always free the JWT claims if it has been allocated.
2024-04-05 18:07:38 +02:00
Kawe Mazidjatari
74da5c7c20 Engine: cleanup data block sender/receiver and add types
* Move member vars to correct place, to match it with the engine.
* Added out-of-band network message ID's for the R5 engine.

Also implemented 'ServerDataBlockSender::SendDataBlock()' and 'ClientDataBlockReceiver::AcknowledgeTransmission()'. NOTE that these are currently not tested, and also not in use! The code uses the version stored in the vftable which is what the engine itself provides. These have been implemented for reference only. If they need to be used, they need to be thoroughly tested first!
2024-04-05 18:06:36 +02:00
Kawe Mazidjatari
99af2877a6 Tier1: fully implement bitbuf classes
All routines are now fully implemented. This was delayed on purpose as some bit buffer functions have changed due to the use of 64bit integers for sizes, and also the coord types. The porting of this has to be done carefully; all
reimplemented functions have been changed to feature 64bit integers (where necessary) for syze types, and all values in coordsize.h have been tweaked to reflect the changes in the R5 engine 1:1, allowing us to properly implement the coord bit buffer functions as well.
2024-04-05 18:02:39 +02:00
Kawe Mazidjatari
48a12e17b6 Tier1: assert on LZSS window size being a power of 2 2024-04-05 18:02:39 +02:00
Kawe Mazidjatari
12485342ae Tier1: optimize V_GetFileExtension()
If we are going to do a strlen to do a reverse check, then just check from the start and avoid another loop.
2024-04-05 18:02:39 +02:00
Kawe Mazidjatari
2bed5d4437 Tier1: add CUtlHashTable 2024-04-05 18:01:23 +02:00
Kawe Mazidjatari
f18a4c17a4 Engine: rebuild CClientState::ProcessCreateStringTable()
Rebuild CClientState::ProcessCreateStringTable() and add notes for multiple 'potential' exploit vectors that are currently in place. Also add wrappers for compressing/decompressing net packets.
2024-04-05 17:59:53 +02:00
Kawe Mazidjatari
ee64d07e5b Tier0: emit text to debug console if debugger is attached
Small quality of life improvement while debugging the application.
2024-04-05 17:59:53 +02:00
Kawe Mazidjatari
0d390955fd Engine: partial rebuild for data block sender/receiver
This patch partially rebuilds the data block sender/receiver. The receiver leaks memory if the sender sends a bogus LZ4 packet, it would allocate memory to copy the encoded data into, from which it would decode it to the scratch buffer, but it would never deallocate this temporary buffer is the LZ4 decoder failed. This has been fixed. The second reason to rebuild these was to look into potential compression optimization. The data block rebuild now also features the latest LZ4 codec.
2024-04-05 17:59:53 +02:00
Kawe Mazidjatari
373b18ddb8 Tier1: use signed size types for buffer size field 2024-04-05 17:56:54 +02:00
Kawe Mazidjatari
20dd5783a3 Tier1: use correct format specifier CCountedStringPoolBase 2024-04-05 17:56:54 +02:00
Kawe Mazidjatari
b883cd6807 Tier1: make utlbuffer flags clear
Use 'FLAGS_NONE' as default.
2024-04-05 17:56:54 +02:00
Kawe Mazidjatari
c42ebe2cfc Tier1: use correct index types for template classes
This ensures that we can never allocate more memory than the maximum value that type could hold, without triggering a compile warning. There was a mention about shorts being slow on PowerPC platforms for utllinkedlist, but the index local type could be overridden (defaults to index storage type). In case it gets used for PowerPC, use 32bit ints as index local types; as the local type gets used for arithmetic operations while storage type is what's getting cached off in memory. This patch had no effect on generated output code, it was mostly implemented to avoid future problems.
2024-04-05 17:56:54 +02:00
Kawe Mazidjatari
b27769fdb5 Tier1: use the Max defined in platform headers
This one is the preferred Max as per Valve Source.
2024-04-05 17:56:54 +02:00
Kawe Mazidjatari
1378a5e5f4 Tier1: use SSIZE_MAX for default str slice buf end
Will be INT_MAX on 32bit platform and INT64_MAX on 64bit.
2024-04-05 17:56:54 +02:00
Kawe Mazidjatari
e59c6c88c9 Tier1: add V_IsAllDigit()
This function checks if a string consists of only numeric characters.
2024-04-05 17:56:52 +02:00
Kawe Mazidjatari
6434b8abe9 Tier0: crash handler refactor & bug fix
Fix syntax error on page protection flags check when determining whether we encountered a return address. Also limit the amount of dynamic memory allocation calls, offer a large enough static buffer where we can just write into when we crash without needing to alloc or realloc anything. Also removed some hacky code like the whitelist set, and made the module handle array static into the class as well. The CrashHandler singleton is now a static class instance.
2024-04-05 17:56:51 +02:00
Kawe Mazidjatari
a2ceab52da Tier1: fix type demotion warning
Promote type to signed size type.
2024-04-05 17:56:51 +02:00
Kawe Mazidjatari
eae0d0964b Tier0: iostream light cleanup
- Take const char* directly instead of making extraneous copies
- Add overload for reading strings into fixed size buffers
2024-04-05 17:56:51 +02:00
Kawe Mazidjatari
79cee02a62 Tier0: bump sigcache minor version
Force rebuild of local cache since many signatures have changed.
2024-04-05 17:56:50 +02:00
Kawe Mazidjatari
6a60a46fc0 RTech: make PakPatchDataHeader_t fields unsigned and add static assert for struct size 2024-04-05 17:56:50 +02:00
Kawe Mazidjatari
f2e49c8a4a IOStream: improve seeking logic
Instead of seeking to the very start again when GetSize() is called, seek back to the original position. Also added dedicated methods for seeking put or get instead of using the mode since this is a fstream
2024-04-05 17:56:49 +02:00
Kawe Mazidjatari
f5a8ca3e7a RTech: fix numeric overflow caused by massive pak files
Pak files above 4GB caused the async file system to throw an error about failing to load audio banks, removed casts and promoted everything correctly. Also removed irrelevant comments in the public pakfile header.
2024-04-05 17:56:49 +02:00
Kawe Mazidjatari
4d74dc5052 RTech: fix bug when trying to print fourcc as string
The printf specifier does not allow limiting buffer reads. Made the FourCCToString more performant by using a fixed size stack array and creating the fourcc into that which we now use to properly print out the asset magic.
2024-04-05 17:56:49 +02:00
Kawe Mazidjatari
082a8e4ef7 RTech: minor cleanup and renaming 2024-04-05 17:56:49 +02:00
Kawe Mazidjatari
05b51fbd8c RTech: update comment on pak header flag
Patches can actually use different compression algorithms, but the patch headers must accommodate for pak file sizes properly. One pak could also be compressed while the others aren't if patch headers present correct sizes (both RTech and ZStream)
2024-04-05 17:56:49 +02:00
Kawe Mazidjatari
22431e903c FileSystem: make CreateDirHierarchy and IsDirectory general function
Moved to general utility
2024-04-05 17:56:48 +02:00
Kawe Mazidjatari
1c2b02b972 RTech: cleanup ZStd pak decoder and fix bugs
* Pak_InitDecoder() now takes the output buf and mask as parameters.
* Pak_InitDecoder() checks if provided masks are a power of 2 (required).
* Pak_ComputeRingBufferFrame() now uses the bit mask instead of modulo to determine # bytes used.
* Fixed a bug where PakDecoder_t::bufferSizeNeeded could be bigger than the file stream, causing a deadlock.
2024-04-05 17:56:48 +02:00
Kawe Mazidjatari
2d6533adf1 RTech: add info about small pak file's ring buffers 2024-04-05 17:56:48 +02:00
Kawe Mazidjatari
e1c889719a RTech: properly implement ring buffer stream decode for Zstd decoder
The new decoder can now decode patched and custom rpaks files, all issues so far have been fixed. The code still needs some testing and bench marking to make sure
2024-04-05 17:56:48 +02:00