91 Commits

Author SHA1 Message Date
Kawe Mazidjatari
abef34d3ef CPackedStore refactor
* Improve naming convention, and make it more consistent.
* Use engine types as much as possible.
* Reduced the amount of string copy constructions.
2023-05-29 21:41:17 +02:00
Kawe Mazidjatari
0fe6af008c Fix data truncation on VPK unpack
If the chunk if uncompressed, continue the loop instead of breaking out of it. Only 2 files were affected by this bug from original VPK's: 'notosansjp-regular.vfont' and 'notosanstc-regular.vfont'.
2023-05-29 15:56:59 +02:00
Kawe Mazidjatari
9f44bcc818 Treat compiler warnings as errors
Treat them as errors globally. Most of the time a warning is a bug, or problem in code that could be solved in a different (better) manner. Thirdparty code have this disabled. The warnings as errors option can be globally disabled through the CMake GUI, but this is not recommended.
2023-05-15 09:44:26 +02:00
Kawe Mazidjatari
8dbc2024c6 CMake code improvements
Use the 'add_module' macro to add modules without creating duplicate code. This macro also takes a reuse PCH as parameter, so modules that need a precompiled header, could reuse those from different targets that compile them. This commit also restructures the group order of the generated solution files for easier code navigation.
2023-05-13 00:14:53 +02:00
Kawe Mazidjatari
f120354e96 Initial port to CMake
* All libraries have been isolated from each other, and build into separate artifacts.
* Project has been restructured to support isolating libraries.
* CCrashHandler now calls a callback on crash (setup from core/dllmain.cpp, this can be setup in any way for any project. This callback is getting called when the apllication crashes. Useful for flushing buffers before closing handles to logging files for example).
* Tier0 'CoreMsgV' function now calls a callback sink, which could be set by the user (currently setup to the SDK's internal logger in core/dllmain.cpp).

TODO:
* Add a batch file to autogenerate all projects.
* Add support for dedicated server.
* Add support for client dll.

Bugs:
* Game crashes on the title screen after the UI script compiler has finished (root cause unknown).
* Curl error messages are getting logged twice for the dedicated server due to the removal of all "DEDICATED" preprocessor directives to support isolating projects. This has to be fixed properly!
2023-05-10 00:05:38 +02:00
Kawe Mazidjatari
69775ca234 CPackedStore: Improve symbol naming 2023-04-05 20:09:23 +02:00
Kawe Mazidjatari
697d11cccb Improve VPKPair_t constructor
* Take string object by const reference.
* Emit warnings when locale or target specified by user isn't supported.
2023-04-04 00:41:53 +02:00
Kawe Mazidjatari
6da5206282 CPackedStore: Fix file handle bug
VPK directory file handle was never closed properly in 'VPKDir_t::Init()';
2023-04-03 22:48:29 +02:00
Kawe Mazidjatari
1f1a487af9 CPackedStore: Fix bug and improve code readability
Moved deduplication code to a separate function. Unnested packing and unpacking code a bit more.
2023-04-03 22:46:10 +02:00
Kawe Mazidjatari
081c548bdd Remove unnecessary member from 'VPKChunkDescriptor_t' 2023-04-03 21:16:15 +02:00
Kawe Mazidjatari
dc0fba9319 CPackedStore: Rename 'useDataSharing' to 'deDuplicate'
More descriptive in what it does (deduplication of existing copies).
2023-04-03 20:08:36 +02:00
Kawe Mazidjatari
b5bd4296e9 CPackedStore: Dynamically allocate source/destination buffers
This allows for multithreading the VPK build process.
2023-04-03 20:02:56 +02:00
Kawe Mazidjatari
475262fc24 Use C-style format specifiers in CPakFile
Enforce consistency within codebase.
2023-04-03 19:12:15 +02:00
Kawe Mazidjatari
417365fa77 /W4: Fix type truncation warnings 2023-04-02 17:39:16 +02:00
Kawe Mazidjatari
e1f0e72af4 /W4: Fix type demotion warnings
Explicitly demote to 'int' to suppress pesky compiler warnings.
2023-04-02 16:03:01 +02:00
Kawe Mazidjatari
cdb03a5d97 Use standalone filesystem functions instead
The member variants do not exist in experimental filesystem implementations.
2023-03-20 00:19:17 +01:00
Kawe Mazidjatari
5806f719c6 Should be u8string 2023-02-12 23:32:58 +01:00
Kawe Mazidjatari
4d68a2edbe Fix deadlock when file entry is truncated
* Fixed endless loop when condition is hit.
* Fixed file name not being formatted correctly.
2023-02-12 23:28:24 +01:00
Kawe Mazidjatari
60ab35b5c4 Light optimizations for string utils
* Changed 'CreateDirectories' to only copy sanitized path if pointer to string is passed.
* Changed 'IsValidBase64' to only tokenize base64 value if pointer to string is passed.
2023-02-04 15:59:45 +01:00
Kawe Mazidjatari
c49209bd1e Log truncated files if they are listed in the VPK build manifest 2023-01-26 03:00:53 +01:00
Kawe Mazidjatari
29b0bc389e Light CPackedStore cleanup 2022-12-04 14:17:14 +01:00
Kawe Mazidjatari
55b90781ee Remove experimental CRC implementation in LZHAM
This implementation was done to calculate buffer CRC on the go, as that was what the VPK file system uses for the file crc's. But it later turned out to be unnecessary, and was never reverted.
2022-12-04 01:33:10 +01:00
Kawe Mazidjatari
b8b79385e1 Open ignore file in text mode 2022-11-24 11:48:23 +01:00
Kawe Mazidjatari
368c4218b4 Move some CPackedStore methods to VPKDir_t
These methods where only used for and by the VPKDir_t structure.
Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
Kawe Mazidjatari
2e9aa9a77c Avoid string copy in 'VPKEntryBlock_t::VPKEntryBlock_t'
Avoid a copy when formatting string to that of the VPK directory tree structure.
2022-11-23 14:49:40 +01:00
Kawe Mazidjatari
2bc570800b Split 'VPKDir_t::Build' into multiple methods
Each process of writing out a VPK directory is now within its own method.
2022-11-23 12:27:57 +01:00
Kawe Mazidjatari
0ab31606eb CPackedStore refactor
The CPackedStore class now heavily relies on the engine's FileSystem and KeyValues API. Since its part of the engine, it would make more sense to utilize the engine's features instead. It also allows for easier modifications.
* The logic behind the build process hasn't changed.
2022-11-23 12:18:33 +01:00
Kawe Mazidjatari
8c7b58b9df Add convar for adjusting LZHAM compressor max helper threads 2022-11-17 20:37:12 +01:00
Kawe Mazidjatari
34d8ea578a Improve consistency throughout packedstore class
No modifications to the existing logic have been performed. Some parameter names have been renamed to improve code readability and maintainability.
2022-11-16 00:54:51 +01:00
Kawe Mazidjatari
f16d538aee CPackedStore: light optimizations
* Don't call copy constructor on entry blocks when building the directory file.
* Don't call copy constructor for string when we don't want to sanitize the directory file name.
2022-10-12 23:39:26 +02:00
Amos
780b968bc7 CPackedStore::GetDirectoryFile optimizations
Break when we have a locale, or found the context.
Removed extraneous logic/vars.
2022-10-11 16:31:00 +02:00
Amos
142c0d1de5 CPackedStore: fix directory file name sanitizer
The sanitizer can retrieve the directory file name from the block file name if this is passed instead. This was broken in the previous system. Sanitization can be opted out if the 3rd parameter of the 'fs_vpk_unpack' command is omitted.
2022-10-11 15:37:34 +02:00
Kawe Mazidjatari
99a33e95f1 Additional VPK lib cleanup and performance improvements 2022-10-09 12:08:54 +02:00
Kawe Mazidjatari
2d7b06f0a0 More VPK compressing/decompressing optimizations
* Use reference to array index.
* Only obtain array size once during loop.
* Use static buffer of 1024*1024 for decompressing.
* Removed extraneous goto.
* Fixed bug where shared data would be mapped wrong due to the last optimization.
* This increased packing performance by 40/60 seconds when rebuilding all vpk's.
2022-10-02 19:24:00 +02:00
Kawe Mazidjatari
0451a75f85 Use static buffer for file chunking and compressing during VPK packing
Avoid using new every time. Chunk can be at most 1024 * 1024.
2022-10-02 12:17:03 +02:00
Kawe Mazidjatari
43fdbc94da CPackedStore::PackAll optimizations
* Hash chunk before compression to determine if it already exists or not (don't run compression if it does).
* Use the result of vector::insert for data sharing (insert already checks if it exists or not, and returns the iterator).
* This optimization saves around 20 seconds total when rebuilding all VPK's.
2022-10-02 02:16:51 +02:00
Kawe Mazidjatari
729475c74c Light cleanup
Use NO_ERROR instead of NULL for error code parameter passed to Error(..).
2022-09-14 01:14:51 +02:00
Kawe Mazidjatari
3d6d6644bd Logging bug fix and error handling improvements
* Replaced the boolean 'fatal' parameter with a error code parameter, anything non-null will prompt a message (fatal) and terminate the process with given error code.
* Fixed bug where the global ostreamsink for spdlog did NOT get cleared in 'SQVM_PrintFunc' when cvar 'sq_showvmoutput' was < 3. Moved to global scope.
* Added error message for when detouring the process has failed, with the error code.
* Only call 'Plat_GetProcessUpTime()' once per log, (improves performance and fixes bug where the error message box would show a different time stamp than what is logged into the console or file).
* All TIER0 loggers only log to notify and console when the SDK engine has fully initialized and detoured all functions.
2022-09-14 00:39:38 +02:00
Kawe Mazidjatari
705b81a9e3 Set VPK compression level through ConVar 2022-09-11 00:16:31 +02:00
Kawe Mazidjatari
66f7e96413 CPackedStore: do not pack empty files
Empty files have no place in a VPK.
2022-09-10 00:07:42 +02:00
Kawe Mazidjatari
34a06147d7 Fix spelling errors
Overall spelling improvements and cleanup..
2022-09-09 19:47:31 +02:00
Kawe Mazidjatari
b6ff2c8798 Light cleanup 2022-09-09 02:28:03 +02:00
Kawe Mazidjatari
2010e11310 Add 'fatal' functionality to Error()
When fatal is set, the function will show a error dialogue, which will halt the process and close when the message is dismissed. Else we will end up in a crash.
2022-08-22 12:42:41 +02:00
Kawe Mazidjatari
329621c6ad Move all public headers into root of 'public' 2022-08-09 17:18:07 +02:00
Kawe Mazidjatari
e92b5d1300 General cleanup
Use c++ methods, remove unnecessary casts, unnecessary use of auto, etc..
2022-08-09 15:19:12 +02:00
Kawe Mazidjatari
d96bb2deb0 Use correct size type format specifier in packetstore.cpp
Using %llu is also correct, but this is the standard approach.
2022-08-08 17:51:09 +02:00
PixieCore
1cd0ae5c87 General code cleanup, Rebuild and hooked RTech::OpenFile
* RTech std namespace removal
* RTech::OpenFile Rebuild.
* FileHandleTracker_t fully reversed, grabbing m_FileHandles from memory now.
* RTech::FindFreeSlotInFiles needs more research.
*
2022-08-06 00:16:11 +02:00
Kawe Mazidjatari
dab5133e07 Rename 2022-06-08 23:35:46 +02:00
Amos
d909878624 Optimizations 2022-06-08 14:03:04 +02:00
Kawe Mazidjatari
9938c724a2 Cleanup 2022-06-07 13:57:29 +02:00