125 Commits

Author SHA1 Message Date
Kawe Mazidjatari
99edd0229d Localize: add language codes
Newly added g_LanguageCodes maps directly with g_LanguageNames.
2024-06-01 11:52:53 +02:00
Kawe Mazidjatari
84abfced6a VpkLib: make regex available to other code and utilize those instead
Prevent copies or similar patterns that might not work as good.
2024-06-01 11:49:16 +02:00
Kawe Mazidjatari
299ab763e4 VpkLib: light cleanup
Improve global names, initialize VPKDirHeader_t structure, make inlines for checking if language or target exists.
2024-06-01 11:49:15 +02:00
Kawe Mazidjatari
241aefd6f3 VpkLib: actually search for corresponding data block file
Instead of just using English as default, actually scan for all supported directory tree files, error if not found. This completes the sanitization logic which allows the user to only pass in the data block file to load the directory tree file
2024-04-05 17:56:50 +02:00
Kawe Mazidjatari
a1ba022fef VpkLib: fix bug when parsing directory tree base name
After the regex update in commit 5266a2e9, this function had to be updated as well since the underscore is no longer returned.
2024-04-05 17:46:56 +02:00
Kawe Mazidjatari
5f09f1434c VpkLib: simplify tree header writer
Make sure the VPK header is always initialized to 0 instead of actual values, as otherwise it would cause the reader to succeed on truncated files if at least the first byte matches that of the VPK magic. This also lead to the simplification of the tree header writer; instead of writing the tree size apart from the header, write the whole thing at once.
2024-04-05 17:46:55 +02:00
Kawe Mazidjatari
bf59f6c2c5 ReVPK: bug fixes & improvements
* Make sure the workspace path actually exists before attempting to pack it.
* Make sure the VPK directory tree file was parsed correctly before unpacking store.
* Log debug output for each pack operation to a file.
* Fix bug in s_DirFileRegex regex pattern, which would include a trailing '_' in the context part of the directory tree file name.
* Fix bug in 'GetLevelName()' and 'GetDirNameParts()' causing it to parse the path as well, prune the path before running the regex.
* Renamed 'GetLevelName()' to 'PackedStore_GetDirLevelName()'.
* Renamed 'GetDirNameParts()' to 'PackedStore_GetDirNameParts()'.
* Write a front-end enable file when building client VPK's.
2024-04-05 17:46:54 +02:00
Kawe Mazidjatari
8ff1404c4d VpkLib: move header check to dir construction and flag error on failure
Moved the construction as otherwise we would iterate over potentially bad or incompatible data.
2024-04-05 17:46:54 +02:00
Kawe Mazidjatari
6f0fdd56ce VpkLib: add comments to name sanitizer
Make it clear what it does.
2024-04-05 17:46:54 +02:00
Kawe Mazidjatari
7304c3782d VpkLib: fix bug in dir tree file name sanitizer
The sanitizer never worked as we never replaced the corresponding string.
2024-04-05 17:46:54 +02:00
Kawe Mazidjatari
465d905056 VpkLib: fix bug in GetManifest()
Check if the parsing was actually successful, else free and return nullptr. The null check at call site never worked since we never return null on failure.
2024-04-05 17:46:53 +02:00
Kawe Mazidjatari
5f554867a1 VpkLib: check for manifest file first before opening the pack file
We do this as if we happen to not have any manifest whatsoever, but already opened the block file, and the block file already existed and was still valid, it would be emptied out. Only open it if we are actually going to write into it!
2024-04-05 17:46:53 +02:00
Kawe Mazidjatari
e6ba904f96 VokLib: rename pack/unpack methods
More correct names.
2024-04-05 17:46:53 +02:00
Kawe Mazidjatari
28cc532e9c ReVPK: add option for num helper threads and compress levels
Allow the user to set the # amount of helper threads and the compress level of the VPK's.
2024-04-05 17:46:53 +02:00
Kawe Mazidjatari
2b0837fa93 VpkLib: allow directory recursion
This is offline tools code, no security risks here.
2024-04-05 17:46:53 +02:00
Kawe Mazidjatari
9b3dd97792 VpkLib: rename CPackedStore to CPackedStoreBuilder
Avoid confusion with the game's CPackedStore class.
2024-04-05 17:46:52 +02:00
Kawe Mazidjatari
6841f5bb19 VpkLib: cleanup packed store class
Make the utility methods statics within the implementation so it could be used by other structs/classes of the VPK builder as well, these utility functions weren't useful outside the class. This also allowed for the removal of the singleton which had no purpose whatsoever.
2024-04-05 17:46:52 +02:00
Kawe Mazidjatari
ee82d5d8e0 Tier1: move KeyValues class to Tier1
The KeyValues class belongs here. Also reimplemented most loading methods for KeyValues, and adjusted the VPK building code to account for it. Pointers to the engine's implementation of KeyValues have been moved to a separate header ('keyvalues_iface.h'), as this allows external tools code to utilize the standalone KeyValues class implementation. Playlist utilities are completely separated from the KeyValues header; these have nothing to do with KeyValues other than manipulating a global KeyValues object for the playlists, and thus have been named as such and moved to rtech/playlists.
2024-04-05 17:42:05 +02:00
Amos
0634261c18 Move global language constants to single file
Deduplicate.
2023-09-15 18:25:31 +02:00
Kawe Mazidjatari
4b7ca6b439 Use size types
Make code more portable and modern.
2023-08-29 22:05:20 +02:00
Kawe Mazidjatari
71b0781715 Utilize 'DevMsg()' for uncertain builds only
Only uncertain builds will contain DevMsg()/DevWarning() prints. For retail, Msg() and Warning() should be used instead.
2023-08-21 19:12:29 +02:00
Kawe Mazidjatari
5f64cd83d8 Fix FileSystem size integer truncation
The implementation in the engine always took/returned signed 64bit size types, but I made a mistake when reversing the virtual function table. All types have been changed to what they should be (mostly signed 64bit, very few unsigned), and usage in-SDK has been adjusted accordingly.
2023-08-09 14:43:54 +02:00
Kawe Mazidjatari
9e50266649 Update format specifier
Use %lu.
2023-07-22 14:42:44 +02:00
Kawe Mazidjatari
ac6becffd8 Adjust types and format specifiers
Make it more correct.
2023-07-15 16:25:35 +02:00
Kawe Mazidjatari
cb2850f938 Update packedstore.cpp 2023-07-10 01:38:29 +02:00
Kawe Mazidjatari
2565ff44ef Light VPK code optimizations
Cache string buffer ptr.
2023-07-08 02:18:18 +02:00
Kawe Mazidjatari
c215fcc171 VPK utility code improvements
- Only process referenced pack files, previously, if an entry in the VPK file was build into a pack of index 128, the code would generate a list of pack files from 0 to 128.
- Added documentation to VPK structure.
2023-07-06 00:49:17 +02:00
Kawe Mazidjatari
5939e05331 Fix incorrect VPK tree during build
This commit fixes the following bugs:
- Use of STL map on CUtlString, CUtlString does not have a lesser operator, thus it always fails to check whether or not the string is already present in the map. The tree structure cannot contain duplicates per tree scope.
- Use of backward slashes in vpk file tree structure, path separators should always be a forwards slash. The code was originally designed to be forwards slash only, but some paths were adjusted to take backwards slash, as the KeyValues class treats the '/' char as a delimiter for subfields. This has therefore lead into the whole code changing to it. The code has now been changed to the point where only the KeyValues utilities take the backward slash paths.
- Trailing slash character on file path, that ended up getting written as-is in the path tree.

The VPK tree builder is now part of a separate nested class in 'VPKDir_t'.
2023-07-05 23:04:07 +02:00
Kawe Mazidjatari
c6e75967bf CPackedStore: light code cleanup
Light code cleanup: use references instead of pointers as much as possible, additional use of const qualifiers, and usage of provided iteration macros for CUtlVector.
2023-07-04 22:08:44 +02:00
Kawe Mazidjatari
eed83f8815 Always make sure a slash gets appended to the filepath
On the game's filesystem, the slash always appeared, when running this code on a reimplemented filesystem, no slash was found at the end of the path. This change ensures there is always a slash.
2023-07-04 21:39:31 +02:00
Kawe Mazidjatari
56da0576a6 Make static to source file
Should be made static to avoid unwanted bloat in compiled output, as each translation unit will create a new initializer/destructor. -10KiB compiled code for GameSDK.
2023-07-03 02:27:09 +02:00
Kawe Mazidjatari
ef69611435 Replace memalloc calls throughout entire SDK
Global 'direct' usage of 'MemAllocSingleton()' has been jettisoned. Where possible, smart pointers were used instead. During the refactor, the following bugs were addressed and fixed:
- The virtual destructor of 'CCVarIteratorInternal' was NOT called on destruction.
- Class function 'KeyValues::MakeCopy' did NOT calculate the buffer size of the wide string correctly, the original calculation was 'len+1*sizeof(wchar_t)', but should've been '(len+1)*sizeof(wchar_t)'.

Some other code changes include:
- Tier0 include 'memstd.h' has been moved above all thirdparty includes, to make sure the memalloc functions get shadowed with ours in third party libraries as well.
- RPak file paths string literals are now defines.
- 'DestroyOverlay' has been refactored to match the assembly of the game.
2023-06-26 22:34:24 +02:00
Kawe Mazidjatari
3f8baf6f68 Allow setting compile options for specific targets
Previously, it was all controlled from the global init (applied to all projects), but some projects need different options. With these changes, you can disable the common options applied in the 'add_module' macro, and set your own if desired.
2023-06-17 00:45:27 +02:00
Kawe Mazidjatari
d77dbdafb9 Remove extraneous subtraction from array size constant
The subtraction is not needed.
2023-06-01 00:08:22 +02:00
Kawe Mazidjatari
abef34d3ef CPackedStore refactor
* Improve naming convention, and make it more consistent.
* Use engine types as much as possible.
* Reduced the amount of string copy constructions.
2023-05-29 21:41:17 +02:00
Kawe Mazidjatari
0fe6af008c Fix data truncation on VPK unpack
If the chunk if uncompressed, continue the loop instead of breaking out of it. Only 2 files were affected by this bug from original VPK's: 'notosansjp-regular.vfont' and 'notosanstc-regular.vfont'.
2023-05-29 15:56:59 +02:00
Kawe Mazidjatari
9f44bcc818 Treat compiler warnings as errors
Treat them as errors globally. Most of the time a warning is a bug, or problem in code that could be solved in a different (better) manner. Thirdparty code have this disabled. The warnings as errors option can be globally disabled through the CMake GUI, but this is not recommended.
2023-05-15 09:44:26 +02:00
Kawe Mazidjatari
8dbc2024c6 CMake code improvements
Use the 'add_module' macro to add modules without creating duplicate code. This macro also takes a reuse PCH as parameter, so modules that need a precompiled header, could reuse those from different targets that compile them. This commit also restructures the group order of the generated solution files for easier code navigation.
2023-05-13 00:14:53 +02:00
Kawe Mazidjatari
f120354e96 Initial port to CMake
* All libraries have been isolated from each other, and build into separate artifacts.
* Project has been restructured to support isolating libraries.
* CCrashHandler now calls a callback on crash (setup from core/dllmain.cpp, this can be setup in any way for any project. This callback is getting called when the apllication crashes. Useful for flushing buffers before closing handles to logging files for example).
* Tier0 'CoreMsgV' function now calls a callback sink, which could be set by the user (currently setup to the SDK's internal logger in core/dllmain.cpp).

TODO:
* Add a batch file to autogenerate all projects.
* Add support for dedicated server.
* Add support for client dll.

Bugs:
* Game crashes on the title screen after the UI script compiler has finished (root cause unknown).
* Curl error messages are getting logged twice for the dedicated server due to the removal of all "DEDICATED" preprocessor directives to support isolating projects. This has to be fixed properly!
2023-05-10 00:05:38 +02:00
Kawe Mazidjatari
69775ca234 CPackedStore: Improve symbol naming 2023-04-05 20:09:23 +02:00
Kawe Mazidjatari
697d11cccb Improve VPKPair_t constructor
* Take string object by const reference.
* Emit warnings when locale or target specified by user isn't supported.
2023-04-04 00:41:53 +02:00
Kawe Mazidjatari
6da5206282 CPackedStore: Fix file handle bug
VPK directory file handle was never closed properly in 'VPKDir_t::Init()';
2023-04-03 22:48:29 +02:00
Kawe Mazidjatari
1f1a487af9 CPackedStore: Fix bug and improve code readability
Moved deduplication code to a separate function. Unnested packing and unpacking code a bit more.
2023-04-03 22:46:10 +02:00
Kawe Mazidjatari
081c548bdd Remove unnecessary member from 'VPKChunkDescriptor_t' 2023-04-03 21:16:15 +02:00
Kawe Mazidjatari
dc0fba9319 CPackedStore: Rename 'useDataSharing' to 'deDuplicate'
More descriptive in what it does (deduplication of existing copies).
2023-04-03 20:08:36 +02:00
Kawe Mazidjatari
b5bd4296e9 CPackedStore: Dynamically allocate source/destination buffers
This allows for multithreading the VPK build process.
2023-04-03 20:02:56 +02:00
Kawe Mazidjatari
475262fc24 Use C-style format specifiers in CPakFile
Enforce consistency within codebase.
2023-04-03 19:12:15 +02:00
Kawe Mazidjatari
417365fa77 /W4: Fix type truncation warnings 2023-04-02 17:39:16 +02:00
Kawe Mazidjatari
e1f0e72af4 /W4: Fix type demotion warnings
Explicitly demote to 'int' to suppress pesky compiler warnings.
2023-04-02 16:03:01 +02:00
Kawe Mazidjatari
cdb03a5d97 Use standalone filesystem functions instead
The member variants do not exist in experimental filesystem implementations.
2023-03-20 00:19:17 +01:00