R5Reloaded/r5sdk
1
0
Fork 0
mirror of https://github.com/Mauler125/r5sdk.git synced 2025-02-09 19:15:03 +01:00
Code Issues Packages Projects Releases Wiki Activity
r5sdk/r5dev/vpklib/packedstore.h

170 lines
7.5 KiB
C
Raw Normal View History

CPackedStore: light optimizations * Don't call copy constructor on entry blocks when building the directory file. * Don't call copy constructor for string when we don't want to sanitize the directory file name.
2022-10-12 23:39:26 +02:00
#ifndef PACKEDSTORE_H
#define PACKEDSTORE_H
/*******************************************************************
* ██████╗ ██╗ ██╗ ██╗██████╗ ██╗ ██╗ ██╗ ██╗██████╗ *
* ██╔══██╗███║ ██║ ██║██╔══██╗██║ ██╔╝ ██║ ██║██╔══██╗ *
* ██████╔╝╚██║ ██║ ██║██████╔╝█████╔╝ ██║ ██║██████╔╝ *
* ██╔══██╗ ██║ ╚██╗ ██╔╝██╔═══╝ ██╔═██╗ ██║ ██║██╔══██╗ *
* ██║ ██║ ██║ ╚████╔╝ ██║ ██║ ██╗ ███████╗██║██████╔╝ *
* ╚═╝ ╚═╝ ╚═╝ ╚═══╝ ╚═╝ ╚═╝ ╚═╝ ╚══════╝╚═╝╚═════╝ *
*******************************************************************/
CPackedStore refactor The CPackedStore class now heavily relies on the engine's FileSystem and KeyValues API. Since its part of the engine, it would make more sense to utilize the engine's features instead. It also allows for easier modifications. * The logic behind the build process hasn't changed.
2022-11-23 12:18:33 +01:00
#include "public/ifilesystem.h"
Code base refactor + major performance and readability improvement. Read description for details. * Codebase restructured to SourceSDK codebase style and .cpp/.h assertion paths in the game executable. * Document most functions with valve style 'Purpose' blocks. * Rename variables to match the rest of the codebase and Valve's naming convention. * Dedicated DLL and the SDKLauncher now share the same codebase as the DevSDK. * Obtain globals or pointers directly instead of waiting for runtime initialized data. * Dynamically search for all functions and globals (this doesn't count for dedicated yet!). * Initialize most in-SDK variables. * Move certain prints and other utilities under ConVars to reduce verbosity and increase performance. * Print all pattern scan results through a virtual function to make it easier to add and debug new patterns in the future. * Type global var pointers appropriately if class or type is known and implemented. * Forward declare 'CClient' class to avoid having 2 'g_pClient' copies. * Add IDA's pseudo definitions for easier prototyping with decompiled assembly code. * RPAK decompress Command callback implementation. * Load decompressed RPaks from 'paks\Win32\' overriding the ones in 'paks\Win64\' (the decompress callback will automatically fix the header and write it to 'paks\Win32\'). * VPK decompress Command callback implementation. * Move CRC32 ands Adler32 to implementation files. * Server will print out more details about the connecting client. * Upgrade ImGui lib to v1.86. * Don't compile id3dx.h for dedicated. * Don't compile id3dx.cpp for dedicated * Implement DevMsg print function allowing to print information to the in-game VGUI/RUI console overlay, ImGui console overlay and the external windows console * Fixed bug where the Error function would not properly terminate the process when an error is called. This caused access violations for critical/non-recoverable errors. * Fixed bug where the game would crash if the console or server browser was enabled while the game was still starting up. * Several bug fixes for the dedicated server (warning: dedicated is still considered work-in-progress!).
2021-12-25 22:36:38 +01:00
#include "thirdparty/lzham/include/lzham.h"
Additional cleanup Cast to proper types. Set the archive offset only instead of the entire block. The entry could still have its own flags. The only thing needed for saving data is sharing the same offset and size.
2022-06-05 12:28:49 +02:00
constexpr unsigned int VPK_HEADER_MARKER = 0x55AA1234;
Proper VPK repacking Initial proper implementation pending cleanup. The new system builds a manifest file when a VPK is unpacked. The manifest files contains data such as the entry flags and texture flags. It also contains a field determining whether the file should be compressed or not. When a user repacks a pack, the system attempts to load this manifest file and does a lookup to the object to retrieve the flags (most of these flags are unknown, but they are used by the engine and are necessary for stuff like cubemaps and texture files to work correctly. Cubemaps won't work with proper flags, and textures (decals, particle system components, etc..) will look washed out without them. I think some also determine whether a file within the VPK should be cached or not, so simply marking everything as 0x101 will probably end up in more CPU time and higher filesystem cache usage (depot/ is only 0x1, I don't think anything there is getting cached ever without the 0x100 flag). User could also repack a VPK while excluding anything that is not in the manifest file. So you could unpack all VPK's into a single directory (each VPK has its own manifest file tied to its level name), and rebuild all the VPK's with only the files that where originally in them. fs_pack_vpk command usage: <locale> <context> <level_name> <manifest_only> locale determines the pak language (default english), context determines whether is a server/client vpk, level_name determines the BSP name of the pak, manifest_only determines whether the pack system should only include files within the manifest (leaving this arg out will build all files into the vpk). The VPK workspace path is determined with ConVar 'fs_packedstore_workspace'.
2022-06-04 01:08:23 +02:00
constexpr unsigned int VPK_MAJOR_VERSION = 2;
constexpr unsigned int VPK_MINOR_VERSION = 3;
Additional cleanup Cast to proper types. Set the archive offset only instead of the entire block. The entry could still have its own flags. The only thing needed for saving data is sharing the same offset and size.
2022-06-05 12:28:49 +02:00
constexpr unsigned int VPK_DICT_SIZE = 20;
constexpr int ENTRY_MAX_LEN = 1024 * 1024;
Add convar for adjusting LZHAM compressor max helper threads
2022-11-17 20:37:12 +01:00
constexpr int PACKFILEPATCH_MAX = 512;
CPackedStore refactor The CPackedStore class now heavily relies on the engine's FileSystem and KeyValues API. Since its part of the engine, it would make more sense to utilize the engine's features instead. It also allows for easier modifications. * The logic behind the build process hasn't changed.
2022-11-23 12:18:33 +01:00
constexpr int PACKFILEINDEX_SEP = 0x0;
Improve consistency throughout packedstore class No modifications to the existing logic have been performed. Some parameter names have been renamed to improve code readability and maintainability.
2022-11-16 00:54:51 +01:00
constexpr int PACKFILEINDEX_END = 0xffff;
Code base refactor + major performance and readability improvement. Read description for details. * Codebase restructured to SourceSDK codebase style and .cpp/.h assertion paths in the game executable. * Document most functions with valve style 'Purpose' blocks. * Rename variables to match the rest of the codebase and Valve's naming convention. * Dedicated DLL and the SDKLauncher now share the same codebase as the DevSDK. * Obtain globals or pointers directly instead of waiting for runtime initialized data. * Dynamically search for all functions and globals (this doesn't count for dedicated yet!). * Initialize most in-SDK variables. * Move certain prints and other utilities under ConVars to reduce verbosity and increase performance. * Print all pattern scan results through a virtual function to make it easier to add and debug new patterns in the future. * Type global var pointers appropriately if class or type is known and implemented. * Forward declare 'CClient' class to avoid having 2 'g_pClient' copies. * Add IDA's pseudo definitions for easier prototyping with decompiled assembly code. * RPAK decompress Command callback implementation. * Load decompressed RPaks from 'paks\Win32\' overriding the ones in 'paks\Win64\' (the decompress callback will automatically fix the header and write it to 'paks\Win32\'). * VPK decompress Command callback implementation. * Move CRC32 ands Adler32 to implementation files. * Server will print out more details about the connecting client. * Upgrade ImGui lib to v1.86. * Don't compile id3dx.h for dedicated. * Don't compile id3dx.cpp for dedicated * Implement DevMsg print function allowing to print information to the in-game VGUI/RUI console overlay, ImGui console overlay and the external windows console * Fixed bug where the Error function would not properly terminate the process when an error is called. This caused access violations for critical/non-recoverable errors. * Fixed bug where the game would crash if the console or server browser was enabled while the game was still starting up. * Several bug fixes for the dedicated server (warning: dedicated is still considered work-in-progress!).
2021-12-25 22:36:38 +01:00
CPackedStore: fix directory file name sanitizer The sanitizer can retrieve the directory file name from the block file name if this is passed instead. This was broken in the previous system. Sanitization can be opted out if the 3rd parameter of the 'fs_vpk_unpack' command is omitted.
2022-10-11 15:37:34 +02:00
static const std::regex BLOCK_REGEX{ R"(pak000_([0-9]{3}))" };
static const std::regex DIR_REGEX{ R"((?:.*\/)?([^_]*_)(.*)(.bsp.pak000_dir).*)" };
More VPK compressing/decompressing optimizations * Use reference to array index. * Only obtain array size once during loop. * Use static buffer of 1024*1024 for decompressing. * Removed extraneous goto. * Fixed bug where shared data would be mapped wrong due to the last optimization. * This increased packing performance by 40/60 seconds when rebuilding all vpk's.
2022-10-02 19:24:00 +02:00
Improve consistency throughout packedstore class No modifications to the existing logic have been performed. Some parameter names have been renamed to improve code readability and maintainability.
2022-11-16 00:54:51 +01:00
static const vector<string> DIR_TARGET =
Use static buffer for file chunking and compressing during VPK packing Avoid using new every time. Chunk can be at most 1024 * 1024.
2022-10-02 12:17:03 +02:00
{
"server",
"client"
};
static const vector<string> DIR_LOCALE =
{
"english",
"french",
"german",
"italian",
"japanese",
"korean",
"polish",
"portuguese",
"russian",
"spanish",
"tchinese"
};
Proper VPK repacking Initial proper implementation pending cleanup. The new system builds a manifest file when a VPK is unpacked. The manifest files contains data such as the entry flags and texture flags. It also contains a field determining whether the file should be compressed or not. When a user repacks a pack, the system attempts to load this manifest file and does a lookup to the object to retrieve the flags (most of these flags are unknown, but they are used by the engine and are necessary for stuff like cubemaps and texture files to work correctly. Cubemaps won't work with proper flags, and textures (decals, particle system components, etc..) will look washed out without them. I think some also determine whether a file within the VPK should be cached or not, so simply marking everything as 0x101 will probably end up in more CPU time and higher filesystem cache usage (depot/ is only 0x1, I don't think anything there is getting cached ever without the 0x100 flag). User could also repack a VPK while excluding anything that is not in the manifest file. So you could unpack all VPK's into a single directory (each VPK has its own manifest file tied to its level name), and rebuild all the VPK's with only the files that where originally in them. fs_pack_vpk command usage: <locale> <context> <level_name> <manifest_only> locale determines the pak language (default english), context determines whether is a server/client vpk, level_name determines the BSP name of the pak, manifest_only determines whether the pack system should only include files within the manifest (leaving this arg out will build all files into the vpk). The VPK workspace path is determined with ConVar 'fs_packedstore_workspace'.
2022-06-04 01:08:23 +02:00
CPackedStore refactor The CPackedStore class now heavily relies on the engine's FileSystem and KeyValues API. Since its part of the engine, it would make more sense to utilize the engine's features instead. It also allows for easier modifications. * The logic behind the build process hasn't changed.
2022-11-23 12:18:33 +01:00
struct VPKKeyValues_t
Proper VPK repacking Initial proper implementation pending cleanup. The new system builds a manifest file when a VPK is unpacked. The manifest files contains data such as the entry flags and texture flags. It also contains a field determining whether the file should be compressed or not. When a user repacks a pack, the system attempts to load this manifest file and does a lookup to the object to retrieve the flags (most of these flags are unknown, but they are used by the engine and are necessary for stuff like cubemaps and texture files to work correctly. Cubemaps won't work with proper flags, and textures (decals, particle system components, etc..) will look washed out without them. I think some also determine whether a file within the VPK should be cached or not, so simply marking everything as 0x101 will probably end up in more CPU time and higher filesystem cache usage (depot/ is only 0x1, I don't think anything there is getting cached ever without the 0x100 flag). User could also repack a VPK while excluding anything that is not in the manifest file. So you could unpack all VPK's into a single directory (each VPK has its own manifest file tied to its level name), and rebuild all the VPK's with only the files that where originally in them. fs_pack_vpk command usage: <locale> <context> <level_name> <manifest_only> locale determines the pak language (default english), context determines whether is a server/client vpk, level_name determines the BSP name of the pak, manifest_only determines whether the pack system should only include files within the manifest (leaving this arg out will build all files into the vpk). The VPK workspace path is determined with ConVar 'fs_packedstore_workspace'.
2022-06-04 01:08:23 +02:00
{
CPackedStore refactor The CPackedStore class now heavily relies on the engine's FileSystem and KeyValues API. Since its part of the engine, it would make more sense to utilize the engine's features instead. It also allows for easier modifications. * The logic behind the build process hasn't changed.
2022-11-23 12:18:33 +01:00
static constexpr uint16_t TEXTURE_FLAGS_DEFAULT = static_cast<uint16_t>(EPackedTextureFlags::TEXTURE_DEFAULT);
static constexpr uint32_t LOAD_FLAGS_DEFAULT = static_cast<uint32_t>(EPackedLoadFlags::LOAD_VISIBLE) | static_cast<uint32_t>(EPackedLoadFlags::LOAD_CACHE);
string m_svEntryPath;
uint16_t m_iPreloadSize;
uint32_t m_nLoadFlags;
uint16_t m_nTextureFlags;
bool m_bUseCompression;
bool m_bUseDataSharing;
VPKKeyValues_t(const string& svEntryPath = "", uint16_t iPreloadSize = NULL, uint32_t nLoadFlags = LOAD_FLAGS_DEFAULT,
uint16_t nTextureFlags = TEXTURE_FLAGS_DEFAULT, bool bUseCompression = true, bool bUseDataSharing = true);
Proper VPK repacking Initial proper implementation pending cleanup. The new system builds a manifest file when a VPK is unpacked. The manifest files contains data such as the entry flags and texture flags. It also contains a field determining whether the file should be compressed or not. When a user repacks a pack, the system attempts to load this manifest file and does a lookup to the object to retrieve the flags (most of these flags are unknown, but they are used by the engine and are necessary for stuff like cubemaps and texture files to work correctly. Cubemaps won't work with proper flags, and textures (decals, particle system components, etc..) will look washed out without them. I think some also determine whether a file within the VPK should be cached or not, so simply marking everything as 0x101 will probably end up in more CPU time and higher filesystem cache usage (depot/ is only 0x1, I don't think anything there is getting cached ever without the 0x100 flag). User could also repack a VPK while excluding anything that is not in the manifest file. So you could unpack all VPK's into a single directory (each VPK has its own manifest file tied to its level name), and rebuild all the VPK's with only the files that where originally in them. fs_pack_vpk command usage: <locale> <context> <level_name> <manifest_only> locale determines the pak language (default english), context determines whether is a server/client vpk, level_name determines the BSP name of the pak, manifest_only determines whether the pack system should only include files within the manifest (leaving this arg out will build all files into the vpk). The VPK workspace path is determined with ConVar 'fs_packedstore_workspace'.
2022-06-04 01:08:23 +02:00
};
Update terminology entirely for packedstore.h/.cpp VPK tooling also works on the R1 game (Titanfall 1)
2022-06-06 14:54:22 +02:00
struct VPKChunkDescriptor_t
Code base refactor + major performance and readability improvement. Read description for details. * Codebase restructured to SourceSDK codebase style and .cpp/.h assertion paths in the game executable. * Document most functions with valve style 'Purpose' blocks. * Rename variables to match the rest of the codebase and Valve's naming convention. * Dedicated DLL and the SDKLauncher now share the same codebase as the DevSDK. * Obtain globals or pointers directly instead of waiting for runtime initialized data. * Dynamically search for all functions and globals (this doesn't count for dedicated yet!). * Initialize most in-SDK variables. * Move certain prints and other utilities under ConVars to reduce verbosity and increase performance. * Print all pattern scan results through a virtual function to make it easier to add and debug new patterns in the future. * Type global var pointers appropriately if class or type is known and implemented. * Forward declare 'CClient' class to avoid having 2 'g_pClient' copies. * Add IDA's pseudo definitions for easier prototyping with decompiled assembly code. * RPAK decompress Command callback implementation. * Load decompressed RPaks from 'paks\Win32\' overriding the ones in 'paks\Win64\' (the decompress callback will automatically fix the header and write it to 'paks\Win32\'). * VPK decompress Command callback implementation. * Move CRC32 ands Adler32 to implementation files. * Server will print out more details about the connecting client. * Upgrade ImGui lib to v1.86. * Don't compile id3dx.h for dedicated. * Don't compile id3dx.cpp for dedicated * Implement DevMsg print function allowing to print information to the in-game VGUI/RUI console overlay, ImGui console overlay and the external windows console * Fixed bug where the Error function would not properly terminate the process when an error is called. This caused access violations for critical/non-recoverable errors. * Fixed bug where the game would crash if the console or server browser was enabled while the game was still starting up. * Several bug fixes for the dedicated server (warning: dedicated is still considered work-in-progress!).
2021-12-25 22:36:38 +01:00
{
CPackedStore: light optimizations * Don't call copy constructor on entry blocks when building the directory file. * Don't call copy constructor for string when we don't want to sanitize the directory file name.
2022-10-12 23:39:26 +02:00
uint32_t m_nLoadFlags; // Load flags.
uint16_t m_nTextureFlags; // Texture flags (only used if the entry is a vtf).
Improve consistency throughout packedstore class No modifications to the existing logic have been performed. Some parameter names have been renamed to improve code readability and maintainability.
2022-11-16 00:54:51 +01:00
uint64_t m_nPackFileOffset; // Offset in pack file.
CPackedStore: light optimizations * Don't call copy constructor on entry blocks when building the directory file. * Don't call copy constructor for string when we don't want to sanitize the directory file name.
2022-10-12 23:39:26 +02:00
uint64_t m_nCompressedSize; // Compressed size of chunk.
uint64_t m_nUncompressedSize; // Uncompressed size of chunk.
Code base refactor + major performance and readability improvement. Read description for details. * Codebase restructured to SourceSDK codebase style and .cpp/.h assertion paths in the game executable. * Document most functions with valve style 'Purpose' blocks. * Rename variables to match the rest of the codebase and Valve's naming convention. * Dedicated DLL and the SDKLauncher now share the same codebase as the DevSDK. * Obtain globals or pointers directly instead of waiting for runtime initialized data. * Dynamically search for all functions and globals (this doesn't count for dedicated yet!). * Initialize most in-SDK variables. * Move certain prints and other utilities under ConVars to reduce verbosity and increase performance. * Print all pattern scan results through a virtual function to make it easier to add and debug new patterns in the future. * Type global var pointers appropriately if class or type is known and implemented. * Forward declare 'CClient' class to avoid having 2 'g_pClient' copies. * Add IDA's pseudo definitions for easier prototyping with decompiled assembly code. * RPAK decompress Command callback implementation. * Load decompressed RPaks from 'paks\Win32\' overriding the ones in 'paks\Win64\' (the decompress callback will automatically fix the header and write it to 'paks\Win32\'). * VPK decompress Command callback implementation. * Move CRC32 ands Adler32 to implementation files. * Server will print out more details about the connecting client. * Upgrade ImGui lib to v1.86. * Don't compile id3dx.h for dedicated. * Don't compile id3dx.cpp for dedicated * Implement DevMsg print function allowing to print information to the in-game VGUI/RUI console overlay, ImGui console overlay and the external windows console * Fixed bug where the Error function would not properly terminate the process when an error is called. This caused access violations for critical/non-recoverable errors. * Fixed bug where the game would crash if the console or server browser was enabled while the game was still starting up. * Several bug fixes for the dedicated server (warning: dedicated is still considered work-in-progress!).
2021-12-25 22:36:38 +01:00
bool m_bIsCompressed = false;
Move some CPackedStore methods to VPKDir_t These methods where only used for and by the VPKDir_t structure. Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
VPKChunkDescriptor_t(FileHandle_t hDirectoryFile);
VPKChunkDescriptor_t(uint32_t nLoadFlags, uint16_t nTextureFlags, uint64_t nPackFileOffset, uint64_t nCompressedSize, uint64_t nUncompressedSize);
Code base refactor + major performance and readability improvement. Read description for details. * Codebase restructured to SourceSDK codebase style and .cpp/.h assertion paths in the game executable. * Document most functions with valve style 'Purpose' blocks. * Rename variables to match the rest of the codebase and Valve's naming convention. * Dedicated DLL and the SDKLauncher now share the same codebase as the DevSDK. * Obtain globals or pointers directly instead of waiting for runtime initialized data. * Dynamically search for all functions and globals (this doesn't count for dedicated yet!). * Initialize most in-SDK variables. * Move certain prints and other utilities under ConVars to reduce verbosity and increase performance. * Print all pattern scan results through a virtual function to make it easier to add and debug new patterns in the future. * Type global var pointers appropriately if class or type is known and implemented. * Forward declare 'CClient' class to avoid having 2 'g_pClient' copies. * Add IDA's pseudo definitions for easier prototyping with decompiled assembly code. * RPAK decompress Command callback implementation. * Load decompressed RPaks from 'paks\Win32\' overriding the ones in 'paks\Win64\' (the decompress callback will automatically fix the header and write it to 'paks\Win32\'). * VPK decompress Command callback implementation. * Move CRC32 ands Adler32 to implementation files. * Server will print out more details about the connecting client. * Upgrade ImGui lib to v1.86. * Don't compile id3dx.h for dedicated. * Don't compile id3dx.cpp for dedicated * Implement DevMsg print function allowing to print information to the in-game VGUI/RUI console overlay, ImGui console overlay and the external windows console * Fixed bug where the Error function would not properly terminate the process when an error is called. This caused access violations for critical/non-recoverable errors. * Fixed bug where the game would crash if the console or server browser was enabled while the game was still starting up. * Several bug fixes for the dedicated server (warning: dedicated is still considered work-in-progress!).
2021-12-25 22:36:38 +01:00
};
See description * Added 2 new FileSystem pointers with new features in their classes. * Register all factory instances created by the GameDLL in the SDK. * Added new command 'fs_mount_vpk' to mount a specified VPK file. * Renamed 'fs_decompress_pak' to 'fs_unpack_vpk'. * Some renaming of Factory and VPK types. * Some light optimizations/cleanup.
2022-04-09 00:59:42 +02:00
struct VPKEntryBlock_t
Code base refactor + major performance and readability improvement. Read description for details. * Codebase restructured to SourceSDK codebase style and .cpp/.h assertion paths in the game executable. * Document most functions with valve style 'Purpose' blocks. * Rename variables to match the rest of the codebase and Valve's naming convention. * Dedicated DLL and the SDKLauncher now share the same codebase as the DevSDK. * Obtain globals or pointers directly instead of waiting for runtime initialized data. * Dynamically search for all functions and globals (this doesn't count for dedicated yet!). * Initialize most in-SDK variables. * Move certain prints and other utilities under ConVars to reduce verbosity and increase performance. * Print all pattern scan results through a virtual function to make it easier to add and debug new patterns in the future. * Type global var pointers appropriately if class or type is known and implemented. * Forward declare 'CClient' class to avoid having 2 'g_pClient' copies. * Add IDA's pseudo definitions for easier prototyping with decompiled assembly code. * RPAK decompress Command callback implementation. * Load decompressed RPaks from 'paks\Win32\' overriding the ones in 'paks\Win64\' (the decompress callback will automatically fix the header and write it to 'paks\Win32\'). * VPK decompress Command callback implementation. * Move CRC32 ands Adler32 to implementation files. * Server will print out more details about the connecting client. * Upgrade ImGui lib to v1.86. * Don't compile id3dx.h for dedicated. * Don't compile id3dx.cpp for dedicated * Implement DevMsg print function allowing to print information to the in-game VGUI/RUI console overlay, ImGui console overlay and the external windows console * Fixed bug where the Error function would not properly terminate the process when an error is called. This caused access violations for critical/non-recoverable errors. * Fixed bug where the game would crash if the console or server browser was enabled while the game was still starting up. * Several bug fixes for the dedicated server (warning: dedicated is still considered work-in-progress!).
2021-12-25 22:36:38 +01:00
{
CPackedStore: light optimizations * Don't call copy constructor on entry blocks when building the directory file. * Don't call copy constructor for string when we don't want to sanitize the directory file name.
2022-10-12 23:39:26 +02:00
uint32_t m_nFileCRC; // Crc32 for the uncompressed entry.
uint16_t m_iPreloadSize; // Preload bytes.
uint16_t m_iPackFileIndex; // Index of the pack file that contains this entry.
Improve consistency throughout packedstore class No modifications to the existing logic have been performed. Some parameter names have been renamed to improve code readability and maintainability.
2022-11-16 00:54:51 +01:00
vector<VPKChunkDescriptor_t> m_vFragments; // Vector of all the chunks of a given entry (chunks have a size limit of 1 MiB, anything over this limit is fragmented into smaller chunks).
CPackedStore: light optimizations * Don't call copy constructor on entry blocks when building the directory file. * Don't call copy constructor for string when we don't want to sanitize the directory file name.
2022-10-12 23:39:26 +02:00
string m_svEntryPath; // Path to entry within vpk.
Update terminology entirely for packedstore.h/.cpp VPK tooling also works on the R1 game (Titanfall 1)
2022-06-06 14:54:22 +02:00
Avoid string copy in 'VPKEntryBlock_t::VPKEntryBlock_t' Avoid a copy when formatting string to that of the VPK directory tree structure.
2022-11-23 14:49:40 +01:00
VPKEntryBlock_t(FileHandle_t pFile, const string& svEntryPath);
Move some CPackedStore methods to VPKDir_t These methods where only used for and by the VPKDir_t structure. Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
VPKEntryBlock_t(const uint8_t* pData, size_t nLen, int64_t nOffset, uint16_t iPreloadSize,
uint16_t iPackFileIndex, uint32_t nLoadFlags, uint16_t nTextureFlags, const string& svEntryPath);
Code base refactor + major performance and readability improvement. Read description for details. * Codebase restructured to SourceSDK codebase style and .cpp/.h assertion paths in the game executable. * Document most functions with valve style 'Purpose' blocks. * Rename variables to match the rest of the codebase and Valve's naming convention. * Dedicated DLL and the SDKLauncher now share the same codebase as the DevSDK. * Obtain globals or pointers directly instead of waiting for runtime initialized data. * Dynamically search for all functions and globals (this doesn't count for dedicated yet!). * Initialize most in-SDK variables. * Move certain prints and other utilities under ConVars to reduce verbosity and increase performance. * Print all pattern scan results through a virtual function to make it easier to add and debug new patterns in the future. * Type global var pointers appropriately if class or type is known and implemented. * Forward declare 'CClient' class to avoid having 2 'g_pClient' copies. * Add IDA's pseudo definitions for easier prototyping with decompiled assembly code. * RPAK decompress Command callback implementation. * Load decompressed RPaks from 'paks\Win32\' overriding the ones in 'paks\Win64\' (the decompress callback will automatically fix the header and write it to 'paks\Win32\'). * VPK decompress Command callback implementation. * Move CRC32 ands Adler32 to implementation files. * Server will print out more details about the connecting client. * Upgrade ImGui lib to v1.86. * Don't compile id3dx.h for dedicated. * Don't compile id3dx.cpp for dedicated * Implement DevMsg print function allowing to print information to the in-game VGUI/RUI console overlay, ImGui console overlay and the external windows console * Fixed bug where the Error function would not properly terminate the process when an error is called. This caused access violations for critical/non-recoverable errors. * Fixed bug where the game would crash if the console or server browser was enabled while the game was still starting up. * Several bug fixes for the dedicated server (warning: dedicated is still considered work-in-progress!).
2021-12-25 22:36:38 +01:00
};
Proper VPK repacking Initial proper implementation pending cleanup. The new system builds a manifest file when a VPK is unpacked. The manifest files contains data such as the entry flags and texture flags. It also contains a field determining whether the file should be compressed or not. When a user repacks a pack, the system attempts to load this manifest file and does a lookup to the object to retrieve the flags (most of these flags are unknown, but they are used by the engine and are necessary for stuff like cubemaps and texture files to work correctly. Cubemaps won't work with proper flags, and textures (decals, particle system components, etc..) will look washed out without them. I think some also determine whether a file within the VPK should be cached or not, so simply marking everything as 0x101 will probably end up in more CPU time and higher filesystem cache usage (depot/ is only 0x1, I don't think anything there is getting cached ever without the 0x100 flag). User could also repack a VPK while excluding anything that is not in the manifest file. So you could unpack all VPK's into a single directory (each VPK has its own manifest file tied to its level name), and rebuild all the VPK's with only the files that where originally in them. fs_pack_vpk command usage: <locale> <context> <level_name> <manifest_only> locale determines the pak language (default english), context determines whether is a server/client vpk, level_name determines the BSP name of the pak, manifest_only determines whether the pack system should only include files within the manifest (leaving this arg out will build all files into the vpk). The VPK workspace path is determined with ConVar 'fs_packedstore_workspace'.
2022-06-04 01:08:23 +02:00
struct VPKDirHeader_t
Code base refactor + major performance and readability improvement. Read description for details. * Codebase restructured to SourceSDK codebase style and .cpp/.h assertion paths in the game executable. * Document most functions with valve style 'Purpose' blocks. * Rename variables to match the rest of the codebase and Valve's naming convention. * Dedicated DLL and the SDKLauncher now share the same codebase as the DevSDK. * Obtain globals or pointers directly instead of waiting for runtime initialized data. * Dynamically search for all functions and globals (this doesn't count for dedicated yet!). * Initialize most in-SDK variables. * Move certain prints and other utilities under ConVars to reduce verbosity and increase performance. * Print all pattern scan results through a virtual function to make it easier to add and debug new patterns in the future. * Type global var pointers appropriately if class or type is known and implemented. * Forward declare 'CClient' class to avoid having 2 'g_pClient' copies. * Add IDA's pseudo definitions for easier prototyping with decompiled assembly code. * RPAK decompress Command callback implementation. * Load decompressed RPaks from 'paks\Win32\' overriding the ones in 'paks\Win64\' (the decompress callback will automatically fix the header and write it to 'paks\Win32\'). * VPK decompress Command callback implementation. * Move CRC32 ands Adler32 to implementation files. * Server will print out more details about the connecting client. * Upgrade ImGui lib to v1.86. * Don't compile id3dx.h for dedicated. * Don't compile id3dx.cpp for dedicated * Implement DevMsg print function allowing to print information to the in-game VGUI/RUI console overlay, ImGui console overlay and the external windows console * Fixed bug where the Error function would not properly terminate the process when an error is called. This caused access violations for critical/non-recoverable errors. * Fixed bug where the game would crash if the console or server browser was enabled while the game was still starting up. * Several bug fixes for the dedicated server (warning: dedicated is still considered work-in-progress!).
2021-12-25 22:36:38 +01:00
{
CPackedStore: light optimizations * Don't call copy constructor on entry blocks when building the directory file. * Don't call copy constructor for string when we don't want to sanitize the directory file name.
2022-10-12 23:39:26 +02:00
uint32_t m_nHeaderMarker; // File magic.
uint16_t m_nMajorVersion; // Vpk major version.
uint16_t m_nMinorVersion; // Vpk minor version.
uint32_t m_nDirectorySize; // Directory tree size.
uint32_t m_nSignatureSize; // Directory signature.
Initial implementation of compressed VPK rebuilding The 'fs_pack_vpk' allows compressing specific files into a VPK (compressed). This implementation has better compression ratio's then currently available VPK tools with compression (at the cost of speed). The implementation is early, a directory iterator with a files for flags is planned soon to fully rebuild VPK's on-demand. A dedicated standalone program is also planned. The output dir and archive formats should match 1:1 with the original game
2022-05-30 02:56:15 +02:00
};
Move some CPackedStore methods to VPKDir_t These methods where only used for and by the VPKDir_t structure. Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
struct VPKPair_t
{
string m_svPackName;
string m_svDirectoryName;
VPKPair_t(string svLanguage, string svTarget, const string& svLevel, int nPatch);
};
Initial implementation of compressed VPK rebuilding The 'fs_pack_vpk' allows compressing specific files into a VPK (compressed). This implementation has better compression ratio's then currently available VPK tools with compression (at the cost of speed). The implementation is early, a directory iterator with a files for flags is planned soon to fully rebuild VPK's on-demand. A dedicated standalone program is also planned. The output dir and archive formats should match 1:1 with the original game
2022-05-30 02:56:15 +02:00
struct VPKDir_t
{
CPackedStore: light optimizations * Don't call copy constructor on entry blocks when building the directory file. * Don't call copy constructor for string when we don't want to sanitize the directory file name.
2022-10-12 23:39:26 +02:00
VPKDirHeader_t m_vHeader; // Dir header.
vector<VPKEntryBlock_t> m_vEntryBlocks; // Vector of entry blocks.
Improve consistency throughout packedstore class No modifications to the existing logic have been performed. Some parameter names have been renamed to improve code readability and maintainability.
2022-11-16 00:54:51 +01:00
uint16_t m_nPackFileCount; // Number of pack patches (pack file count-1).
vector<string> m_vPackFile; // Vector of pack file names.
Move some CPackedStore methods to VPKDir_t These methods where only used for and by the VPKDir_t structure. Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
string m_svDirectoryPath;// Path to vpk_dir file.
Code base refactor + major performance and readability improvement. Read description for details. * Codebase restructured to SourceSDK codebase style and .cpp/.h assertion paths in the game executable. * Document most functions with valve style 'Purpose' blocks. * Rename variables to match the rest of the codebase and Valve's naming convention. * Dedicated DLL and the SDKLauncher now share the same codebase as the DevSDK. * Obtain globals or pointers directly instead of waiting for runtime initialized data. * Dynamically search for all functions and globals (this doesn't count for dedicated yet!). * Initialize most in-SDK variables. * Move certain prints and other utilities under ConVars to reduce verbosity and increase performance. * Print all pattern scan results through a virtual function to make it easier to add and debug new patterns in the future. * Type global var pointers appropriately if class or type is known and implemented. * Forward declare 'CClient' class to avoid having 2 'g_pClient' copies. * Add IDA's pseudo definitions for easier prototyping with decompiled assembly code. * RPAK decompress Command callback implementation. * Load decompressed RPaks from 'paks\Win32\' overriding the ones in 'paks\Win64\' (the decompress callback will automatically fix the header and write it to 'paks\Win32\'). * VPK decompress Command callback implementation. * Move CRC32 ands Adler32 to implementation files. * Server will print out more details about the connecting client. * Upgrade ImGui lib to v1.86. * Don't compile id3dx.h for dedicated. * Don't compile id3dx.cpp for dedicated * Implement DevMsg print function allowing to print information to the in-game VGUI/RUI console overlay, ImGui console overlay and the external windows console * Fixed bug where the Error function would not properly terminate the process when an error is called. This caused access violations for critical/non-recoverable errors. * Fixed bug where the game would crash if the console or server browser was enabled while the game was still starting up. * Several bug fixes for the dedicated server (warning: dedicated is still considered work-in-progress!).
2021-12-25 22:36:38 +01:00
Move some CPackedStore methods to VPKDir_t These methods where only used for and by the VPKDir_t structure. Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
VPKDir_t()
{
m_vHeader.m_nHeaderMarker = VPK_HEADER_MARKER; m_vHeader.m_nMajorVersion = VPK_MAJOR_VERSION;
m_vHeader.m_nMinorVersion = VPK_MINOR_VERSION; m_vHeader.m_nDirectorySize = NULL, m_vHeader.m_nSignatureSize = NULL;
Light CPackedStore cleanup
2022-12-04 14:17:14 +01:00
m_nPackFileCount = NULL;
Move some CPackedStore methods to VPKDir_t These methods where only used for and by the VPKDir_t structure. Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
};
VPKDir_t(const string& svDirectoryFile);
VPKDir_t(const string& svDirectoryFile, bool bSanitizeName);
Initial implementation of compressed VPK rebuilding The 'fs_pack_vpk' allows compressing specific files into a VPK (compressed). This implementation has better compression ratio's then currently available VPK tools with compression (at the cost of speed). The implementation is early, a directory iterator with a files for flags is planned soon to fully rebuild VPK's on-demand. A dedicated standalone program is also planned. The output dir and archive formats should match 1:1 with the original game
2022-05-30 02:56:15 +02:00
Move some CPackedStore methods to VPKDir_t These methods where only used for and by the VPKDir_t structure. Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
void Init(const string& svPath);
string StripLocalePrefix(const string& svDirectoryFile) const;
string GetPackFile(const string& svDirectoryPath, uint16_t iPackFileIndex) const;
void WriteHeader(FileHandle_t hDirectoryFile) const;
void WriteTreeSize(FileHandle_t hDirectoryFile) const;
uint64_t WriteDescriptor(FileHandle_t hDirectoryFile, std::map<string, std::map<string, std::list<VPKEntryBlock_t>>>& vMap) const;
Split 'VPKDir_t::Build' into multiple methods Each process of writing out a VPK directory is now within its own method.
2022-11-23 12:27:57 +01:00
void BuildDirectoryTree(const vector<VPKEntryBlock_t>& vEntryBlocks, std::map<string, std::map<string, std::list<VPKEntryBlock_t>>>& vMap) const;
void BuildDirectoryFile(const string& svDirectoryFile, const vector<VPKEntryBlock_t>& vEntryBlocks);
Proper VPK repacking Initial proper implementation pending cleanup. The new system builds a manifest file when a VPK is unpacked. The manifest files contains data such as the entry flags and texture flags. It also contains a field determining whether the file should be compressed or not. When a user repacks a pack, the system attempts to load this manifest file and does a lookup to the object to retrieve the flags (most of these flags are unknown, but they are used by the engine and are necessary for stuff like cubemaps and texture files to work correctly. Cubemaps won't work with proper flags, and textures (decals, particle system components, etc..) will look washed out without them. I think some also determine whether a file within the VPK should be cached or not, so simply marking everything as 0x101 will probably end up in more CPU time and higher filesystem cache usage (depot/ is only 0x1, I don't think anything there is getting cached ever without the 0x100 flag). User could also repack a VPK while excluding anything that is not in the manifest file. So you could unpack all VPK's into a single directory (each VPK has its own manifest file tied to its level name), and rebuild all the VPK's with only the files that where originally in them. fs_pack_vpk command usage: <locale> <context> <level_name> <manifest_only> locale determines the pak language (default english), context determines whether is a server/client vpk, level_name determines the BSP name of the pak, manifest_only determines whether the pack system should only include files within the manifest (leaving this arg out will build all files into the vpk). The VPK workspace path is determined with ConVar 'fs_packedstore_workspace'.
2022-06-04 01:08:23 +02:00
};
Code base refactor + major performance and readability improvement. Read description for details. * Codebase restructured to SourceSDK codebase style and .cpp/.h assertion paths in the game executable. * Document most functions with valve style 'Purpose' blocks. * Rename variables to match the rest of the codebase and Valve's naming convention. * Dedicated DLL and the SDKLauncher now share the same codebase as the DevSDK. * Obtain globals or pointers directly instead of waiting for runtime initialized data. * Dynamically search for all functions and globals (this doesn't count for dedicated yet!). * Initialize most in-SDK variables. * Move certain prints and other utilities under ConVars to reduce verbosity and increase performance. * Print all pattern scan results through a virtual function to make it easier to add and debug new patterns in the future. * Type global var pointers appropriately if class or type is known and implemented. * Forward declare 'CClient' class to avoid having 2 'g_pClient' copies. * Add IDA's pseudo definitions for easier prototyping with decompiled assembly code. * RPAK decompress Command callback implementation. * Load decompressed RPaks from 'paks\Win32\' overriding the ones in 'paks\Win64\' (the decompress callback will automatically fix the header and write it to 'paks\Win32\'). * VPK decompress Command callback implementation. * Move CRC32 ands Adler32 to implementation files. * Server will print out more details about the connecting client. * Upgrade ImGui lib to v1.86. * Don't compile id3dx.h for dedicated. * Don't compile id3dx.cpp for dedicated * Implement DevMsg print function allowing to print information to the in-game VGUI/RUI console overlay, ImGui console overlay and the external windows console * Fixed bug where the Error function would not properly terminate the process when an error is called. This caused access violations for critical/non-recoverable errors. * Fixed bug where the game would crash if the console or server browser was enabled while the game was still starting up. * Several bug fixes for the dedicated server (warning: dedicated is still considered work-in-progress!).
2021-12-25 22:36:38 +01:00
class CPackedStore
{
public:
Create RCON ConCommand
2022-02-06 15:59:46 +01:00
void InitLzCompParams(void);
void InitLzDecompParams(void);
Proper VPK repacking Initial proper implementation pending cleanup. The new system builds a manifest file when a VPK is unpacked. The manifest files contains data such as the entry flags and texture flags. It also contains a field determining whether the file should be compressed or not. When a user repacks a pack, the system attempts to load this manifest file and does a lookup to the object to retrieve the flags (most of these flags are unknown, but they are used by the engine and are necessary for stuff like cubemaps and texture files to work correctly. Cubemaps won't work with proper flags, and textures (decals, particle system components, etc..) will look washed out without them. I think some also determine whether a file within the VPK should be cached or not, so simply marking everything as 0x101 will probably end up in more CPU time and higher filesystem cache usage (depot/ is only 0x1, I don't think anything there is getting cached ever without the 0x100 flag). User could also repack a VPK while excluding anything that is not in the manifest file. So you could unpack all VPK's into a single directory (each VPK has its own manifest file tied to its level name), and rebuild all the VPK's with only the files that where originally in them. fs_pack_vpk command usage: <locale> <context> <level_name> <manifest_only> locale determines the pak language (default english), context determines whether is a server/client vpk, level_name determines the BSP name of the pak, manifest_only determines whether the pack system should only include files within the manifest (leaving this arg out will build all files into the vpk). The VPK workspace path is determined with ConVar 'fs_packedstore_workspace'.
2022-06-04 01:08:23 +02:00
Set VPK compression level through ConVar
2022-09-11 00:16:31 +02:00
lzham_compress_level GetCompressionLevel(void) const;
Update terminology entirely for packedstore.h/.cpp VPK tooling also works on the R1 game (Titanfall 1)
2022-06-06 14:54:22 +02:00
Move some CPackedStore methods to VPKDir_t These methods where only used for and by the VPKDir_t structure. Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
vector<VPKEntryBlock_t> GetEntryBlocks(FileHandle_t hDirectoryFile) const;
vector<VPKKeyValues_t> GetEntryValues(const string& svWorkspace) const;
vector<VPKKeyValues_t> GetEntryValues(const string& svWorkspace, KeyValues* pManifestKeyValues) const;
Update terminology entirely for packedstore.h/.cpp VPK tooling also works on the R1 game (Titanfall 1)
2022-06-06 14:54:22 +02:00
VPK system improvements More detailed operation statistics. Discard VPK directory files who's header doesn't match engine requirements. Separated manifest properly between language and context (you can now package for each language and context from a single workspace).
2022-06-05 17:28:39 +02:00
string GetNameParts(const string& svDirectoryName, int nCaptureGroup) const;
Improve consistency throughout packedstore class No modifications to the existing logic have been performed. Some parameter names have been renamed to improve code readability and maintainability.
2022-11-16 00:54:51 +01:00
string GetLevelName(const string& svDirectoryName) const;
Add support for .vpkignore file Folders can be globally excluded using the '.vpkignore' file (placed in the workspace root). To exclude specific files, use the manifest file.
2022-06-06 23:08:53 +02:00
CPackedStore refactor The CPackedStore class now heavily relies on the engine's FileSystem and KeyValues API. Since its part of the engine, it would make more sense to utilize the engine's features instead. It also allows for easier modifications. * The logic behind the build process hasn't changed.
2022-11-23 12:18:33 +01:00
KeyValues* GetManifest(const string& svWorkspace, const string& svManifestName) const;
Improve consistency throughout packedstore class No modifications to the existing logic have been performed. Some parameter names have been renamed to improve code readability and maintainability.
2022-11-16 00:54:51 +01:00
vector<string> GetIgnoreList(const string& svWorkspace) const;
Proper VPK repacking Initial proper implementation pending cleanup. The new system builds a manifest file when a VPK is unpacked. The manifest files contains data such as the entry flags and texture flags. It also contains a field determining whether the file should be compressed or not. When a user repacks a pack, the system attempts to load this manifest file and does a lookup to the object to retrieve the flags (most of these flags are unknown, but they are used by the engine and are necessary for stuff like cubemaps and texture files to work correctly. Cubemaps won't work with proper flags, and textures (decals, particle system components, etc..) will look washed out without them. I think some also determine whether a file within the VPK should be cached or not, so simply marking everything as 0x101 will probably end up in more CPU time and higher filesystem cache usage (depot/ is only 0x1, I don't think anything there is getting cached ever without the 0x100 flag). User could also repack a VPK while excluding anything that is not in the manifest file. So you could unpack all VPK's into a single directory (each VPK has its own manifest file tied to its level name), and rebuild all the VPK's with only the files that where originally in them. fs_pack_vpk command usage: <locale> <context> <level_name> <manifest_only> locale determines the pak language (default english), context determines whether is a server/client vpk, level_name determines the BSP name of the pak, manifest_only determines whether the pack system should only include files within the manifest (leaving this arg out will build all files into the vpk). The VPK workspace path is determined with ConVar 'fs_packedstore_workspace'.
2022-06-04 01:08:23 +02:00
Move some CPackedStore methods to VPKDir_t These methods where only used for and by the VPKDir_t structure. Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
string FormatEntryPath(const string& svPath, const string& svName, const string& svExtension) const;
Proper VPK repacking Initial proper implementation pending cleanup. The new system builds a manifest file when a VPK is unpacked. The manifest files contains data such as the entry flags and texture flags. It also contains a field determining whether the file should be compressed or not. When a user repacks a pack, the system attempts to load this manifest file and does a lookup to the object to retrieve the flags (most of these flags are unknown, but they are used by the engine and are necessary for stuff like cubemaps and texture files to work correctly. Cubemaps won't work with proper flags, and textures (decals, particle system components, etc..) will look washed out without them. I think some also determine whether a file within the VPK should be cached or not, so simply marking everything as 0x101 will probably end up in more CPU time and higher filesystem cache usage (depot/ is only 0x1, I don't think anything there is getting cached ever without the 0x100 flag). User could also repack a VPK while excluding anything that is not in the manifest file. So you could unpack all VPK's into a single directory (each VPK has its own manifest file tied to its level name), and rebuild all the VPK's with only the files that where originally in them. fs_pack_vpk command usage: <locale> <context> <level_name> <manifest_only> locale determines the pak language (default english), context determines whether is a server/client vpk, level_name determines the BSP name of the pak, manifest_only determines whether the pack system should only include files within the manifest (leaving this arg out will build all files into the vpk). The VPK workspace path is determined with ConVar 'fs_packedstore_workspace'.
2022-06-04 01:08:23 +02:00
Move some CPackedStore methods to VPKDir_t These methods where only used for and by the VPKDir_t structure. Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
void BuildManifest(const vector<VPKEntryBlock_t>& vBlock, const string& svWorkspace, const string& svManifestName) const;
Proper VPK repacking Initial proper implementation pending cleanup. The new system builds a manifest file when a VPK is unpacked. The manifest files contains data such as the entry flags and texture flags. It also contains a field determining whether the file should be compressed or not. When a user repacks a pack, the system attempts to load this manifest file and does a lookup to the object to retrieve the flags (most of these flags are unknown, but they are used by the engine and are necessary for stuff like cubemaps and texture files to work correctly. Cubemaps won't work with proper flags, and textures (decals, particle system components, etc..) will look washed out without them. I think some also determine whether a file within the VPK should be cached or not, so simply marking everything as 0x101 will probably end up in more CPU time and higher filesystem cache usage (depot/ is only 0x1, I don't think anything there is getting cached ever without the 0x100 flag). User could also repack a VPK while excluding anything that is not in the manifest file. So you could unpack all VPK's into a single directory (each VPK has its own manifest file tied to its level name), and rebuild all the VPK's with only the files that where originally in them. fs_pack_vpk command usage: <locale> <context> <level_name> <manifest_only> locale determines the pak language (default english), context determines whether is a server/client vpk, level_name determines the BSP name of the pak, manifest_only determines whether the pack system should only include files within the manifest (leaving this arg out will build all files into the vpk). The VPK workspace path is determined with ConVar 'fs_packedstore_workspace'.
2022-06-04 01:08:23 +02:00
Improve consistency throughout packedstore class No modifications to the existing logic have been performed. Some parameter names have been renamed to improve code readability and maintainability.
2022-11-16 00:54:51 +01:00
void PackWorkspace(const VPKPair_t& vPair, const string& svWorkspace, const string& svBuildPath, bool bManifestOnly);
Move some CPackedStore methods to VPKDir_t These methods where only used for and by the VPKDir_t structure. Also renamed variable and method names to reflect new changes regarding the use of KeyValues by creating a structure with all values defined during file lookup rather than doing this during the pack process.
2022-11-23 17:09:48 +01:00
void UnpackWorkspace(const VPKDir_t& vDirectory, const string& svWorkspace = "");
Remove experimental CRC implementation in LZHAM This implementation was done to calculate buffer CRC on the go, as that was what the VPK file system uses for the file crc's. But it later turned out to be unnecessary, and was never reverted.
2022-12-04 01:33:10 +01:00
void ValidateCRC32PostDecomp(const string& svAssetPath, const uint32_t nFileCRC);
Update terminology entirely for packedstore.h/.cpp VPK tooling also works on the R1 game (Titanfall 1)
2022-06-06 14:54:22 +02:00
private:
Add convar for adjusting LZHAM compressor max helper threads
2022-11-17 20:37:12 +01:00
size_t m_nChunkCount; // The number of fragments for this asset.
Light CPackedStore cleanup
2022-12-04 14:17:14 +01:00
lzham_compress_params m_lzCompParams; // LZham compression parameters.
Improve consistency throughout packedstore class No modifications to the existing logic have been performed. Some parameter names have been renamed to improve code readability and maintainability.
2022-11-16 00:54:51 +01:00
lzham_decompress_params m_lzDecompParams; // LZham decompression parameters.
std::unordered_map<string, VPKChunkDescriptor_t&> m_mChunkHashMap;
Code base refactor + major performance and readability improvement. Read description for details. * Codebase restructured to SourceSDK codebase style and .cpp/.h assertion paths in the game executable. * Document most functions with valve style 'Purpose' blocks. * Rename variables to match the rest of the codebase and Valve's naming convention. * Dedicated DLL and the SDKLauncher now share the same codebase as the DevSDK. * Obtain globals or pointers directly instead of waiting for runtime initialized data. * Dynamically search for all functions and globals (this doesn't count for dedicated yet!). * Initialize most in-SDK variables. * Move certain prints and other utilities under ConVars to reduce verbosity and increase performance. * Print all pattern scan results through a virtual function to make it easier to add and debug new patterns in the future. * Type global var pointers appropriately if class or type is known and implemented. * Forward declare 'CClient' class to avoid having 2 'g_pClient' copies. * Add IDA's pseudo definitions for easier prototyping with decompiled assembly code. * RPAK decompress Command callback implementation. * Load decompressed RPaks from 'paks\Win32\' overriding the ones in 'paks\Win64\' (the decompress callback will automatically fix the header and write it to 'paks\Win32\'). * VPK decompress Command callback implementation. * Move CRC32 ands Adler32 to implementation files. * Server will print out more details about the connecting client. * Upgrade ImGui lib to v1.86. * Don't compile id3dx.h for dedicated. * Don't compile id3dx.cpp for dedicated * Implement DevMsg print function allowing to print information to the in-game VGUI/RUI console overlay, ImGui console overlay and the external windows console * Fixed bug where the Error function would not properly terminate the process when an error is called. This caused access violations for critical/non-recoverable errors. * Fixed bug where the game would crash if the console or server browser was enabled while the game was still starting up. * Several bug fixes for the dedicated server (warning: dedicated is still considered work-in-progress!).
2021-12-25 22:36:38 +01:00
};
///////////////////////////////////////////////////////////////////////////////
extern CPackedStore* g_pPackedStore;
CPackedStore: light optimizations * Don't call copy constructor on entry blocks when building the directory file. * Don't call copy constructor for string when we don't want to sanitize the directory file name.
2022-10-12 23:39:26 +02:00
#endif // PACKEDSTORE_H
Reference in New Issue Copy Permalink