Decompressinator

NMS-Decompressinator

A batch script that uses PSARC (Sony PlayStation SDK Tool) and NMSResign to automatically process the game ".pak" files by unpacking them and repacking with compression disabled to improve game performance by eliminating CPU spikes caused by on-the-fly ".pak" decompression without running the game fully unpacked, which introduces longer loading times due to the file system strain caused by the game needing to open handles to thousands of files.

The amount of improvement this has on performance is variable from PC to PC, but it should make a small difference even on high end machines. If nothing else, it addresses that commonly experienced lag spike when leaving the atmosphere of a planet, and other similar smaller lag spikes that occur during gameplay situations where data is loading, and should generally improve the loading times when loading into a save file, warping from system to system, or using teleporters.

Obviously this performance boost comes at the cost of storage space. Uncompressing the files will roughly triple the amount of required storage space. As of No Man's Sky Worlds Part 1, assuming you delete the backup of the compressed files afterwards or ran with the -no-backup argument, the game takes up ~44.8GB of storage total.

Features:

Works on all No Man's Sky versions.
Doesn't conflict with any mods.
Tries to be smart about running again after No Man's Sky updates or branch switches by checking file timestamps. (Can be bypassed using -force.)
Backs up the base game files in a sub folder in case things go wrong. (Can be bypassed by using -no-backup, which also slightly reduces the required free space footprint during runtime by a bit.)
Disables the File Tampering warning by using NMSResign to generate BankSignatures.bin.

Usage:

Download and Extract the latest release.
Copy NMS-Decompressinator.bat, psarc.exe, and NMSResign.exe over to your No Man's Sky\GAMEDATA\PCBANKS folder.
Ensure you have sufficient free space. As of No Man's Sky Worlds Part 1, the uncompressed ".pak" files will consume 44.7GB of storage.
Run the NMS-Decompressinator.bat.
Wait patiently. This takes quite a while to run. You will see a completion message when it is done.
Test the game and make sure it works.
(Optional) Delete PackedFileBackup folder to reclaim the storage space that the original packed files take up.

After updates or changing beta branches, you may see a File Tampering warning upon starting up No Man's Sky. You can simply run the script again after updates or changing beta branches to unpack the updated files and get rid of the Tamper warning.

Subsequent runs of the script should also be faster, as it tries to skip files that haven't been updated by checking file timestamps. This behavior can be bypassed using the -force argument.

Technical Explanation:

No Man's Sky uses a format called PSARC (PlayStation Archive) for compressing it's files, with the file extension ".pak". These ".pak" files are located in the subfolder of the install of the game GAMEDATA\PCBANKS.

These files, like the name suggests are archives that store game files with compression and are decompressed on the fly by the game as needed. Externally, they can decompressed using various tools, including PsArcTool, or the PlayStation SDK tool PSARC. Even while in game the decompression requires a small amount of work to perform for the CPU and thus, during loading sequences and cases where the game must draw data out of these ".pak" files, the game will try to decompress this data as quickly as possible so it can immediately load it into memory.

This process causes CPU usage spiking, and when CPU usage spikes, it can draw CPU cycles away from other operations on the computer, as well as the rest of the game's tasks, resulting in lag spikes. As mentioned above, the most common example of this in No Man's Sky is when you exit the atmosphere of a planet. The game needs to load in meshes for many space objects, including the Space Anomaly and Space Stations, and this data is drawn out of NMSARC.MeshMisc.pak, and when it does so, it causes an obvious lag spike. All methods for fixing this involve the same sort of solution. Unpacking the file responsible for the spike, and either running it unpacked directly, or repacking the offending files with compression disabled.

Now, it's possible to run No Man's Sky with completely decompressed, unpacked files by unpacking these archives and moving the data out of PCBANKS and into the raw GAMEDATA folder, but this presents a different unique problem. Unpacked, the game no longer has problems with getting the files quickly, but instead, is beholden to being limited by disk file system access. No Man's Sky typically avoids this because most of the data is loaded out of only a handful of files (76 ".pak" files at the time of writing). So it opens one handle to a single file, decompresses and immediately has in memory all of the files it needed from that ".pak".

When you decompress and unpack the game files, rather than accessing only a few individual files, it has to access thousands of files. This is more a consequence of how modern PC file systems work than anything. One example to illustrate the cause is how when you copy a large amount of small files from one disk to another, the graph Windows displays spikes up and down with each tiny file and your overall transfer rate is worse with hundreds of tiny files that take up little space than when transferring fewer larger files between the same two disks. This is because the transfer rate drops when it has to update the file system with the new data, it is waiting on the file system to continue. The problem is similar here with file reads but on a smaller scale. The end result of running the game fully unpacked is that you get fewer lag spikes thanks to not needing decompression, but your overall loading times suffer compared to loading from ".pak" files because of the strain on the file system. The more files the game needs to load, the worse loading times will be.

Fortunately there is a solution. It is possible to repack the game files into archives that have data compression disabled entirely. PSARC (the PlayStation SDK tool) can be used to create ".pak" files that have compression disabled, resulting in ".pak" files that do not require any work to decompress on the fly by the game. Decompressinator automates the decompression and repacking of ".pak" files inside the GAMEDATA\PCBANKS folder, and it also updates the BankSigniture.bin file to match the new decompressed ".pak" files using NMSResign to prevent the game from complaining about the files being tampered with.