I wanted to know how much space we could save by packing files differently. The useful answer came from a boring log file: it got much smaller, but the smallest version took longer to put back together.
That’s where choosing a compressor gets interesting. Are you putting a box away for a year, or opening it every few seconds?
TL;DR
Text and repeated records compressed well in this test. Already-packed video barely changed. I’d try Zstandard first for a service that needs files back quickly. LZ4 was quicker on the logs but kept a larger file; bzip2 saved the most bytes but took longer to restore them. The file itself matters as much as the algorithm’s name.
This grew out of checking Helumi’s storage pitch. These are separate compression tests, not tests of its Mac app or infrastructure.
A smaller way to say the same thing
Imagine sending a friend AAAAAAAAAAAA. You could agree that 12:A means “write A twelve times.” You’ve sent four characters instead of twelve, and your friend can still recover the message.
That’s the basic idea: find a shorter way to describe what repeats. Real formats need extra rules, so this little example isn’t a working archive format for every message. Other compressors can reuse whole words or longer sequences, or predict which byte is likely to come next.
A file with lots of repetition gives them something to work with. A file that’s already been packed well may have very little left.
The same file, four different answers
Our generated log file started at 846,281 bytes. Here’s what four settings did to it. “Compress” means packing the bytes; “restore” means getting the original bytes back. These times don’t include opening an app, reading the file from disk or downloading it.
| Algorithm | Stored bytes | Compress | Restore |
|---|---|---|---|
| LZ4 level 0 | 245,312Largest | 0.92 msFastest | 0.71 msFastest |
| Zstandard level 3 | 145,322Middle | 5.89 msFast | 1.87 msFast |
| bzip2 level 9 | 102,414Smallest | 121.93 msSlower | 94.61 msSlowest |
| Brotli quality 11 | 107,701Small | 1,707.62 msSlowest | 4.01 msMiddle |
bzip2 saved 87.9% of the original space. That’s useful if I want a small archive that I won’t open often. But Zstandard restored the same source data in 1.87 ms, compared with bzip2’s 94.61 ms. If people are waiting for files, I’d care about that difference.
LZ4 went further on speed and gave up more space. Brotli 11 spent about 1.71 seconds compressing the logs and still didn’t make the smallest file. More time spent looking for patterns doesn’t guarantee a better result.
The size and time chart
The colours compare these four settings on this one file. They’re not promises about every file or computer. One millisecond is a thousandth of a second, and these are medians from three desktop runs. The full results are below.
The file can change the answer completely
Take the two PNGs. A smooth gradient still had patterns to exploit: Zstandard 19 shrank it from 9,619 to 7,099 bytes. The noise PNG grew with every archive setting we tried. Same extension, very different contents.
The PDFs repeated text and an image over eight pages, so they had plenty to reuse. Our artificial JPEG also compressed again because it contained smooth areas and a large flat rectangle. I wouldn’t use that result to promise the same saving on someone’s camera collection.
The video was the other useful result. The H.264 file started at 531,451 bytes. Even its smallest archive was 529,601 bytes, a saving of just 1,850 bytes, or 0.35%. The video encoder had already done much of the packing.
We could make it much smaller by re-encoding it at lower quality: 152,958 bytes, about 71.2% less, in one 2.87-second run. But the picture changed. That’s a different choice from keeping an exact backup of the recording.
For the generated illustration, lossless WebP kept the same RGB pixels in 6,244 bytes, against PNG’s 9,619. It took longer to encode. So I’d also ask whether the file is in a suitable format before spending more effort on an archive around it.
Where I’d use this
For a file-serving path, I’d start by testing a low Zstandard setting. For very tight processing time, I’d try LZ4. For an archive stored once and rarely restored, I’d compare the denser options, including bzip2. Those are starting points from these results, not a universal ranking.
There’s another catch for cloud files: sometimes you only need a small piece. Packing the whole file together can make that piece harder to reach. Our small prototype packs separate blocks and keeps an index, like a book with page numbers. It uses existing techniques, including the idea behind Zstandard’s seekable format.
That lets you restore a block without starting at the beginning of the file. Smaller blocks make small reads cheaper, but add bookkeeping and can lose repetition across block boundaries. I’d measure that tradeoff with real project files before building a service around it.
A file system is another layer again. It decides how files occupy storage. An archive saving 100 bytes may still use the same number of disk blocks. We measured file bytes and process memory here, not physical disk savings or a ranking of file systems.
The experiment is small and deliberately artificial. It helps show what to look for. For an actual storage decision, I’d use the files people really open and measure how long they wait to get them back.
The measurements, if you want them
All ten files, with original size, compressed size and time
These are the smallest outputs among eleven archive settings, excluding the block prototype and media conversions. Times belong to the displayed setting. XZ 9 ties XZ 6 on the PPM and JPEG; Zstandard 22 ties level 19 on the PNG and video; Brotli 11 ties quality 5 on noise and random bytes. The table uses the lower setting’s timings for each tie. Keeping noise and random bytes uncompressed is smaller than any of these archives.
| File | Smallest archive setting | Original bytes | Compressed bytes | Saved | Encode ms | Decode ms |
|---|---|---|---|---|---|---|
guide.txt | Brotli 11 | 2,344 | 869 | 62.9% | 3.19 | 0.05 |
events.jsonl | bzip2 9 | 846,281 | 102,414 | 87.9% | 121.93 | 94.61 |
gradient.ppm | XZ 6 | 1,179,663 | 77,168 | 93.5% | 180.22 | 10.78 |
gradient.png | Zstandard 19 | 9,619 | 7,099 | 26.2% | 2.09 | 0.03 |
gradient.jpg | XZ 6 | 24,294 | 11,708 | 51.8% | 12.39 | 0.80 |
report-plain.pdf | Brotli 11 | 63,979 | 17,741 | 72.3% | 54.89 | 0.41 |
report-compressed.pdf | Brotli 11 | 50,166 | 20,881 | 58.4% | 58.20 | 0.27 |
motion.mp4 | Zstandard 19 | 531,451 | 529,601 | 0.35% | 158.37 | 0.28 |
noise.png | Brotli 5 | 788,090 | 788,095 | 5 bytes larger | 10.40 | 1.18 |
random.bin | Brotli 5 | 524,288 | 524,293 | 5 bytes larger | 5.52 | 0.88 |
The saving is (original - compressed) / original * 100. Archive headers count. The comparison plots express stored bytes as a percentage of the original, so 100% means no saving.
All eleven log settings, including memory use
The input is the same 846,281-byte log file in every row. RAM is the operating system’s peak working set for the whole process, including Python, libraries, buffers, warm-up and verification. It isn’t isolated codec memory. One MiB is 1,048,576 bytes.
| Setting | Compressed bytes | Encode ms | Decode ms | Peak process RAM MiB |
|---|---|---|---|---|
| gzip 6 | 145,042 | 15.82 | 3.06 | 29.6 |
| gzip 9 | 136,773 | 59.03 | 3.09 | 29.2 |
| bzip2 9 | 102,414 | 121.93 | 94.61 | 34.2 |
| XZ 6 | 111,700 | 425.53 | 8.22 | 51.8 |
| XZ 9 | 111,700 | 440.10 | 9.60 | 99.8 |
| Brotli 5 | 134,273 | 21.24 | 3.98 | 31.1 |
| Brotli 11 | 107,701 | 1,707.62 | 4.01 | 47.9 |
| Zstandard 3 | 145,322 | 5.89 | 1.87 | 29.5 |
| Zstandard 19 | 113,657 | 599.02 | 1.67 | 45.3 |
| Zstandard 22 | 113,657 | 513.80 | 1.33 | 45.5 |
| LZ4 0 | 245,312 | 0.92 | 0.71 | 29.6 |
XZ 9 used almost twice XZ 6’s process memory for identical output. Zstandard 22 happened to finish sooner than 19, but three runs without controlled background load don’t prove that the higher setting is generally faster.
What we generated and how the test ran
The ten files were original prose, 6,000 synthetic log records, a gradient in PPM, PNG and JPEG, a noise PNG, two eight-page PDFs, a six-second video and seeded random bytes. The noise and random bytes are controls, not photographs or an encryption test. The PDFs repeat content; one uses stream compression and the other doesn’t. FFmpeg generated the H.264 video with testsrc2, CRF 23.
We tested gzip 6/9, bzip2 9, XZ/LZMA2 6/9, Brotli 5/11, Zstandard 3/19/22 and LZ4 0. Three block configurations bring the total to 140 file/setting pairs. Every warm-up restored exact bytes; every measured decode passed SHA-256 verification.
Each pair ran in a fresh process on Windows 11, Python 3.12 and a Ryzen 7 5800H with about 16 GiB RAM. Runs were sequential, with one warm-up and three measurements. Times exclude file reads, process startup and hash checks. Small synthetic inputs and uncontrolled background load limit what the timings can tell us.
The explanations of codec behaviour are interpretations, not traces of their internal decisions. bzip2’s manual describes block sorting; Brotli’s specification describes its static dictionary and string references. Neither guarantees a win on an extension.
The image and video conversions
The source PPM contains 1,179,648 RGB bytes and a 15-byte header, totalling 1,179,663 bytes. Image times are three-run medians of the encoder call, without a separate warm-up.
| Image encoding | Output bytes | Encode ms | Same RGB pixels? |
|---|---|---|---|
| PNG, level 9 | 9,619 | 26.03 | Yes |
| WebP, lossless, method 6 | 6,244 | 497.27 | Yes |
| JPEG, quality 60 | 16,274 | 2.04 | No |
| JPEG, quality 90 | 24,294 | 1.92 | No |
WebP saved 35.1% against PNG while restoring the same RGB pixels. That doesn’t restore the PNG file’s original bytes or prove preservation of arbitrary metadata. JPEG quality 60 was larger than PNG on this illustration despite changing the pixels. The XZ archive of the raw PPM was 77,168 bytes, much larger than either lossless image encoding.
The video transcode used CRF 35 and had average YUV PSNR of 35.63 dB against the decoded source. PSNR measures numerical differences, not a person’s judgement of quality. Its 2.87-second timing includes launching FFmpeg and file I/O, unlike the archive timings. It is a lossy conversion of an already lossy source, not exact file recovery. The media results keep those checks.
The block experiment and file-system choices
The prototype tries Zstandard 3 on each block and keeps whichever is smaller: the compressed result or the original bytes. Its index counts toward size. It tests 4 KiB, 64 KiB and 1 MiB blocks, with full-file and individual-block recovery checks. It loads the whole container into memory; it isn’t a remote-range-read or bounded-memory benchmark.
The logs occupied about 16.7% with 64 KiB blocks, compared with 17.2% for one Zstandard 3 frame. A reset can sometimes help a fast match finder, but this doesn’t establish a general advantage for chunking. The 4 KiB container added 1,684 bytes to the random file. Trying compression before keeping raw bytes still spends CPU time. A sample-based skip rule wasn’t tested.
The read-cost panel is a model, not network traffic. An aligned 4 KiB read that decodes a whole 1 MiB block processes 256 times the requested bytes. Compression, caching, encryption boundaries and reads crossing blocks change actual costs.
For Btrfs, I’d evaluate its built-in Zstandard option, starting at the documented default level 3. It compresses smaller extents, and enabling it doesn’t automatically recompress existing files. For OpenZFS, I’d compare LZ4’s low overhead with Zstandard’s density. Record sizes and sector rounding matter. Neither file system, nor APFS, NTFS or ext4, was benchmarked here.
For opaque encrypted objects, I’d keep compression on suitable plaintext before client-side encryption, with a defined decoder and index. Proper encryption is expected to hide useful patterns. We didn’t measure encryption or network costs, deduplication, concurrency, battery use or recovery after corruption.
What about newer compression research?
There are interesting results beyond these codecs. The cost often moves into the predictions a decoder has to reproduce. These are developer and paper results checked on 3 November 2026; we didn’t run them locally or mix them into our charts.
| System | Reported result or approach | Cost that matters |
|---|---|---|
| CMIX | Mixes predictive models to optimise lossless compression ratio | Its author recommends at least 32 GB RAM and explicitly trades CPU and memory for size |
| ts_zip | Reports enwik8 shrinking from 100 MB to 13.83 MB | Requires model resources; its developer quotes up to 1 MB/s on an RTX 4090 and warns about experimental format compatibility |
| Nacrith, February 2026 | Reports 11.74 MB for enwik8, combining a small language model with adaptive predictors | The paper requires about 500 MB of shared weights and reports slow prediction on its older test GPU |
| Chained lightweight predictors, April 2026 | Reports ratios near PAC with 1.2 to 6.3 times its encoding throughput | Consumer-GPU results in an under-review paper; adaptive prediction also requires work at decode time |
| NeuralLVC, revised September 2026 | Uses temporal prediction and masked diffusion for lossless video | Reports about 0.06 frames per second end-to-end on CIF video using a GH200 at one tested setting |
Nacrith increases probability-table precision to waste less coding range and reuses its key-value cache instead of processing old context again. The chained-predictor work reduces model work when higher-order prediction doesn’t help. Weights, tokenizers, numeric behaviour and state still have to agree at both ends. Shared model weights can be spread across many files, but must count in a self-contained archive’s cost.
Beating a low-order byte entropy estimate doesn’t break Shannon’s source-coding limit; a richer model may see dependencies that estimate misses. JPEG XL is another narrower option worth testing: it supports lossless JPEG recompression with original-file reconstruction. It’s established, not a new 2026 discovery, and we didn’t test it here.
Download the files or repeat the experiment
The sample ZIP, manifest, CSV, full JSON and environment record keep the inputs, hashes and all the measurements.
With FFmpeg installed, download the benchmark script and requirements, then run:
python -m venv .venv
# Activate the environment for your operating system.
python -m pip install -r requirements.txt
python benchmark.py --output workThe figure script also needs Matplotlib 3.11.2:
python -m pip install matplotlib==3.11.2
python visualize.py work figures
# Keep the existing media measurements when redrawing figures:
python visualize.py work figures --figures-onlyEncoder versions and builds can change generated bytes. Use the supplied ZIP and hashes for these exact inputs. This is a small diagnostic experiment, not a survey of every known algorithm; timings will vary on another run.