HN Showcase
Back to browse
GitHub Repository

An experiment building a new type of compression algorithm

4 starsRust

A compression tool that beats xz on x86_64 ELFs by 6%

by mohsen1·Mar 2, 2026·3 points·1 comment
Visit ProjectView on HN

AI Analysis

●●●BangerWizardryBig BrainNiche Gem

AI-designed ELF preprocessing beats xz by 6% on 103 real binaries, perfectly reversible.

Strengths
  • •Verified on 103 Alpine binaries with bit-for-bit decompression correctness—not hand-waved benchmarks.
  • •Novel architectural insight: structural entropy separation (branches, relocations, jump tables) into independent LZMA streams unlocks better compression ratios.
  • •Deterministic, reproducible pipeline with no metadata overhead—pure preprocessing, wraps standard LZMA.
Weaknesses
  • •x86_64 ELF only—no ARM, Windows PE, or other architectures limits adoption to Linux server/container scenarios.
  • •Unclear if gains justify complexity in practice: 6% savings across 103 packages is real, but runtime decompression overhead and memory footprint not disclosed.
Target Audience

Systems engineers, binary size optimization specialists, package maintainers, embedded systems developers

Similar To

xz with BCJ filter · ZSTD --ultra · Brotli -11

Post Description

I wanted to see if AI (mostly ChatGPT Pro and Gemini Pro 3.1) could figure out how to compress executable binaries better than existing generic tools without me actually knowing much about compression engineering or ELF internals.

The result is an experiment called fesh. It works strictly as a deterministic pre-processor pipeline wrapping LZMA (xz). The AI kept identifying "structural entropy boundaries" and instructed me to extract near-branches, normalize jump tables, rewrite .eh_frame DWARF pointers to absolute image bases, delta-encode ELF .rela structs with ZigZag mappings, and force column transpositions before compressing them in separated LZMA channels.

Surprisingly, it actually works. The CI strictly verifies that compression is perfectly reversible (bit-for-bit identity match) across 103 Alpine Linux x86_64 packages. According to the benchmarks, it consistently produces smaller payloads than xz -9e --x86 (XZ BCJ), ZSTD, and Brotli across the board—averaging around 6% smaller than maximum XZ BCJ limits.

I honestly have no idea how much of this is genuinely novel versus standard practices in extreme binary packing (like Crinkler/UPX).

Repo: https://github.com/mohsen1/fesh

Does this architecture have any actual merits for standard distribution formats, or is this just overfitting the LZMA dictionary to Alpine's compiler outputs? I'd love to hear from people who actually understand compression math.

Similar Projects