Back to browse
GitHub Repository

An experiment building a new type of compression algorithm

4 starsRust

I used LLMs to build a compression tool that beats xz on x86_64 ELFs

by mohsen1·Mar 2, 2026·4 points·3 comments

AI Analysis

●●●BangerWizardryBig BrainBold Bet

LLM-guided structural preprocessing beats xz -9e on 103/103 Alpine binaries tested.

Strengths
  • 103/103 perfect benchmark wins with bit-for-bit reversibility proves correctness rigorously
  • Deterministic preprocessing pipeline (delta-encoding, MoE mapping, ZigZag structs) is architecturally clever
  • Addresses real pain: binary size matters for containers, embedded systems, distribution bandwidth
Weaknesses
  • Niche use case: x86_64 ELF only, no ARM/Windows/generic binary support yet
  • Long-term viability unclear: relies on xz/LZMA; if xz evolves, fesh must adapt
Target Audience

Systems engineers, embedded developers, container/binary distribution optimization specialists

Similar To

UPX · xz · Brotli

Post Description

I wanted to see if AI (mostly ChatGPT Pro and Gemini Pro 3.1) could figure out how to compress executable binaries better than existing generic tools without me actually knowing much about compression engineering or ELF internals.

The result is an experiment called fesh. It works strictly as a deterministic pre-processor pipeline wrapping LZMA (xz). The AI kept identifying "structural entropy boundaries" and instructed me to extract near-branches, normalize jump tables, rewrite .eh_frame DWARF pointers to absolute image bases, delta-encode ELF .rela structs with ZigZag mappings, and force column transpositions before compressing them in separated LZMA channels.

Surprisingly, it actually works. The CI strictly verifies that compression is perfectly reversible (bit-for-bit identity match) across 103 Alpine Linux x86_64 packages. According to the benchmarks, it consistently produces smaller payloads than xz -9e --x86 (XZ BCJ), ZSTD, and Brotli across the board—averaging around 6% smaller than maximum XZ BCJ limits.

I honestly have no idea how much of this is genuinely novel versus standard practices in extreme binary packing (like Crinkler/UPX).

Repo: https://github.com/mohsen1/fesh

Does this architecture have any actual merits for standard distribution formats, or is this just overfitting the LZMA dictionary to Alpine's compiler outputs? I'd love to hear from people who actually understand compression math.

Similar Projects