Back to browse
GitHub Repository

Train the smallest LM you can that fits in 16MB. Best model wins!

5,124 starsPython

GolfStudent v2 - 24M-param LLM in 15MB using GPTQ-lite + Muon

by whitestone1121·Mar 25, 2026·1 point·0 comments

AI Analysis

●●SolidBig BrainWizardry

24M params in 15MB using GPTQ-lite and Muon optimizer for OpenAI's Parameter Golf challenge.

Strengths
  • Aggressive constraint optimization combining GPTQ-lite quantization with Muon+EMA.
  • Schedule-Free optimization and value residuals push parameter efficiency boundaries.
  • Part of OpenAI-sponsored challenge with $1M compute credits for participants.
Weaknesses
  • Challenge PR submission, not a standalone usable product or library.
  • Niche appeal limited to ML researchers interested in parameter-constrained training.
Category
Target Audience

ML researchers, model optimization engineers, constraint optimization enthusiasts

Similar To

NanoGPT Speedrun · Parameter Golf Challenge

Similar Projects