Back to browse
GitHub Repository

High-throughput long-context LLMs. Scaling context via RandNLA and massive vocab capacity through MAXIS Loss and Fisher-SVD.

28 starsPython

MaximusLLM, Breaking transformer's O(N^2) and O(V) scaling bottlenecks

by yousef_g·Mar 13, 2026·1 point·0 comments

AI Analysis

●●SolidBig BrainBold Bet

Claims 17.5x training speedup with Matryoshka embeddings for native RAG.

Strengths
  • MAXIS Loss compresses embeddings to 64 dimensions for faster vector search
  • RandNLA Attention enables constant-time throughput regardless of sequence length
  • Papers on SSRN and HuggingFace models available for independent verification
Weaknesses
  • Only 2 GitHub stars means limited community validation and adoption
  • Bold 17.5x speedup claims need independent reproduction by other researchers
Category
Target Audience

ML researchers and engineers training long-context models

Similar To

Mamba · Linear Attention · RetNet

Similar Projects

AI/ML●●●Banger

MaximusLLM – Train 262k-vocab LLMs on a single 16GB GPU

Ghost Logit math bypasses 262k vocab OOM without materializing full matrices.

Big BrainWizardryZero to One
yousef_g
202mo ago