AI/ML●●●Banger
Pencil Puzzle Bench – LLM Benchmark for Multi-Step Verifiable Reasoning
62k puzzle benchmark reveals reasoning depth, cost variance, and stark US vs China model gaps.
Big BrainCrowd PleaserSolve My Problem
bluecoconut
503mo ago
Back to browse
JazzBench, an LLM reasoning benchmark using jazz improvisation
by mikerubini·Jun 9, 2026·2 points·0 comments
AI Analysis
○PassBig Brain
Interesting eval philosophy, but this is a blog post with no shipped code or tool.
Strengths
- •Novel framing of evals around taste and constraint satisfaction
- •Charlie Parker solos provide genuine ground truth for scoring
Weaknesses
- •No repository, no code, no way to actually run JazzBench
- •Spec and manifesto without implementation to evaluate
Category
Target Audience
LLM researchers, AI engineers building evals
Similar Projects
AI/ML●●●Banger
Republic of Agents: Benchmark for Social Reasoning in LLMs
Mafia-as-benchmark with learning-between-batches mechanism; public, inspectable sessions.
Zero to OneBig BrainWizardry
kkonstantin
103mo ago
AI/ML●●●Banger
ModelX – Prediction Exchange for LLMs
Sealed-batch auctions remove inference speed bias from LLM trading benchmarks.
Big BrainDark Horse
Entropnt
401mo ago
AI/ML●●●Banger
Reducing LLM input tokens by 70%
Cuts token costs 70% with receipts proving no accuracy drop on hard evals.
Zero to OneSolve My Problem
Jbunga
563327d ago
AI/ML●●●Banger
Auto LLM Ranker – Describe a task in English and get ranked models
Task-specific LLM benchmarking beats generic leaderboards that ignore your actual workload.
Big BrainDark HorseZero to One
gauravvij137
303mo ago
AI/ML●Mid
CivBench a long-horizon AI benchmark for multi-agent games
Civilization matches expose model divergence that static benchmarks miss—but it's a spectacle, not a measurement.
Rabbit HoleBig Brain
mbh159
12243mo ago