AI/ML●●Solid
2500 vision benchmarks / evals for Vision Language Models
Daily arXiv scraping with Claude classification beats manual curation.
Niche GemBig Brain
zakariaelhjouji
101mo ago
Back to browse
GitHub Repository
Benchmark any LLM against your data. Pick the best model, then make it better.
4 starsPython
Verdict – model evals on your own data, not someone else's benchmark
by agunapal·May 7, 2026·2 points·0 comments
AI Analysis
●●SolidSolve My ProblemSlick
Run your own data against GPT-5 and Llama to pick the winner.
Strengths
- •Pluggable metrics include both ROUGE scores and LLM judges.
- •Unified CLI interface supports OpenAI, Anthropic, and local Ollama.
- •Focuses on custom data rather than generic public benchmarks.
Weaknesses
- •Crowded category with many existing eval frameworks like RAGAS.
- •Another Python wrapper around standard provider APIs.
Category
Target Audience
ML engineers and prompt engineers
Similar To
RAGAS · LangSmith · DeepEval
Similar Projects
AI/ML●●Solid
LLMadness – March Madness Model Evals
Claude Opus spent $59.55 versus MiMo-Flash at $0.39 for identical bracket predictions.
Dark HorseBig Brain
rjkeck2
522mo ago
Data●●Solid
A seedable stream shuffler modeled as a roundabout network (Python)
Shuffling metaphor with real math—97.5% Fisher-Yates quality but solves no obvious problem over standard random.
Big BrainNiche GemWizardry
velocitatem
103mo ago
AI/ML●●Solid
Which AI model is best for real data analysis?
Transparent benchmark for data analysis LLMs with verifiable notebook artifacts.
Big BrainNiche Gem
pplonski86
211mo ago
AI/ML●●Solid
Speechos – Benchmark 25 speech AI models locally, no cloud needed
Side-by-side model comparison eliminates guessing which speech engine fits your hardware.
Dark HorseSolve My Problem
hamuf
113mo ago
Developer Tools●Mid
Skylos – A Python dead code finder benchmarked against 9 libraries
Benchmarked dead code finder across FastAPI, Pydantic, Flask—but Vulture, Bandit already solve this.
Solve My Problem
duriantaco
312mo ago