Developer Tools●●Solid
Aludel – LLM eval workbench for Phoenix apps
Phoenix LiveView embedding beats switching to LangSmith for Elixir teams.
Niche GemShip It
wood-archer
202mo ago
Back to browse
GitHub Repository
A GUI-first evaluation workbench for local LLMs running on Ollama. Build personal test suites, run sequential evaluations across installed models, visualize results through dashboards, and make keep-or-delete decisions. Think "Postman for local LLM evaluation."
19 starsTypeScript
ModelSweep - Open-Source Benchmarking for Local LLMs
by leonickson·Mar 17, 2026·2 points·0 comments
AI Analysis
●●SolidShip ItNiche GemSlick
Postman for local LLMs with LLM-as-Judge and Elo ratings built in.
Strengths
- •Sequential model testing with automatic VRAM preload/unload management
- •Four evaluation modes including adversarial red team testing scenarios
- •Fully local execution with zero data leaving the machine
Weaknesses
- •Two-day build means bugs and rough edges still present
- •Ollama-only limits broader model runner compatibility
Category
Target Audience
Developers testing local LLMs, Ollama users, AI researchers
Similar To
LangSmith · MLflow · LM Evaluation Harness
Similar Projects
AI/ML●●●Banger
Auto LLM Ranker – Describe a task in English and get ranked models
Task-specific LLM benchmarking beats generic leaderboards that ignore your actual workload.
Big BrainDark HorseZero to One
gauravvij137
302mo ago
Developer Tools●Mid
FC-Eval – CLI to Benchmark Local or Cloud LLMs on Function Calling
AST-based validation for function calling tests, but BFCL already covers this ground.
Ship ItNiche Gem
gauravvij137
302mo ago
AI/ML●●●Banger
LLM Sycophancy Benchmark: Opposite-Narrator Contradictions
Opposite-narrator test catches models agreeing with both sides of same dispute.
Big BrainDark Horse
zone411
302mo ago
Developer Tools●●Solid
A minimal context engine with streaming API
Git-like versioning for prompts running entirely locally with Ollama.
Niche GemSolve My Problem
tonelord
201mo ago
Security●●Solid
AgentToolBench-Code – security benchmark for AI coding agents
Expands corpus to 16 CVE-anchored scenarios to break model ties.
Big BrainNiche Gem
allenwu06
108d ago