AI/ML●●Solid
I built a small audit layer for LLM-as-judge decisions
Flags unsupported LLM judge verdicts by tracing claims back to evidence.
Niche GemBig Brain
ML0037
201mo ago
Back to browse
GitHub Repository
Find what your AI agent gets wrong — before you have a rubric. Qualitative eval for PMs.
8 starsPython
GEDD – A Systematic Evidence Driven LLM as a Judge Framework
by balasvce2026·Jun 13, 2026·2 points·0 comments
AI Analysis
●●SolidBig BrainNiche Gem
Qualitative eval workflow for PMs when LangSmith and Arize target ML engineers.
Strengths
- •Axial coding and failure codebooks borrow from qualitative research methods, not just metrics.
- •Bridges domain experts and engineers with shared annotation vocabulary instead of abstract scores.
- •Exports session.json handoff with validated prompts ready for MLflow regression gates.
Weaknesses
- •AWS samples repo with 5 stars signals reference implementation, not production tool.
- •LLM-as-a-judge evaluation is crowded with LangSmith, Arize Phoenix, and custom frameworks.
Category
Target Audience
Product managers and ML engineers evaluating AI agents
Similar To
LangSmith · Arize Phoenix · MLflow
Similar Projects
AI/ML●●●Banger
AgentCarousel – behavioral tests for AI agents, with signed evidence
Cryptographically signed test evidence for FDA and EU AI Act compliance is genuinely novel.
Big BrainZero to One
neemsio
201mo ago
AI/ML●Mid
Claude Code skills for building LLM evals
Structured eval workflow for Claude Code when LangSmith and Braintrust already exist.
Niche GemShip It
paulaq
203mo ago
Security●●Solid
When your agent LLM judge become your enemy
Warning labels on retrieved documents actually make attacks five times more successful.
Big Brain
DmitriyBuchilin
102mo ago
AI/ML●●●Banger
I made a small helper for checking model-graded answers
Structurally verifies LLM judge reasoning instead of paying for a second model check.
Big BrainSolve My ProblemDark Horse
ML0037
201mo ago
AI/ML●●Solid
I made a small helper for checking model-graded answers
Flags LLM judge verdicts unsupported by evidence without needing a second model.
Big BrainNiche Gem
ML0037
101mo ago