AI/ML●●Solid
Reproducible open-source STT API benchmarks with full methodology
Fixes WER scores by normalizing '$50' and 'fifty dollars' as equivalent.
Solve My ProblemCozy
jilijeanlouis
113mo ago
Back to browse
GitHub Repository
Benchmark transcription APIs against real meeting audio. Measure WER, diarization, latency, and cost.
3 starsPython
Open-source benchmark for transcription APIs on meeting audio
by eyepaqio·Apr 3, 2026·2 points·0 comments
AI Analysis
●●SolidSolve My ProblemNiche Gem
Tests Deepgram and AssemblyAI on actual crosstalk instead of clean audiobook samples.
Strengths
- •Targets meeting-specific artifacts like crosstalk and screen-share audio bleed in real tests.
- •CLI skips missing API keys instead of crashing on configuration errors.
- •Measures cost per hour alongside technical metrics for real budgeting decisions.
Weaknesses
- •Only supports four adapters; missing Google Cloud Speech and Azure AI.
- •Requires manual ground-truth JSON setup for custom audio samples.
Category
Target Audience
Backend developers, CTOs choosing transcription infrastructure
Similar To
Hugging Face Spaces · MLPerf
Similar Projects
Developer Tools●●●Banger
Trying to fix the web scraping industry's benchmark problem
Finally exposes vendor BS by ranking scrapers on hard targets like Amazon and Cloudflare.
Big BrainNiche GemShip It
rohitshenoy
1804d ago
Productivity●●Solid
Dikaletus – meeting recording and transcription using Mistral AI
Dual-channel audio capture via PulseAudio is a genuinely useful detail most wrappers miss.
Niche GemCozy
phillc73
202mo ago
Developer Tools●●Solid
UFFS, a Rust NTFS search engine benchmarked against Everything
Direct MFT parsing beats Everything by 2.8× in 30/30 benchmark cells.
WizardrySolve My Problem
rnio
1112d ago
Developer Tools●●●Banger
Mdarena – Benchmark your Claude.md against your own PRs
Mining your own PRs as benchmarks beats generic SWE-bench tasks for agent config tuning.
Big BrainSolve My Problem
hudsongr
2243mo ago
AI/ML●●Solid
Keyterm Filtering for Voice AI
Cuts keyterm hallucinations by 60% before audio hits Deepgram.
Solve My ProblemNiche Gem
mayowa_osibodu
302mo ago