Security●●Solid
An open source benchmark for prompt-injection detectors
Dual-axis measurement comparing detectors at same catch rate, not arbitrary thresholds.
Big Brain
gugit
2022d ago
Back to browse
Prompt injection detector beats ProtectAI by 19% accuracy, 8.9x smaller
by Karan047·Apr 8, 2026·3 points·2 comments
AI Analysis
●●●BangerDark HorseSolve My Problem
Beats ProtectAI by 19% accuracy and runs 9x smaller on CPU.
Strengths
- •INT8 ONNX quantization enables 101ms CPU inference without GPU dependency.
- •95.84% precision drastically reduces false positives blocking legitimate user prompts.
- •Benchmarked against ProtectAI on independent Qualifire dataset, not just training data.
Weaknesses
- •Requires contact info sharing to access model files on Hugging Face, adding friction.
- •No hosted API or managed service offered, just raw weights for self-hosting.
Category
Target Audience
LLM application developers, AI security engineers
Similar To
ProtectAI · Lakera Guard · Rebuff
Post Description
ProtectAI's deberta-v3 is the most downloaded prompt injection classifier on HuggingFace. We benchmarked directly against it on Qualifire's independent dataset (excluded from our training).
* 91.68% vs 72.28% accuracy
* 95.84% vs 65.33% precision — they block 1 in 3 legitimate users
* 83MB vs 739MB ONNX size
* 101ms vs 646ms on CPU
Similar Projects
Security●●Solid
LLMSecure – prompt injection detection, no signup
Dynamic LLM sandbox analysis detects injections that static pattern matching tools miss.
Solve My ProblemSlick
eliadmualem
213mo ago
Security●Mid
Wardstone – Prompt injection and jailbreak detection API
Another guardrail API competing with Lakera, but claims sub-30ms latency.
SlickShip It
jaaackrl
104mo ago
Security●●Solid
Wolf Defender, a open-weight prompt-injection detection model
Outperforms existing open-source injection detectors on ProtectAI and Qualifire benchmarks.
Niche GemDark Horse
patronusprotect
204mo ago
Security●●Solid
SentryGuard – detect Agentjacking prompt injection in Sentry events
Scans Sentry events for Agentjacking before your AI agent reads them.
Solve My ProblemShip It
tangxia123
1027d ago
Security●●Solid
ClawGuard – Detect 42 prompt injection patterns in <10ms
42 prompt injection patterns detected in under 10ms with zero dependencies.
Niche GemShip It
joergmichno
234mo ago