Prompt injection detector beats ProtectAI by 19% accuracy, 8.9x smaller

Name: Prompt injection detector beats ProtectAI by 19% accuracy, 8.9x smaller
Availability: InStock
Author: Karan047

by Karan047·Apr 8, 2026·3 points·2 comments

Visit Project View on HN

AI Analysis

●●●BangerDark HorseSolve My Problem

Beats ProtectAI by 19% accuracy and runs 9x smaller on CPU.

Strengths

•INT8 ONNX quantization enables 101ms CPU inference without GPU dependency.
•95.84% precision drastically reduces false positives blocking legitimate user prompts.
•Benchmarked against ProtectAI on independent Qualifire dataset, not just training data.

Weaknesses

•Requires contact info sharing to access model files on Hugging Face, adding friction.
•No hosted API or managed service offered, just raw weights for self-hosting.

Post Description

ProtectAI's deberta-v3 is the most downloaded prompt injection classifier on HuggingFace. We benchmarked directly against it on Qualifire's independent dataset (excluded from our training).

* 91.68% vs 72.28% accuracy

* 95.84% vs 65.33% precision — they block 1 in 3 legitimate users

* 83MB vs 739MB ONNX size

* 101ms vs 646ms on CPU