Developer Tools●Mid
LLM-JSON-guard – Middleware to auto-repair broken AI outputs
JSON repair middleware; several alternatives (Outlines, instructor, Marvin) already solve this better.
Solve My Problem
harshvermadr30
113mo ago
Back to browse
GitHub Repository
Ultra-low-latency reverse proxy that repairs truncated & malformed JSON in LLM streaming responses (OpenAI, Anthropic, Vertex AI, Bedrock) — fixes JSONDecodeError / serde_json EOF on truncated tool calls.
1 starsRust
Suture – a reverse proxy that repairs truncated JSON in LLM streams
by nabucodonosor·Jun 4, 2026·1 point·0 comments
AI Analysis
●●●BangerBig BrainSolve My ProblemShip It
Fixes truncated JSON on the wire in ~10µs without SDK changes or retries.
Strengths
- •SSE-aware repair of reassembled tool-call arguments across delta events, not raw bytes
- •Transparent gzip/brotli/deflate handling with ~10µs overhead per chunk
- •Supports OpenAI, Anthropic, Vertex AI, and Bedrock without provider-specific code
Weaknesses
- •Only fixes truncation errors, doesn't help with malformed JSON from the model itself
- •Rust implementation may limit adoption for teams without Rust deployment experience
Category
Target Audience
Backend developers building LLM applications with streaming tool calls
Similar To
LiteLLM · OpenRouter
Similar Projects
Developer Tools●●Solid
Partial-zod – streaming JSON parser for LLMs (zero deps, Zod-native)
Zero-dependency Zod streaming parser when zod-stream requires ecosystem buy-in.
SlickNiche Gem
millerjoe
101mo ago
Developer Tools●●●●Gem
Jsonchunk – Parse incomplete JSON from streaming LLM responses
Missing primitive: tolerant JSON parser for streaming LLM output, typed and <1KB.
Zero to OneSolve My ProblemCozy
jbingen
103mo ago
Developer Tools●●●Banger
VectorJSON – O(n) streaming parser to handle LLM JSON outputs
Replaces O(n²) token re-parsing with true O(n) streaming; Vercel SDK does 4K re-parses on 50KB payloads.
Big BrainSolve My ProblemWizardry
teamchong
113mo ago
Developer Tools●●Solid
Agent Firewall – Go proxy to kill LLM death spirals
Wire-protocol circuit breaker for agents when LangSmith costs too much.
Solve My ProblemShip It
wuweiaxin
212mo ago
Developer Tools●●Solid
PrismCat – Local transparent proxy and debugging console for LLM APIs
Subdomain routing means zero /etc/hosts edits—just change base_url and capture everything.
Big BrainSolve My Problem
etgpao
229d ago