Infrastructure●●Solid
AI Cost Firewall – OpenAI-compatible gateway with semantic caching
LLM gateway with Redis + Qdrant caching, but LiteLLM does this.
SlickShip It
vcaluser
112mo ago
Back to browse
GitHub Repository
A transparent, 100%-local semantic cache for LLM APIs — drop-in proxy, one line to integrate, written in Rust
0 starsRust
Cachet – A drop-in semantic cache for LLM APIs, 100% local, in Rust
by Abhi_2112·Jun 23, 2026·5 points·0 comments
AI Analysis
●●SolidBig BrainShip It
Semantic caching without a vector DB—just swap your base URL.
Strengths
- •Single binary drop-in proxy, no vector database dependencies
- •Real-time dashboard showing hit rates and dollars saved
- •Pure Rust with ~47MB distroless Docker image
Weaknesses
- •Semantic caching for LLMs already exists in multiple tools
- •Only supports OpenAI and Anthropic-compatible APIs currently
Category
Target Audience
Backend developers, AI engineers
Similar To
CacheLLM · LLMCache · Portkey
Similar Projects
AI/ML●Mid
AI-Gateway – Open-source semantic caching proxy to reduce LLM API costs
Semantic caching proxy when Helicone and Portkey already dominate.
Ship It
arnab777
101d ago
Infrastructure●Mid
Nexus Gateway – Reduce LLM API Costs Using Semantic Caching
Semantic caching for LLM APIs exists (Anthropic prompt caching, Langchain, Miniplex, vLLM); gateway routing is table stakes.
Ship ItSolve My Problem
Sunnyanand_dev
213mo ago
AI/ML●●●Banger
CacheCore – semantic agent caching with dependency invalidation
Semantic caching with dependency invalidation beats standard Redis wrappers for agent costs.
Big BrainSolve My Problem
fabriziorocco
241mo ago
Developer Tools●●●Banger
Isartor – Pure-Rust prompt firewall, deflects 60-95% of LLM traffic
Local semantic caching cuts LLM costs without changing your code.
Solve My ProblemSlick
zippode
312mo ago
AI/ML●●●Banger
SCP – A protocol that drops LLM API calls to zero in 60fps physics loop
Subsumption Architecture revival cuts LLM calls with pattern cache misses.
Big BrainWizardry
srk0102
102mo ago