AI/ML●Mid
AI-Gateway – Open-source semantic caching proxy to reduce LLM API costs
Semantic caching proxy when Helicone and Portkey already dominate.
Ship It
arnab777
103d ago
Back to browse
GitHub Repository
AI-Gateway reverse proxy that uses semantic caching and aims to reduce LLM API bills and token costs by 40-70%.
3 starsGo
Built AI-Gateway reverse proxy to reduce LLM API costs and token burn
by arnab777·Jun 25, 2026·2 points·0 comments
AI Analysis
●MidSolve My ProblemShip It
Semantic caching for LLMs when LiteLLM and Helicone already do this.
Strengths
- •One-click deploy to Railway and Render with Redis included
- •Zero code changes required as a drop-in reverse proxy
- •Live demo available for immediate testing before deployment
Weaknesses
- •Semantic caching already implemented by LiteLLM, Helicone, and LangChain
- •No clear differentiation from established caching solutions in the market
Category
Target Audience
Developers building AI applications with high API costs
Similar To
LiteLLM · Helicone · LangChain Cache
Similar Projects
Developer Tools●●●Banger
AgentReady – Drop-in proxy that cuts LLM token costs 40-60%
Drop-in proxy that cuts GPT token costs 40-60% without changing app code.
Ship ItSolve My ProblemSlick
christalingx
8134mo ago
Developer Tools●●Solid
Personal AI gateway for OpenClaw – tokenomics
OpenAI-compatible proxy with PII masking and token budgets—but LiteLLM, Helicone already do this.
Solve My ProblemBig Brain
crawdog
203mo ago
Infrastructure●Mid
Nexus Gateway – Reduce LLM API Costs Using Semantic Caching
Semantic caching for LLM APIs exists (Anthropic prompt caching, Langchain, Miniplex, vLLM); gateway routing is table stakes.
Ship ItSolve My Problem
Sunnyanand_dev
213mo ago
Developer Tools●●Solid
CacheLens – Local-first cost tracking proxy for LLM APIs
Local budget caps block requests before provider dashboards even update the bill.
Solve My ProblemCozy
stephenlthorn
203mo ago
Developer Tools●●Solid
Lumen–free Real-time LLM token and cost monitor
Wire-protocol proxy means zero code changes to existing LLM clients.
Solve My ProblemSlick
Datagrout
2014d ago