Infrastructure●Mid
Mantis, A self-hosted LLM gateway
Another LLM gateway when LiteLLM and Portkey already dominate the space.
Ship It
rizsyed1
5013d ago
Back to browse
GitHub Repository
Self-hosted LLM gateway. Cost effective, deterministic, and fast. Secure and private by default.
0 starsGo
Foreman, a self-hosted LLM gateway for cost aware model routing
by AndrewLiu96·Jul 8, 2026·2 points·0 comments
AI Analysis
●●SolidSolve My ProblemNiche Gem
Cache-aware LLM routing that doesn't burn prompts to save pennies.
Strengths
- •Prompt cache awareness prevents burning cached tokens to save pennies
- •Detailed trace headers show every routing decision and rejected alternatives
- •Single Go binary, self-hosted keeps provider keys inside your network
Weaknesses
- •LLM gateway category is crowded with LiteLLM, Portkey, Helicone
- •Zero GitHub stars at launch suggests very early, untested stage
Category
Target Audience
Engineering teams using multiple LLM providers, cost-conscious AI deployments
Similar To
LiteLLM · Portkey · Helicone
Similar Projects
Infrastructure●Mid
Nexus Gateway – Reduce LLM API Costs Using Semantic Caching
Semantic caching for LLM APIs exists (Anthropic prompt caching, Langchain, Miniplex, vLLM); gateway routing is table stakes.
Ship ItSolve My Problem
Sunnyanand_dev
214mo ago
AI/ML●●Solid
Nexa-Gauge – LLM eval framework, now with self-hosted model support
Cache-aware LLM eval with self-hosted model support beats Ragas on flexibility.
Solve My ProblemSlick
Sardhendu
201mo ago
Infrastructure●●Solid
UnifyRoute – Self-hosted OpenAI-compatible LLM gateway with failover
Drop-in OpenAI API gateway with failover—LiteLLM does this but this has a dashboard.
Solve My ProblemSlick
unifyroute
114mo ago
Infrastructure●●Solid
LunarGate – a self-hosted OpenAI-compatible LLM gateway
Go gateway with circuit breakers, but auth isn't production-ready yet.
Ship ItNiche Gem
jmartenka
303mo ago
Infrastructure●●Solid
Selixes - Self-hosted LLM failover gateway, budget caps and PII
Sub-15ms failover and atomic budget caps, but LiteLLM and Portkey exist.
Solve My ProblemShip It
selixes
3011d ago