Infrastructure●●Solid
Wyolet Relay – high throughput, open source LLM router
Self-hosted LLM gateway pooling keys across providers for failover and cost tracking.
Ship ItSolve My Problem
aaliboyev
307d ago
Back to browse
Mantis, A self-hosted LLM gateway
by rizsyed1·Jun 26, 2026·4 points·0 comments
AI Analysis
●MidShip It
Another LLM gateway when LiteLLM and Portkey already dominate the space.
Strengths
- •Single Terraform command deploys the entire AWS stack.
- •AWS Bedrock guardrails integration for sensitive data masking.
Weaknesses
- •LLM gateway category already has LiteLLM, Portkey, Helicone with more adoption.
- •No clear differentiation beyond AWS-native deployment.
Category
Target Audience
Small teams building multi-LLM applications who want self-hosted control
Similar To
LiteLLM · Portkey · Helicone
Post Description
Hey HNers - Riz here.
I got together with a few guys and we built an LLM gateway.
It's designed for small teams working on early-stage products, and can be deployed to AWS using a single command (i.e. `mantis deploy`).
It's self-hosted, and is designed to belong to you.
Similar Projects
Infrastructure●●Solid
UnifyRoute – Self-hosted OpenAI-compatible LLM gateway with failover
Drop-in OpenAI API gateway with failover—LiteLLM does this but this has a dashboard.
Solve My ProblemSlick
unifyroute
113mo ago
Infrastructure●●Solid
LunarGate – a self-hosted OpenAI-compatible LLM gateway
Go gateway with circuit breakers, but auth isn't production-ready yet.
Ship ItNiche Gem
jmartenka
302mo ago
Infrastructure●●Solid
Styx, Open-source AI gateway with intelligent auto-routing (MCP-native)
First gateway with native MCP server—connect Claude Code or Cursor in one command.
SlickSolve My Problem
timmx7
203mo ago
Infrastructure●Mid
Nexus Gateway – Reduce LLM API Costs Using Semantic Caching
Semantic caching for LLM APIs exists (Anthropic prompt caching, Langchain, Miniplex, vLLM); gateway routing is table stakes.
Ship ItSolve My Problem
Sunnyanand_dev
213mo ago
AI/ML●Mid
Consciousness Gateway – AI routing with consciousness-first alignment
Product Algebra routing plus an explicit 'dharma' pipeline (no-self regularization, entropy/mindfulness metrics, compassion and ethos scores) is a strikingly specific approach — it moves beyond cost/capability heuristics into cross-modal interaction scoring and reputation-driven incentives. There's real engineering here (1s perception loop, SQLite memory, Telegram UX, multi-provider SDK support), but the repo reads young and claim-heavy: I want reproducible benchmark artifacts, links from the code to the cited 439-model experiments, and clearer deployment/security guidance before trusting it for critical workloads.
Bold BetBig Brain
AIconscious
204mo ago