HN Showcase
Back to browse
GitHub Repository

A restful API designed to route user prompts to various AI model providers.

175 starsPython

RelayFreeLLM – Free AI Gateway with Auto-Failover (Update))

by markfront·May 6, 2026·1 point·0 comments
Visit ProjectView on HN

AI Analysis

●●●BangerCrowd PleaserSolve My Problem

Eliminates 429 errors by rotating across eight free providers automatically.

Strengths
  • •Four-mode context management includes extractive summarization without LLM calls.
  • •Global provider lock prevents race conditions when multiple requests hit limits.
  • •Drop-in OpenAI-compatible endpoint requires zero code changes to existing apps.
Weaknesses
  • •Relies entirely on free tiers that could vanish or change terms overnight.
  • •No built-in cost tracking or usage analytics beyond availability checks.
Category
AI/ML
Target Audience

Developers building apps on free LLM tiers

Similar To

LiteLLM · Portkey · OpenRouter

Post Description

Author here. Since the last post, RelayFreeLLM now supports NVIDIA's free catalog, adds 4-mode context management (with extractive summarization that doesn't need an LLM), session affinity, output normalization, and a global provider lock to eliminate 429 errors entirely. 8 providers, one endpoint, all free.

Similar Projects