HN Showcase
Back to browse
GitHub Repository

A restful API designed to route user prompts to various AI model providers.

118 starsPython

RelayFreeLLM – Free AI Gateway with Auto-Failover (Update))

by markfront·May 6, 2026·1 point·0 comments
Visit ProjectView on HN

AI Analysis

●●●BangerCrowd PleaserSolve My Problem

Eliminates 429 errors by rotating across eight free providers automatically.

Strengths
  • •Four-mode context management includes extractive summarization without LLM calls.
  • •Global provider lock prevents race conditions when multiple requests hit limits.
  • •Drop-in OpenAI-compatible endpoint requires zero code changes to existing apps.
Weaknesses
  • •Relies entirely on free tiers that could vanish or change terms overnight.
  • •No built-in cost tracking or usage analytics beyond availability checks.
Category
AI/ML
Target Audience

Developers building apps on free LLM tiers

Similar To

LiteLLM · Portkey · OpenRouter

Post Description

Author here. Since the last post, RelayFreeLLM now supports NVIDIA's free catalog, adds 4-mode context management (with extractive summarization that doesn't need an LLM), session affinity, output normalization, and a global provider lock to eliminate 429 errors entirely. 8 providers, one endpoint, all free.

Similar Projects

Developer Tools●●Solid

Node.js LLM internationalization compiler: Scan code and Auto-Translate

It hooks into your build process, finds t(...) calls, and only generates missing translations via an LLM while preserving hand-edited entries — useful ergonomics many i18n tools skip. Comes with Vite/Webpack plugins, watch/batching modes and unused-key cleanup; the hard parts left to you are translation quality, API cost, and secrets management.

Solve My ProblemNiche Gem
mfcmatheus
203mo ago