LLM Gateway for OpenAI/Anthropic Written in Golang

Hi HN - I'm Nathan. I spent a bunch of years building Shopify subscriptions software, living in the land of failed payments, retries, and "if this breaks, it breaks real money." We built a lot of automation around recovery: intelligent retry logic, routing decisions, backoffs, and all the messy edge cases you only find at scale.

When I started building AI/LLM features, I kept running into the same class of problems - except harder to reason about. Multiple providers, model quirks, intermittent failures, retries/fallbacks, and a constant question of "what actually happened?" Observability was the recurring pain point. I wanted something that didn't feel like a black box, especially once you're running real workloads and latency or errors spike for reasons that aren't obvious.

So I started building the tool I wished I had: an open-source LLM gateway / proxy in Go.

I fell into Go mostly for practical reasons: high concurrency and throughput without fighting the runtime, and a strongly-typed codebase that stays pleasant as it grows. Over time it turned into something more personal - I've found my home in Go, and this project is where I've been putting that energy.

Open source is a deliberate choice here. Coming from payments + ecommerce, trust isn't a tagline - it's operational. People need to understand what's happening under the hood, and they need to be able to verify it. I've been building software for ~15 years, and I wanted to contribute something real back to the communities that taught me how to build reliable systems.

Repo: https://github.com/ongoingai/gateway

Feedback, criticism, "you're doing this wrong," feature ideas, weird edge cases you're hitting - all welcome. If you've built anything similar (AI infra, gateways, proxies, high-throughput Go services), I'd especially love to hear what you'd consider non-negotiable for something like this.

Cheers, Nathan @ OngoingAI