AI Fusion — a self-hosted AI gateway
A self-hosted gateway that every AI request from my apps routes through — pooling dozens of free API keys across 5 providers behind one OpenAI-compatible endpoint.
Why it exists
At some point I had four apps, each with its own AI request logic: its own keys, its own retries, its own rate-limit bugs. I decided I wanted to solve the problem once — and then never again.
That’s how AI Fusion was born: one endpoint, one token, and behind them — dozens of free API keys from five different providers, each with its own limits and moods.
How it works
A client says, for example, “give me a fast free model”. The gateway turns that into an ordered list of attempts: which provider, which model, which key. If a key hits a limit, it “cools down” for a while and the request quietly continues to the next one. If every key is resting, the response isn’t an error but an honest message: “retry in 40 seconds”. Clients know exactly when to come back instead of guessing with exponential backoff.
All keys are encrypted at rest, every request is measured and virtually billed, and metrics flow into Prometheus. It speaks three ingress “dialects” — OpenAI, Anthropic and Gemini — so whatever SDK I already use just works.
What I learned
That rate limits are a more interesting engineering problem than the AI models themselves. And that the best infrastructure is the kind your apps don’t even know exists.