What self-hosting actually buys you
Be specific about this, because the honest list is shorter than the marketing list.
Open-source AI infrastructure
How to Self-Host an LLM Gateway
A practical walkthrough for running an LLM gateway on your own infrastructure: what self-hosting actually buys you, how to read the licence, and a working Docker Co…
Read the licence before you commit
"Open source" is doing a lot of work in this category, and the differences are load-bearing. Everything below was checked against the actual LICENSE file in each repository on 31 August 2026.
The deployment
Here is a working stack. Everstack is used as the concrete example because it is what we build, and because the gateway ships in the same binary as the observability that makes it useful. The shape generalises: every gateway in this category needs a metadata store, and every one worth running needs somewhere to put telemetry.
Taking it to production
The Compose file above is a development stack. Five things separate it from something you can page on.
When you should not self-host
If you are a team of five with no platform engineer, self-hosting is the wrong call. You will spend more on operating three stateful services than a hosted gateway costs, and you will spend it in the currency you have least of. Use a hosted gateway and revisit the decision when either your volume or your compliance obligations change.