Migrating from LiteLLM: What Actually Breaks

A practical migration guide from LiteLLM to another self-hosted gateway. What moves with a base URL change, what has to be rewritten, what does not carry over at all, and when you should not migrate.

LiteLLM is the most widely deployed self-hosted AI gateway there is, with 57,637 GitHub stars and commits landing daily. Most teams evaluating a move are not moving because something is wrong with it. They are moving because they hit the enterprise line, or because they need things a gateway does not do.

This is a practical guide to what that migration involves. It is written by a competitor, so the most useful thing it can do is be accurate about the parts that are easy, the parts that are not, and when you should stay.

First: should you?

Stay on LiteLLM if:

You need a provider it supports and your alternative does not. LiteLLM's coverage is the widest in the category and this can settle the question on its own. Check your model list before anything else.
The MIT core covers you. If you do not need virtual keys, budgets, SSO or per-team spend tracking, the enterprise directory never affects you and LiteLLM is free forever.
Routing is genuinely the whole problem. Adding a platform to solve a proxy-sized problem is over-buying.
You use the Python SDK as a library rather than the proxy as a service. That is a first-class way to use LiteLLM and it does not port anywhere.

Consider moving if:

You have hit the enterprise line. LiteLLM's root LICENSE places everything under enterprise/ under a separate licence, and their enterprise page lists virtual keys, budgets, RPM and TPM limits, spend by key, user, team and org, SSO with SCIM, and OIDC or JWT auth. Enterprise pricing is not published, so this arrives as a sales conversation rather than a line item you can plan for.
You need what happens after the request: agent sessions, isolated code execution, evaluations, human review. These are separate systems around LiteLLM.

What moves for free

The endpoint. LiteLLM's proxy is OpenAI-compatible, and so are most alternatives. Applications calling through the OpenAI SDK move with a base URL and key change:

# before
client = OpenAI(
    base_url="http://litellm-proxy:4000",
    api_key=os.environ["LITELLM_MASTER_KEY"],
)

# after
client = OpenAI(
    base_url="https://gateway.internal/v1",
    api_key=os.environ["GATEWAY_API_KEY"],
)

That is genuinely it for application code, provided you were calling the proxy over HTTP. This is why gateway migrations are cheaper than observability migrations: no instrumentation to unpick.

Model identifiers, usually. LiteLLM mostly uses bare model names in the proxy, so gpt-4o-mini stays gpt-4o-mini. Check any provider-prefixed identifiers you use.

What has to be rewritten

Direct litellm SDK usage. This is the big one, and how much of it you have determines the size of the job.

# This does not port. It is LiteLLM as a library, not as a service.
import litellm
response = litellm.completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "..."}],
)

Code like this has to be rewritten against the OpenAI SDK pointed at the new gateway, or against the new gateway's own SDK. Grep for import litellm before you estimate anything.

model_list configuration. LiteLLM's YAML config expresses providers, model aliases, API bases and per-model settings. Every gateway has an equivalent concept and none of them share a format. This is mechanical but not automatic.

Router settings. Routing strategy, fallback ordering, retry policy, cooldowns and timeouts all need re-expressing. Do this deliberately rather than porting defaults, since you probably tuned some of these after an incident and the reasoning is worth carrying over.

Custom callbacks. LiteLLM's callback system for logging and metrics is LiteLLM-specific. If you have written custom callbacks, budget real time.

Virtual keys and budgets, if you were on enterprise. The concepts map to API keys with per-tenant limits on most alternatives, but the objects have to be recreated.

What does not carry over

Be realistic about these up front:

Historical spend and usage records in the LiteLLM database. There is no importer. Keep the old database readable for as long as you need the history, and plan for a reporting seam at the cutover date.
Existing virtual key objects and their budget state. Keys get reissued, and consumed-budget counters restart.
Custom router strategies expressed in LiteLLM configuration.
LiteLLM-specific request headers and any client code that sets them.

A rollout that does not hurt

1. Grep for import litellm first. This single command sizes the whole migration. If it returns nothing, you are doing a configuration port. If it returns hundreds of hits, you are doing a code change and should plan accordingly.

2. Check your model list. Enumerate every model identifier in use and verify each is supported on the target. A single unsupported niche provider can block a full cutover, and finding that out in week three is expensive.

3. Stand the new gateway up alongside. Do not replace. Run both.

4. Move one low-risk service. Something with real traffic, a rollback you can execute in a minute, and a team who will notice if latency changes.

5. Compare cost accounting for a week. Both gateways should agree with each other and with the provider invoice. If they do not, find out why before moving anything else. Watch specifically for cached input tokens, which are billed at a discount and are a common source of disagreement.

6. Test fallback deliberately. Break the primary provider in staging on both systems and confirm traffic moves. A fallback that has never fired is a configuration rather than a capability, and migration is when you find that out.

7. Move the rest, then delete the LiteLLM config rather than leaving it to rot as a second source of truth.

The honest summary

The endpoint change is trivial. The migration cost is proportional to how much you used LiteLLM as a Python library rather than as a service, and to how many custom callbacks you wrote.

If you used it as a proxy with a YAML config, this is a day or two. If you built on the SDK, it is a project.

And if the MIT core covers what you need, the correct answer may well be to stay. LiteLLM is good software with the largest community in this category, and switching costs are real.

If you are moving to Everstack specifically, the detailed comparison covers the feature and licence differences, and the gateway docs cover configuration.