LLM vendor lock-in: the 30% your per-token price comparison misses

The request format is the cheapest thing you will ever have to move

Changing model vendor looks like a three-line job, and for the request format it genuinely is. Anthropic publishes a compatibility layer that lets you keep the OpenAI SDK, point the base URL at api.anthropic.com, swap the key and write a Claude model name. The call returns. Nothing in your code knows anything changed.

Anthropic's own documentation then tells you not to ship it. The compatibility layer, in their words, is primarily intended to test and compare model capabilities and is not considered a long-term or production-ready solution for most use cases. That is the vendor arguing against the easy version of portability on their own docs page, which is a more useful signal than any analyst note on the subject.

The reason is in one sentence further down that page, and it is the sentence worth pinning above the migration ticket: most unsupported fields are silently ignored rather than producing errors.

Read the list below as failure modes, not as a feature matrix. None of them throw. Your integration tests pass, your latency dashboard looks normal, and the properties your application was quietly relying on are gone.

That is the shape of the whole problem, and the API layer is the mild version of it. Everything expensive about switching vendors fails the same way: successfully.

• The strict flag for function calling is ignored, so tool-use JSON is no longer guaranteed to follow your schema.
• response_format is ignored. Code that assumed JSON mode now parses whatever came back.
• Prompt caching is not supported through the compatibility layer at all — the cost model of a long shared prefix disappears without a line changing.
• seed, logprobs, presence_penalty, frequency_penalty and service_tier are ignored.
• System and developer messages are hoisted and concatenated into a single initial system message, newline-separated, because Anthropic supports one. A prompt that placed instructions mid-conversation no longer does.

A migration is four projects, and only the first one gets budgeted

The middle rows are where estimates go wrong, because they are the rows that cannot be inspected from the outside. A prompt that was tuned against one model's instruction-following behaviour will produce plausible output on another and worse output on the cases that matter. The only artefact that detects this is an evaluation set built from your own failures, which means the evaluation set — not the prompt — is the portable asset. Teams that never built one do not have a migration problem; they have a measurement problem that a migration exposes.

One honest note on ownership: rows two and three have different owners in most organisations, and neither of them is the platform team that will be asked how long the switch takes.

The exit path is a configuration file, or it does not exist

An exit strategy that is not a file somebody can edit is a sentence in a slide deck. The concrete version is boring: every application calls one internal endpoint with an alias like chat-default, the alias maps to a provider route, and switching vendors means editing the map and reloading. No application holds a provider key, so no application needs a deployment to change model.

That is the entire mechanism. It is worth having precisely because it is unimpressive.

What a gateway does not do deserves more airtime than what it does. It does not port your prompts. It does not port your evaluation set. It does not port cache economics, because those are per model and per provider. It does not port a provider-specific tool surface — server-side execution, batch endpoints, an MCP connector — and if your agent was built on one, the gateway routes a request that the other side cannot answer.

The strongest argument against the whole approach is that a gateway is itself a dependency, and a lowest-common-denominator one. Route everything through a compatibility layer and you converge on the intersection of what every provider supports, which is a plain chat completion — meaning you pay a frontier-model premium for features the abstraction forbids you from using. That objection is correct and it is not hypothetical.

Our answer is two-part rather than a rebuttal. First, run the gateway as software inside your own tenant, so the exit from the gateway is the thing you were doing before it existed: direct SDK calls with a key. Second, make it the default path and not a mandate. Workloads that genuinely need a vendor-specific capability call that vendor directly and are recorded as deliberately non-portable, with a named owner. A register of five deliberate exceptions is a governable position. An undocumented assumption that everything is portable is not.

# One alias, two routes. Switching vendors edits this file, not the apps.
# https://docs.litellm.ai/docs/proxy/configs

model_list:
  - model_name: chat-default
    litellm_params:
      model: anthropic/claude-opus-5
      api_key: "os.environ/ANTHROPIC_API_KEY"
  - model_name: chat-fallback
    litellm_params:
      model: <second-provider>/<second-model>
      api_key: "os.environ/FALLBACK_API_KEY"

litellm_settings:
  fallbacks: [{"chat-default": ["chat-fallback"]}]
  num_retries: 3

The exit plan, in the order it actually gets built

1. Put every application behind one alias. Nothing else in this list works while twelve services each hold their own provider key.
2. Build the negative case into the evaluation set. Relevance tests prove the model answers; only a set built from your own past failures proves a replacement is not worse where it matters.
3. Re-count your top ten production prompts against every candidate model's own counter, and store the numbers with a date. Costs and context budgets are both denominated in that number.
4. Name the deliberately non-portable workloads. For each one, record the capability, the provider it belongs to, and what reversing it would cost in engineering weeks.
5. Run the drill on a schedule and write down two figures each time. The evaluation pass rate on the fallback route, and tokens per request under the fallback tokenizer.

Alias first, evaluations second, counts third, drill fourth. Reversing that order produces a document rather than a capability, because every later step depends on a decision only the earlier one can make.

For EU financial entities this stopped being optional. DORA — Regulation (EU) 2022/2554 — requires in Article 28(8) that financial entities put exit strategies in place for ICT services supporting critical or important functions, and requires the exit plans to be comprehensive, documented, sufficiently tested and periodically reviewed. The regulation has applied since 17 January 2025. If a model sits inside a critical or important function, we would rewrite it is not a tested plan, and an untested plan is the specific thing the article names.

Everyone else is not exempt from the arithmetic, only from the audit. The drill below costs an afternoon a quarter and is the only way to find out whether the file you wrote in month one still describes the system you have in month nine.

# Exit drill. Run it on a schedule, not on the day you need it.
# DORA Art. 28(8) requires exit plans to be documented, tested and reviewed:
# https://eur-lex.europa.eu/eli/reg/2022/2554/oj

# 1. Flip the alias. One file, no application deploy.
sed -i 's/chat-default/chat-fallback/' gateway/routes.env

# 2. Run the evaluation set that defines "not worse" — not a smoke test.
pytest evals/ --junitxml="exit-drill-$(date +%F).xml"

# 3. Record both numbers in the exit plan, dated: evaluation pass rate,
#    and tokens per request under the fallback model's tokenizer.