LiteLLM vs. OpenRouter — who operates the layer matters more than the models

Both give you many models behind one API. The difference is who is in the middle.

LiteLLM and OpenRouter solve the same surface problem: an application should call one endpoint and reach any model, without a key and an SDK per provider. That is where the similarity ends. LiteLLM is software you run inside your own tenant; OpenRouter is a hosted service that sits between you and the models. Every request you send through OpenRouter passes through their infrastructure. Every request through LiteLLM stays inside your boundary.

For a team with no residency constraint and a desire to move fast, OpenRouter is the less work by a wide margin. For a team whose requests may not leave the estate, or whose spend attribution has to survive an audit, the hosted option is not available in the first place.

If requests may not leave your tenant, or spend attribution has to survive an audit, the hosted trade stops being available and LiteLLM is the answer. If neither constraint applies, OpenRouter is genuinely less work — and we will say so.

What the self-hosted gateway actually buys

The value of LiteLLM is not the routing — that is commodity. It is what the routing enables when the gateway is yours: per-team budgets enforced at the gateway rather than discovered at month end, request logging with retention set to what your works council agreed, and the ability to route one workload to a self-hosted model and another to a frontier API from the same configuration file.

The honest cost is that a gateway is a component you operate. On the day it fails, everything fails together — which is why it ships with fallback routing, health checks and its own alerting, and why the runbook is part of the deliverable.

One alias, two routes

Switching vendors or adding a provider is a configuration change, not a deployment.

# LiteLLM config: one API for the estate
model_list:
  - model_name: chat-default
    litellm_params:
      model: anthropic/claude-opus-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: chat-fallback
    litellm_params:
      model: openrouter/google/gemini-2.5-pro
      api_key: os.environ/OPENROUTER_API_KEY

router_settings:
  routing_strategy: simple-shuffle
  fallbacks:
    chat-default: ["chat-fallback"]

Three decisions that settle the gateway question

Each one is answerable in an afternoon, and each decides the architecture.

• May requests leave the estate? If the answer is no, the hosted option is off the table and LiteLLM (or equivalent self-hosted software) is the only lane.
• Does spend attribution have to survive an audit? Per-team budgets and request logs are the difference between a gateway that is an optimisation and one that is a control.
• How many applications will hold keys? The first integration is a key in an environment variable. The fifth is a governance problem — the gateway should exist before the fifth team asks.

The first Claude integration is a key in an environment variable. The fifth is a governance problem.

Continue on the gateway and the stack

• LiteLLM enterprise setup — The production guide: deployment, budgets, logging and the runbook.
• LLM cost optimisation — Semantic caching, routing and attribution once the gateway is live.