One gateway in front of every model you use
An LLM gateway puts one API in front of Claude, open-weight models and any other provider. Pexon deploys LiteLLM inside your own cloud tenant with per-team budgets, audit-grade request logging and a routing policy held in configuration, so changing model or provider never means redeploying the applications that call it.
Every application holds its own key, and nobody can attribute the bill
The first Claude integration is a key in an environment variable. The fifth is a governance problem. Spend arrives as one unattributable line in a cloud invoice, no team can be told what it costs, and swapping model or provider means a deployment in every application that calls it. The requests themselves are fine. What is missing is the layer that would let you see them, price them and route them.
What we deploy
LiteLLM inside your own tenant
One OpenAI-compatible endpoint in front of Claude on Bedrock, Vertex or the Anthropic API, plus any open-weight model you run yourself. Deployed in your cloud account under your own IAM roles, so no request leaves the boundary your security review already approved.
Routing policy in configuration, not in code
Which model answers which query is a policy line: by workload, by team, by cost ceiling, by whether the payload may leave the tenant. Changing it is a config change and a restart, not a release across every calling service.
Per-team budgets and spend attribution
Every request carries a team or project key. Tokens and cost are recorded against it, budgets are enforced at the gateway rather than discovered at month end, and the monthly figure per team is the thing that makes an AI budget conversation possible at all.
Audit-grade request logging
Who asked what, which model answered, what it cost, and when. Retention and pseudonymisation configured to whatever your works council and data protection officer actually agreed to, because an observability stack that identifies individual engineers gets switched off.
Fallback routing and failure handling
A gateway on the request path is a new failure point unless it is treated as one. Health checks, automatic fallback to a second provider, and alerting on error rate rather than on uptime.
What changes
- Model and provider changes stop being deployments and become configuration.
- Spend is attributable per team, which is the precondition for anyone owning it.
- Contract-sensitive workloads can be routed to self-hosted models without the calling applications knowing.
- One audit trail for every model call in the company, instead of one per application.
- The next model generation is a routing decision rather than a migration project.
A two-week blueprint. We inventory what already calls a model, measure current spend and where it lands, and deploy the gateway against one real workload with budgets and logging live. You end with a running gateway and a cost figure per team, not a slide about one.
Questions buyers ask
What does an LLM gateway actually do that calling the API directly does not?
It gives you one place to enforce budgets, log requests for audit, and decide which model answers which query. Without it, every application holds its own key, nobody can attribute spend to a team, and changing model means a deployment in each of them. The gateway turns those from code changes into configuration changes.
LiteLLM or OpenRouter?
OpenRouter is a hosted service; LiteLLM is software you run. If requests may not leave your tenant, or spend attribution has to survive an audit, you run it yourself and LiteLLM is the answer. If neither constraint applies, OpenRouter is less work. We deploy LiteLLM because our customers usually have at least one of those constraints.
Does a gateway add latency?
A few milliseconds of routing overhead, which is invisible next to model inference time. What it can add is a failure point, so it runs with the same availability treatment as anything else on the request path: health checks, fallback routing to a second provider and alerting on error rate rather than on uptime alone.
How does per-team cost attribution work?
Every request carries a team or project key, and the gateway records tokens and cost against it. That produces a monthly figure per team rather than one unattributable line in a cloud invoice, which is what makes an AI budget conversation possible at all.
Can we route some workloads to a self-hosted model?
That is the main reason to have a gateway. Contract-sensitive or high-volume workloads route to open weights running on your own GPUs, everything else goes to Claude, and the split is a policy line rather than an architecture decision. We size the self-hosted side separately.
Not a sales call. An architecture call.
Thirty minutes with the architect who would actually run the engagement.