Hosting
One gateway in front of every model you use
An LLM gateway puts one API in front of Claude, open-weight models and any other provider. Pexon deploys LiteLLM inside your own cloud tenant with per-team budgets, audit-grade request logging and a routing policy held in configuration, so changing model or provider never means redeploying the applications that call it.
What changes
Before and after one routing layer
| Question | Without a gateway | With one |
|---|---|---|
| What did AI cost team X? | Not answerable | A monthly figure per team key |
| Change model or provider | A deployment per calling service | A config change and a restart |
| Who asked what, and when | Per-application logs, if any | One audit trail for every call |
| Route sensitive work locally | Rewrite the caller | A policy line |
| A provider has an outage | Each application fails its own way | Fallback to a second provider |
Every application holds its own key, and nobody can attribute the bill
Spend arrives as one unattributable line in a cloud invoice, no team can be told what it costs, and swapping model or provider means a deployment in every application that calls it.
The requests themselves are fine. What is missing is the layer that would let you see them, price them and route them — and that layer is cheap to add early and expensive to retrofit once a dozen services hold their own credentials.
The first Claude integration is a key in an environment variable. The fifth is a governance problem.
What we deploy
- One OpenAI-compatible endpoint in front of Claude on Bedrock, Vertex or the Anthropic API, plus any open-weight model you run yourself.
- Routing policy held in configuration: by workload, by team, by cost ceiling, by whether the payload may leave the tenant.
- Per-team budgets enforced at the gateway rather than discovered at month end, with tokens and cost recorded against a team or project key.
- Request logging with retention and pseudonymisation set to what your works council and data protection officer actually agreed to.
- Health checks and automatic fallback to a second provider, with alerting on error rate rather than on uptime alone.
LiteLLM inside your own cloud account, under your own IAM roles, so no request leaves the boundary your security review already approved.
Trade-off
A gateway is a new component on the request path, and on the day it fails everything fails together. That is the honest cost, and it is why it ships with fallback routing and its own alerting rather than as a proxy somebody stood up. If you have one application calling one model, you do not need this yet — add it when the second team asks for a key, not before.
Why run it rather than buy it
Self-hosted when the constraint is real, hosted when it is not
OpenRouter is a hosted service and it is less work. If requests may not leave your tenant, or if spend attribution has to survive an audit, that trade stops being available and you run the gateway yourself. Most of our customers have at least one of those constraints, which is why we deploy LiteLLM rather than resell a router.
If neither applies to you, we will say so. A gateway you run is a component you own, and owning one you did not need is a bad trade.
Where this connects
- Claude in your own cloud tenant — the deployment underneath, when requests may not leave your boundary
- GPU and inference sizing — what the self-hosted side of the routing rule costs to run
- LLM cost optimisation — caching and model routing once the spend is finally visible
Questions buyers ask
What does an LLM gateway actually do that calling the API directly does not?
It gives you one place to enforce budgets, log requests for audit, and decide which model answers which query. Without it, every application holds its own key, nobody can attribute spend to a team, and changing model means a deployment in each of them. The gateway turns those from code changes into configuration changes.
LiteLLM or OpenRouter?
OpenRouter is a hosted service; LiteLLM is software you run. If requests may not leave your tenant, or spend attribution has to survive an audit, you run it yourself and LiteLLM is the answer. If neither constraint applies, OpenRouter is less work. We deploy LiteLLM because our customers usually have at least one of those constraints.
Does a gateway add latency?
A few milliseconds of routing overhead, which is invisible next to model inference time. What it can add is a failure point, so it runs with the same availability treatment as anything else on the request path: health checks, fallback routing to a second provider and alerting on error rate rather than on uptime alone.
How does per-team cost attribution work?
Every request carries a team or project key, and the gateway records tokens and cost against it. That produces a monthly figure per team rather than one unattributable line in a cloud invoice, which is what makes an AI budget conversation possible at all.
Can we route some workloads to a self-hosted model?
That is the main reason to have a gateway. Contract-sensitive or high-volume workloads route to open weights running on your own GPUs, everything else goes to Claude, and the split is a policy line rather than an architecture decision. We size the self-hosted side separately.
Next step
Start with what already calls a model
A two-week blueprint at €4,900. We inventory what already calls a model, measure where the spend lands, and deploy the gateway against one real workload with budgets and logging live. You end with a running gateway and a cost figure per team, not a slide about one.