The Azure or OpenAI bill is escalating and nobody owns it.
LLM cost optimisation attacks three things at once: repeat queries served from a semantic cache at near-zero marginal cost, each request routed to the cheapest model that still answers it correctly, and usage attributed per department so the bill stops being one unexplained line in the cloud invoice.
The starting point
No caching, no routing, no attribution.
Most escalating inference bills share the same three causes: every query hits the largest available model, near-identical questions are recomputed from scratch, and nothing ties the spend back to the team that generated it.
Why the bill grows
The three causes are architectural, which is why they are fixable
No caching means the same onboarding question, asked by ten teams, is billed ten times at full price. Providers price cached input at a small fraction of uncached input, so the saving from a semantic cache compounds with every repeated question across the organisation — and in most estates it is the single largest saving available.
No routing means a lookup and a multi-step reasoning task both bill at the largest model's price, because nothing sends the lookup anywhere cheaper. Routing by query complexity, validated against the same eval set used for release, sends each query to the cheapest model that answers it correctly.
No attribution means the invoice is one line nobody owns. The teams consuming the budget cannot see their share, so there is no mechanism for the behaviour to change — which is why the usage dashboard is the third lever, not an afterthought.
What we build
Three levers
- Semantic caching. Repeat and near-repeat queries served from cache at near-zero marginal cost, which in most estates is the single largest saving available.
- Dynamic routing. Each query sent to the cheapest model that answers it correctly, validated against the same eval set used for release.
- Attribution. A usage dashboard showing which departments and which use cases consume the budget, so the conversation can move from total to ownership.
The baseline
Measured against your own starting position, not an industry figure
The first two weeks are measurement: which models are called for which query types, how much repeated input is paid for uncached, which departments and applications generate the spend. The levers then go in one at a time, each behind the eval set your release process already uses.
A query only moves to a smaller model or to cache when it demonstrably still answers correctly — the FAQ answer is not marketing, it is the operating rule. If the baseline shows the spend is already efficient, we say so and the engagement stops there.
The fourth lever nobody lists
Budgets, alerts and stop conditions
Caching, routing and attribution make the bill smaller and legible. The fourth control makes it safe: per-session and per-agent token budgets, alerts when a workload crosses its expected band, and a stop on unattended loops that exceed a configurable threshold.
This is the control that lets agent workloads run at night at all — the pattern behind the surprise bills in the Claude Code ecosystem, where parallel terminals and night agents keep generating tokens after the human went home. A budget that halts the loop is the difference between an experiment and an incident.
What changes
- Spend becomes predictable enough to budget rather than merely observe.
- Teams see their own consumption and adjust without being policed.
- Model choice becomes an engineering decision with evidence behind it.
- Agent workloads run unattended because they are bounded, not because nobody thought about it.
Three weeks, fixed price.
We measure your reduction against your own baseline in the first two weeks.
Any number quoted before that measurement would be a guess dressed as a commitment.
Where this leads
Cost questions
Will cheaper models make answers worse?
Only if routing is done blindly. A query moves to a smaller model when the eval set shows it still answers correctly, and it moves back when that stops being true.
How much will we save?
That depends entirely on your starting position, so we measure it against your own baseline during the first two weeks rather than quoting a range up front.
Next step
Not a sales call. An architecture call.
Thirty minutes with the architect who would actually run the engagement.