The Azure or OpenAI bill is escalating and nobody owns it.
LLM cost optimisation attacks three things at once: repeat queries served from a semantic cache at near-zero marginal cost, each request routed to the cheapest model that still answers it correctly, and usage attributed per department so the bill stops being one unexplained line in the cloud invoice.
No caching, no routing, no attribution.
Most escalating inference bills share the same three causes: every query hits the largest available model, near-identical questions are recomputed from scratch, and nothing ties the spend back to the team that generated it.
Three levers
Semantic caching
Repeat and near-repeat queries served from cache at near-zero marginal cost, which in most estates is the single largest saving available.
Dynamic routing
Each query sent to the cheapest model that answers it correctly, validated against the same eval set used for release.
Attribution
A usage dashboard showing which departments and which use cases consume the budget, so the conversation can move from total to ownership.
What changes
- Spend becomes predictable enough to budget rather than merely observe.
- Teams see their own consumption and adjust without being policed.
- Model choice becomes an engineering decision with evidence behind it.
We measure your reduction against your own baseline in the first two weeks. Any number quoted before that measurement would be a guess dressed as a commitment.
Cost questions
Will cheaper models make answers worse?
Only if routing is done blindly. A query moves to a smaller model when the eval set shows it still answers correctly, and it moves back when that stops being true.
How much will we save?
That depends entirely on your starting position, so we measure it against your own baseline during the first two weeks rather than quoting a range up front.
Not a sales call. An architecture call.
Thirty minutes with the architect who would actually run the engagement.