The bill is escalating and nobody can say why.
AI FinOps makes inference spend predictable. Pexon adds semantic caching so repeat questions cost almost nothing, routes queries to the cheapest model that answers them correctly, and reports usage per department so the bill stops being one unattributable line in the cloud invoice.
Cost is an architecture problem.
Most escalating AI bills are not a pricing problem but a design one: no caching, every query hitting the largest available model, and no attribution back to the team that generated it. The same workload can cost ten times as much on one architecture as on another, and the difference is decided before the invoice is ever written.
AI FinOps is the discipline of making that spend predictable: semantic caching so repeat questions cost almost nothing, model routing so each query is answered by the cheapest model that gets it right, and per-department reporting so the bill stops being one unattributable line in the cloud invoice.
None of this is a discount negotiation. The levers are architectural, they are installed in the gateway between your applications and the model APIs, and they are measurable from the first week against your own baseline.
Five drivers, each with a fix that is not a price negotiation
When we start a cost engagement we do not look at the model price list first. We look at the request traffic, because every driver below shows up there before it shows up on the bill. The table is the working list we go through — each row is a pattern we find in real deployments, with the fix we install.
| Cost driver | Symptom | Fix |
|---|---|---|
| Every query hits the largest model | Simple lookups, classification and extraction all bill at frontier prices because nothing routes by difficulty. | Route by query complexity against an eval set, so routine work lands on a model that answers it correctly and costs a fraction. |
| No caching, so the same question is paid for repeatedly | Ten teams ask the same onboarding question and the bill shows ten full-price completions. | Semantic caching at the gateway: repeated or near-identical inputs are answered from cache, where vendors bill cached input at a small fraction of uncached input. |
| Agent loops and parallel sessions run unattended | Night agents and parallel terminals keep generating tokens after the human went home — the pattern that turns a €2,500 single-account bill into a surprise. | Per-session and per-agent token budgets, plus a stop on unattended loops that exceed a configurable threshold. |
| The bill cannot be attributed | One line on the cloud invoice, no way to say which department, project or application generated it. | Usage attribution at the gateway: every request carries department, project and application tags, and reporting is per tag, not per invoice line. |
| Retention and duplication of prompt context | The same system prompt, tool definitions and document excerpts are re-sent on every turn, multiplying input cost. | Prompt structure that maximises cache hits — stable prefixes, cached tool definitions — before any model change is discussed. |
The first three rows usually account for the bulk of the overspend. The fourth — attribution — is the one that decides whether the fix survives, because a cost centre that cannot see its own usage has no reason to change behaviour. The fifth is the quiet one: teams often reach for a cheaper model when the actual lever is prompt structure.
Four levers, installed in the gateway, not in the contract
Every lever below lives between your applications and the model API. That is deliberate: it means the levers apply to every application at once, without changing a line of application code, and it means they can be measured centrally.
- Semantic caching. Near-identical questions — the same onboarding query from ten teams — are answered from cache. Providers bill cached input at a small fraction of uncached input, so the saving compounds with every repeated question across the organisation.
- Model routing by query complexity. A lookup, a classification and a multi-step reasoning task are not the same workload, but most deployments send all three to the same model. Routing sends each to the cheapest model that answers it correctly, validated against the same eval set used for release.
- Attribution and per-department reporting. Every request carries department, project and application tags. The monthly report shows spend per tag — which is the only form of the bill a cost centre can actually act on.
- Budgets and stop conditions. Per-session and per-agent token budgets, with unattended loops halted when they exceed a threshold. This is the control that makes night agents and parallel terminals safe to run at all.
The caching and routing levers are implemented at the LLM gateway; the pattern is described in depth in the LiteLLM proxy enterprise guide. The cost side is an architecture engagement, not a reseller discount.
Baseline first, then levers, then proof
We do not quote a reduction range up front, because a reduction range without a baseline is marketing. The engagement starts with two weeks of measurement on your own traffic: which models are called for which query types, how much repeated input is paid for uncached, which departments and applications generate the spend.
Then the levers go in, one at a time, each behind the same eval set your release process already uses. A query only moves to a smaller model or to cache when it demonstrably still answers correctly. The result is a bill that is lower and, more importantly, predictable — with the attribution in place to keep it that way.
The measurable commitment is to your own baseline, not to an industry figure. If the baseline shows the spend is already efficient, we say so and the engagement stops there — the architecture is the answer either way.
The bill is a diagram, not a price list
The single most expensive mistake in AI cost control is treating the problem as a vendor negotiation. Pricing pages change quarterly; your architecture changes rarely. An organisation that has fixed caching, routing and attribution once does not re-negotiate them, and it does not re-learn them when a new model ships.
That is also why cost control and private AI belong together: the same gateway that routes to the cheapest model can route sensitive workloads to an EU-hosted or self-hosted one. Cost, compliance and control are one architecture decision, not three procurement decisions. The private AI pages cover that side; this one covers the bill.
Common questions
Will cheaper routing make the answers worse?
Only if it is done blindly. Routing decisions are validated against the same eval set used for release, so a query only moves to a smaller model when it demonstrably still answers correctly.
How quickly does this pay for itself?
That depends entirely on your starting position, and we measure it against your own baseline in the first two weeks rather than quoting a reduction range up front.
Not a sales call. An architecture call.
Thirty minutes with the architect who would actually run the engagement.