Stop rebuilding the platform every time a model ships
A German asset manager in the cooperative banking network stopped rebuilding its platform at every model release. Pexon introduced a versioning pattern with development, main and legacy tracks on half-yearly upgrade windows, a Terraform module library for application stacks, and Kubernetes right-sizing so bank APIs no longer sat on GPU nodes.
The engagement at a glance
Written for the platform lead at a regulated financial services company whose teams lose a sprint to every model generation, and whose cloud bill contains at least one workload sitting on hardware it does not need.
| Customer | German asset manager inside the cooperative banking network, providing fund services to member banks |
|---|---|
| Cloud | Microsoft Azure with AKS, plus residual AWS |
| Stack | Terraform and Terragrunt, GitLab CI/CD migrated from Azure DevOps, Python with NumPy and Pandas, FastAPI, Azure Defender, Grafana, FinOps tooling, application stack library |
| Compliance | BaFin BAIT, MaRisk, DORA, GDPR, ISO 27001 |
| Context | Innovation lab building AI-supported services for banks in the network |
Every model release turned into an unplanned migration
An innovation lab inside a regulated asset manager was building AI-supported services for banks in its network. Two costs kept surfacing that had nothing to do with whether the models worked.
The first was cadence. Each new model generation arrived as a forced migration sprint. Teams stopped what they were doing, upgraded, revalidated and resumed, and the roadmap absorbed the interruption every time. This is the tax nobody budgets for when they approve an AI project, and over a year it is larger than the inference bill.
The second was infrastructure that had grown per project. Every application stack was a bespoke construction, so provisioning was slow, uniformity was absent, and a security or audit change had to be renegotiated per project. Underneath that sat the ordinary and expensive Kubernetes mistake: workloads scheduled onto nodes far more capable than they needed.
Three tracks, two upgrade windows a year
The model versioning pattern replaced reaction with cadence. Three tracks run in parallel, development, main and legacy, with upgrade windows twice a year. A new model release lands in development and waits for the next window rather than interrupting delivery.
The effect is that teams get a predictable migration slot instead of an unplanned sprint. Nothing about this is technically difficult, which is exactly why it is worth stating plainly: the problem was never that the upgrade was hard, it was that it was unscheduled.
In a supervised institution the pattern doubles as a compliance artefact. Being able to say which model version answered a given request at a given time is the kind of question BAIT and DORA conversations arrive at, and a versioning scheme designed only for engineering convenience rarely answers it.
Trade-off: Running three tracks means paying to keep a legacy track alive and accepting that you are deliberately not on the newest model for up to six months. Teams that need a brand-new capability the week it ships will find this frustrating. It was the right trade because the cost of unplanned migration across many teams exceeded the value of being early, and because a regulated platform has to be able to say what it was running last quarter.
A module library, and not running bank APIs on GPU nodes
Application stacks moved from project-specific constructions to a shared Terraform module library, assembled like components rather than built from scratch. Deployments got faster, and more usefully they got uniform, which is what allows a control to be applied once instead of per project.
FinOps right-sizing on Kubernetes addressed the placement problem directly: applications are deliberately mapped to node types, so a bank API does not sit on a GPU node. This is among the most common expensive mistakes in a shared cluster and among the cheapest to fix, because it costs attention rather than engineering.
Around that, observability improved with a Grafana instance for database monitoring, the security posture was strengthened through wider Azure Defender use, and CI/CD was consolidated by migrating from Azure DevOps to GitLab as the standard stack. Consolidating CI is worth more than the tool comparison suggests: two CI systems means two of every pipeline and two places for a control to be missing.
- Terraform module library as a shared component system for application stacks
- Development, main and legacy model tracks on half-yearly upgrade windows
- Node placement matched to workload, keeping non-accelerated services off GPU nodes
- Grafana observability for database monitoring
- Wider Azure Defender coverage for security posture
- Azure DevOps to GitLab migration as the single standard stack
What changed
The platform is more stable, easier to maintain and faster to deploy onto. Model upgrades became a scheduled event with a known window rather than an interruption, and provisioning a new application stack became assembly rather than construction.
None of this is glamorous, and that is the point. The cost of running AI in production is dominated by unplanned migrations and misplaced workloads, not by the price of tokens, and both are fixed with platform engineering rather than with a better model.
What this case study does not prove
Three model tracks are not free, and this page should not be read as a recommendation to run them everywhere. You pay to keep a legacy track alive, and you accept being up to six months behind the newest capability. For a team whose product depends on a model feature the week it ships, that is a bad trade and the honest answer is to take the migration pain instead.
The right-sizing result is the most transferable thing here and the least impressive to write about. It is not clever. It is looking at which workloads are scheduled onto which node types and moving the ones that do not need an accelerator, and the reason it keeps being available as a saving is that nobody is measured on it.
We also cannot claim a percentage on the cloud bill, because the platform changed in several ways at once during the same period and attributing the delta to one of them would be invention. What can be said is what was changed and why, and that the platform is faster to deploy onto and easier to maintain than it was.
Before and after
| Dimension | Before | After | Effect |
|---|---|---|---|
| Model upgrades | Forced migration sprint per release | Three tracks, half-yearly windows | Predictable slot instead of an interruption |
| Application stacks | Project-specific constructions | Shared Terraform module library | Faster to provision, uniform to audit |
| Node placement | Workloads on more capable nodes than needed | Right-sized per workload | Spend follows requirement, not default |
| CI/CD | Azure DevOps alongside other tooling | GitLab as the single standard | One pipeline set, one access model |
Questions buyers ask about this
How often should you upgrade the model behind a production AI system?
On a fixed cadence you choose, not on the vendor's release schedule. This platform runs three parallel tracks, development, main and legacy, with half-yearly upgrade windows. Teams get a predictable migration slot instead of an unplanned sprint every time a new model generation ships, which is what actually destroys roadmaps.
What is the cheapest AI cost saving most companies have not made?
Node placement. Running workloads that need no accelerator on GPU nodes is one of the most common and most expensive mistakes in a shared Kubernetes cluster. Deliberately mapping applications to node types, so a bank API never lands on a GPU node, costs nothing but attention and pays back every month.
Does a Terraform module library actually pay for itself?
It does once several teams are provisioning similar application stacks. Replacing project-specific constructions with a shared module library makes deployments faster and, more importantly, makes them uniform, which is what lets a security or audit change be applied once rather than negotiated per project.
How do you run AI infrastructure under BAIT, MaRisk and DORA?
The regulatory requirements mostly translate into ordinary platform engineering: reproducible infrastructure, documented change paths, observability you can show an auditor, and a security posture that is configured rather than assumed. The AI-specific part is being able to explain which model version answered when, which is why the versioning pattern is a compliance artefact as much as an engineering one.
Why migrate from Azure DevOps to GitLab mid-platform?
To standardise on one stack rather than maintain two. The value is not in the tool choice but in ending the split, because a platform with two CI systems has two of every pipeline, two access models and two places for a control to be missing.
Your AI bill is probably not mostly tokens
Unplanned model migrations and workloads on the wrong nodes usually cost more than inference does. The two-week blueprint measures both against your own platform and tells you which one to fix first.