One retrieval platform, 800 banks, no shared context
A central IT provider for more than 800 German cooperative banks runs its retrieval platform on its own OpenShift infrastructure. Pexon introduced semantic chunking in place of rigid partitioning, tuned pgvector with HNSW indexing so latency stayed stable under tenant load, and exposed chat exports through a standard API and Model Context Protocol.
The engagement at a glance
Written for the platform owner at a regulated IT provider who has to serve hundreds of tenants from one system, and for whom a leak between two of them is a supervisory incident rather than a defect ticket.
| Customer | Central IT provider for a German cooperative banking network |
|---|---|
| Scale | More than 800 client banks, around 300 active users each |
| Infrastructure | Red Hat OpenShift, with legacy Azure services progressively decoupled |
| Stack | Python, PostgreSQL with pgvector and HNSW indexing, semantic chunking, REST API, Model Context Protocol, CI/CD |
| Compliance | BaFin including MaRisk, KRITIS, strict tenant separation with permission checks at tenant level |
| Status | Two core chatbots at production readiness, elastic scale-out in final validation |
The hard part was never the model
A central IT provider serves more than 800 cooperative banks, each with roughly 300 active users. Two retrieval chatbots existed and needed to reach production readiness. The obstacles were not model quality. They were chunking, index performance under load, tenant isolation and a codebase that had grown without common principles.
Each of those becomes qualitatively different at this multiple. Chunking that is merely acceptable produces answers that are individually plausible and collectively unreliable, and at 800 tenants nobody will notice the pattern until a supervisor does. An index that performs adequately in a single-tenant test degrades unevenly under concurrent load from hundreds of institutions. And tenant separation stops being an application concern and becomes the thing the whole architecture is judged on.
The regulatory frame is BaFin, including MaRisk, plus KRITIS obligations. In that frame, a control has to be demonstrable. Asserting that the system keeps client banks separate is worth nothing; showing where in the retrieval path the check happens is worth everything.
Semantic chunking, because the split decides the answer
Rigid markdown partitioning was replaced with semantic chunking. This sounds like an implementation detail and is closer to the whole ballgame: chunking decides what the model is allowed to read, and a split at a structural boundary frequently lands in the middle of an argument. The retrieved fragments are then individually valid and jointly misleading, which is the worst failure mode available because nothing about the output looks wrong.
Splitting on meaning instead improves answer quality without changing the model, without changing the prompt and without adding cost per query. In a regulated environment that combination is rare and worth reaching for first.
Trade-off: Semantic chunking costs more at ingestion time and makes reindexing slower, which matters when a document corpus turns over often. Rigid partitioning is faster, cheaper and entirely predictable. It was the right trade here because the corpus is comparatively stable and the cost of a confidently wrong answer in a supervised institution is not symmetric with the cost of a slower reindex.
pgvector inside PostgreSQL, tuned rather than replaced
Vectors live in PostgreSQL with pgvector and HNSW indexing, tuned so that latency stayed stable as tenant load rose rather than degrading at the tail. The platform scales elastically on OpenShift toward the full 800-plus tenant population instead of duplicating a deployment per tenant.
Keeping the vectors in PostgreSQL was a deliberate choice against introducing a dedicated vector database. The team already runs, backs up, monitors and audits PostgreSQL. A new datastore would have meant a new backup story, a new access model and a new security review, and it would have bought throughput the workload did not need.
Permission checks run at tenant level inside the retrieval path. That is the control that makes strict separation between client banks demonstrable, and it is the reason this can be one platform rather than 800 of them.
- Semantic chunking in place of rigid markdown partitioning
- pgvector with HNSW indexing, tuned for stable latency under tenant load
- Elastic scale-out on OpenShift toward 800+ tenants, in final validation
- Permission checks at tenant level in the retrieval path
- Standardised codebase and coding principles enforced through CI/CD
- Chat exports as a standard REST API and over Model Context Protocol
What changed
Two core chatbots reached production readiness. The codebase was standardised behind shared coding principles enforced in CI/CD pipelines, which is what stops a platform serving hundreds of tenants from drifting into hundreds of special cases.
Chat exports are exposed both as a standardised API and over Model Context Protocol, so downstream systems and agents consume conversation data through one interface rather than a bespoke integration each time. A further retrieval bot was built and taken to market on the same foundation, which is the practical test of whether a platform is a platform.
What this case study does not prove
It does not prove that pgvector is the right vector store for you. It was right here because the platform team already ran, backed up and audited PostgreSQL, and because the workload did not need throughput beyond what a tuned HNSW index delivers. A team without that PostgreSQL operating history, or with a workload an order of magnitude larger, should reach a different conclusion and should not cite this page as support for the one made here.
The elastic scale-out to the full tenant population was in final validation rather than complete. We are reporting it that way deliberately: at 800 tenants the difference between validated and nearly validated is where the interesting failures live, and a case study that blurs that distinction is not worth reading.
Semantic chunking is also not free. It costs more at ingestion and makes reindexing slower, and in a corpus that turns over daily the trade would likely go the other way. What generalises is the principle rather than the setting: chunking decides what the model is allowed to read, so it deserves more attention than the model choice it usually loses to.
Before and after
| Dimension | Before | After | Effect |
|---|---|---|---|
| Chunking | Rigid markdown partitioning | Semantic chunking | Better answers with no change to model or prompt |
| Vector index | Untuned, degrading under concurrent load | pgvector with HNSW, tuned | Latency stable as tenant load rises |
| Tenant isolation | An application concern | Permission check in the retrieval path | Separation demonstrable, not asserted |
| Codebase | Grown without common principles | Standardised, enforced in CI/CD | One platform instead of 800 special cases |
Questions buyers ask about this
Can a RAG platform meet BaFin and KRITIS requirements?
Yes, and the deciding factor is where it runs and how tenants are separated rather than which model answers. This platform runs on the provider's own Red Hat OpenShift infrastructure with permission checks at tenant level, which is what makes strict separation between more than 800 client banks demonstrable rather than asserted.
Why pgvector rather than a dedicated vector database?
Because the operational surface matters more than benchmark throughput in a regulated environment. pgvector keeps vectors inside PostgreSQL, which the platform team already runs, backs up and audits. With HNSW indexing and tuning, latency stayed stable under high tenant load, and no new datastore had to be brought through a security review.
What does semantic chunking actually change?
It decides what the model gets to read. Rigid markdown partitioning splits documents at structural boundaries that often fall mid-argument, so retrieval returns fragments that are individually valid and jointly misleading. Semantic chunking splits on meaning, which improves answer quality without touching the model at all.
How do you serve hundreds of tenants from one platform without leakage?
Permission checks run at tenant level and the platform scales elastically on OpenShift rather than by duplicating deployments per tenant. The isolation has to sit in the retrieval path itself, because at this multiple a single leak between two client banks is a regulatory incident rather than a bug.
What is Model Context Protocol used for here?
Chat exports are exposed both as a standardised REST API and over Model Context Protocol, so downstream systems and agents can consume conversation data without a bespoke integration each time. It is an interface decision rather than a feature, and it is what stops the platform becoming a dead end for everything built after it.
Multi-tenant retrieval fails at the seams, not the model
If you are serving regulated tenants from one platform, the questions that decide the architecture are chunking, index behaviour under load and where the permission check sits. The two-week blueprint answers all three against your own corpus.