Every multi-tenant SaaS team already knows how to keep customers' rows apart in a database. Adding an AI feature introduces three new stores that most teams have never had to isolate: the retrieval index, the prompt, and the cache. We learned about each of them the careful way, in a review before launch rather than in an incident after it, and this post is what that review found.

Where tenant data now travels

In a conventional SaaS product, a customer's data lives in tables keyed by tenant and is read by code that always filters on that key. An AI feature copies some of that data into a vector index for retrieval, assembles it into prompts sent to a model, stores model outputs, and caches prompts and responses to control cost. Each copy is a place where the tenant key can be forgotten.

Where a tenant's data goes once an AI feature exists: the index, the prompt, the outputs and the cache each need the tenant boundary re-established.
Where a tenant's data goes once an AI feature exists: the index, the prompt, the outputs and the cache each need the tenant boundary re-established.

The review that found the gaps

Before launching a document-assistant feature for a client's platform, we ran a tenant-isolation review as a set of attacks against a staging environment with two test tenants. Three findings came back.

The retrieval index was shared, with tenant recorded as metadata, and the retrieval query filtered on it correctly. But a background re-indexing job had a code path that dropped the metadata when a document was updated, leaving a small number of chunks with no tenant at all, which the filter treated as matching everyone.

The response cache keyed on the prompt text. Two tenants asking the same question about a shared template document would hit the same cache entry, which was harmless until the answer included content from the first tenant's own documents.

The prompt included a "recent conversation" section assembled from a session store that was keyed by user, not by tenant. A user who belonged to two tenants, which happened with consultants and partners, could carry context across.

None of these was a model problem. All of them were the tenant key going missing on a path nobody had drawn.

The design we settled on

Tenant is part of every key, everywhere. The index is either one physical index per tenant or a shared index where the tenant identifier is part of the primary key of every chunk, enforced by the write path, not just recorded as optional metadata. Cache keys include the tenant. Session stores are keyed by tenant and user together.

Retrieval filters before ranking, from the identity, not the request. The tenant used for filtering comes from the authenticated session, never from a parameter the client sends. A request cannot ask for another tenant's data because it never gets to name the tenant.

Prompts are assembled from filtered sources only. Everything that goes into a prompt passes through the same tenant-scoped access layer as the rest of the product. There is no side path where a job or a helper reads documents "for the model" without that layer.

Outputs are stored as tenant data. Model responses, summaries and extracted fields are records in the tenant's own space, subject to the same retention and deletion rules. When a tenant leaves, their AI-derived data leaves with them, including their chunks in the index and their cache entries.

The isolation controls: tenant in every key, filtering from identity before ranking, one access layer for prompt assembly, outputs stored as tenant data.
The isolation controls: tenant in every key, filtering from identity before ranking, one access layer for prompt assembly, outputs stored as tenant data.

Testing it, continuously

The two-tenant attack suite from the review became a permanent test. It creates documents in tenant A with distinctive markers, then asks tenant B every question it can think of, including injection attempts that instruct the model to reveal "all documents." Any response containing an A marker fails the build. It runs on every change to retrieval, prompting, caching or indexing, and it runs against the provider's cache behaviour as well as our own, because some providers cache prompt prefixes and the tenant boundary must hold there too.

Provider and residency questions

Multi-tenant products often serve customers in different regions with different data-residency expectations. An AI feature that sends prompts to a provider in one region may break a promise made to a tenant in another. We make the model routing tenant-aware: which provider, which region, and whether an in-boundary model is required, is a per-tenant setting with a safe default, and the egress is logged per tenant so that the question "did our data leave the region?" has an answer.

Purposefully isolated

Tenant isolation in AI features is not a new principle. It is the old principle applied to three new stores that are easy to forget because they were built quickly to make a demo work. The teams that ship safely draw the data flow, put the tenant in every key, filter from identity, and keep the attack suite running. It is a week of work at design time. It is a very different week if a customer finds it first.

  1. Draw where tenant data goes: index, prompt, outputs, cache, provider.
  2. Put the tenant in every key; enforce it on the write path.
  3. Filter retrieval from the authenticated identity, before ranking.
  4. Assemble prompts only through the tenant-scoped access layer.
  5. Store outputs as tenant data with the same retention and deletion.
  6. Keep a two-tenant attack suite in the build.