Design a multi-tenant SaaS platform

A helpdesk and CRM sold to businesses looks like one app: an inbox of support tickets, a contact list, some automations. Underneath is a platform running that app for 50,000 companies at once, from a 3-person D2C brand on the free plan to a bank with 20,000 agents, on the same code and API. Every request must see exactly one company’s data, because a cross-tenant leak is the worst incident a SaaS business can have. Tenants are wildly unequal — the top 1% send over a third of the traffic — and the big ones want data that stays in India, their own encryption keys, a 99.99% SLA and upgrades only on Sunday mornings. You design how tenants are created, identified, routed, isolated, protected from each other, moved and upgraded. For the vocabulary, start with what a tenant is.

Understand the problem

The product is a customer-support helpdesk with a light CRM: tickets from email, chat and phone, contacts and companies, SLA timers, automations and reports. Its customers are businesses, so a tenant is one customer organisation with its own users, data, settings and bill. You are not designing ticket routing or the email parser. You are designing the multi-tenant platform every one of those features runs on. Be clear about that split in the first minute.

Functional requirements

1. Customers should be able to sign up and get a working tenant in seconds on the free and standard plans, and enterprises should get a dedicated environment within the hour.
2. Every request should be resolved to exactly one tenant and served only with that tenant’s data — web app, public API, webhooks, background jobs and exports alike.
3. Small and large tenants should get different isolation behind one codebase and one API: pooled infrastructure for most, a dedicated cell for those that need or pay for it.
4. Each tenant should be able to configure its own plan limits, feature flags, custom fields, workflows and single sign-on without the vendor forking the code or the schema.
5. Operators should be able to move a tenant between shards, cells or regions, and roll out releases, without downtime and without risking every tenant at once.

Non-functional requirements

• Scale. 50,000 tenants with about 2 million seats. Peak load around 60,000 API requests a second. Skewed: the largest tenant sends about 4% of all traffic, and the top 1% of tenants send over a third.
• Isolation. Zero cross-tenant reads or writes. No single bug — a missing filter, a wrong cache key, a misrouted request — may be enough to leak data.
• Latency. p99 under 300 ms for reads and 500 ms for writes, of which tenant resolution and routing get under 5 ms.
• Availability per tenant. 99.95% a month for standard plans (about 22 minutes of downtime), 99.99% for enterprise (about 4 minutes), measured per tenant, not averaged over the fleet.
• Blast radius. An infrastructure failure or a bad deploy affects at most one cell — about 5% of tenants.
• Provisioning. A pooled tenant is usable in under 30 seconds at p99; a dedicated cell in under an hour.
• Data residency. Each tenant picks India, the EU or the US, and its databases, backups, search indexes, attachments and logs with personal data stay there.
• Cost. A free tenant must cost under ₹50 a month in infrastructure.

Out of scope

• Billing and invoicing. The platform emits usage events; turning them into invoices is its own design: usage metering and billing.
• The helpdesk features themselves: ticket assignment rules, email parsing, the chat widget.
• Product analytics and the data warehouse, except where they touch isolation (see the follow-ups).
• Third-party apps and a marketplace.
• The insides of SAML and OpenID Connect. We assume a standard identity library; see OAuth 2.0 and OpenID Connect.

The set-up

Plan the approach

Say the shape early: a control plane that knows about tenants, and a data plane of identical cells that serves them. The control plane creates tenants, decides where they live, holds their configuration and moves them; it may be slow but must be correct. The data plane serves requests and must keep working when the control plane is down. Build each functional requirement simply, then spend the deep dives on the hard parts: data isolation, tenant context, routing, noisy neighbours, live moves and safe releases.

Core entities

• Tenant. An immutable ID (a UUID, never reused), a mutable slug such as acme, display name, plan, home region, isolation mode (pooled, enterprise-shared or dedicated), lifecycle state (provisioning, active, suspended, offboarding, deleted) and custom domains.
• Cell. A complete, independent copy of the stack: ID such as in-p07, region, type, size, tested capacity, software version, and whether it is open to new tenants.
• Directory entry. The routing record: tenant → region, cell, database shard and state (active, read-only, moving), plus a version number.
• User and membership. A user belongs to one or more tenants with roles in each. An identity connection per tenant holds its SAML or OIDC settings.
• Tenant configuration. Plan entitlements, limits, feature flags, custom-field definitions and workflows — versioned, so any change can be rolled back.
• Operation. A long-running workflow such as provision, move or offboard, with steps and state, so it can resume after a crash.

API

There are three surfaces: the control-plane API for sign-up and operators, the tenant-facing product API, and tenant configuration.

Create a tenant. Called by the sign-up site. It returns at once; provisioning runs as a workflow.

POST /v1/tenants
Authorization: Bearer eyJhbGciOiJFUzI1NiIs…
Idempotency-Key: 6f3e2d1c-signup-acme
Content-Type: application/json

{
  "slug": "acme",
  "display_name": "Acme Foods Pvt Ltd",
  "plan": "standard",
  "home_region": "in",
  "admin": { "email": "priya@acme.example", "name": "Priya Nair" }
}

{
  "tenant_id": "7c1e4f2a-3b9d-4e61-a0f5-2d8c9b6e1a47",
  "status": "provisioning",
  "operation": "op_01JBX7Q2",
  "url": "https://acme.helpdesk.example"
}

A product call. The host names the tenant; the token proves it. The tenant ID is never taken from the body or a query parameter.

GET /api/v2/tickets?status=open&limit=50
Host: acme.helpdesk.example
Authorization: Bearer eyJhbGciOiJFUzI1NiIs…

The token’s payload, issued by the identity service in the tenant’s region after login or single sign-on:

{
  "iss": "https://login.in.helpdesk.example",
  "aud": "helpdesk-api",
  "sub": "u_48213",
  "tid": "7c1e4f2a-3b9d-4e61-a0f5-2d8c9b6e1a47",
  "tslug": "acme",
  "roles": ["agent"],
  "exp": 1793091323
}

Tenant configuration, by the tenant’s admins:

PATCH /v1/tenants/7c1e4f2a-3b9d-4e61-a0f5-2d8c9b6e1a47/config
Content-Type: application/json

{
  "sso": { "protocol": "saml", "metadata_url": "https://idp.acme.example/saml/metadata" },
  "custom_fields": [{ "key": "order_id", "type": "string", "indexed": true }],
  "features": { "ai_summaries": true }
}

Operator actions, internal and audited:

POST /v1/tenants/{id}:move       body: target_cell = in-d12      → 202 + operation
POST /v1/tenants/{id}:suspend    non-payment: read-only, data kept
POST /v1/tenants/{id}:offboard   export window, then deletion
POST /v1/releases/v413:rollout   body: plan = default-waves
POST /v1/releases/v413:halt      stop the waves and roll back the current one
GET  /v1/tenants/{id}/health     per-tenant SLIs, error budget, cell, version
GET  /v1/cells/in-p07            capacity used, tenant count, version, state

Back-of-envelope estimates

These numbers decide the tenancy model, the cell size and how many database shards you run (method: back-of-envelope estimation).

Tenants and skew. Assume four tiers:

Enterprises send more traffic per seat because of integrations — telephony, chatbots, their own pipelines calling the API. The top 1% of tenants (the 40 enterprises and the 460 biggest business tenants) send roughly 38% of traffic.

Requests. At the busiest hour about 40% of seats are online: 800k agents. An open helpdesk tab makes about 4.5 calls a minute (list refreshes, opening tickets, replies, search): 800k × 4.5 ÷ 60 = 60k requests/s at peak, about 20k/s averaged over a day. About 15% are writes: 9k write requests/s.

The biggest tenant. 20,000 seats, half online at its peak: 10,000 × 4.5 ÷ 60 = 750 req/s from people, about three times that with integrations: ≈ 2,250 req/s, nearly 4% of the platform — almost a small cell on its own.

Cells. Pooled tiers carry 70% of traffic: 42k req/s. Load-test a cell to 4,000 req/s and run it at about 60% at peak: 2,500 req/s. 42k ÷ 2,500 ≈ 17, so run 20 pooled cells for headroom and the three-region split: 49,960 ÷ 20 ≈ 2,500 tenants per cell, about 5% of tenants — exactly the blast-radius target. Enterprises get 4 shared enterprise cells (28 tenants, about 3,150 req/s each on bigger machines) and 12 dedicated cells: 36 cells in all.

Data. About 2 tickets per seat per day: 4M tickets a day. With messages and audit trail a ticket is about 8 KB in PostgreSQL: 32 GB a day, about 12 TB a year. Three years online plus 50% for indexes: 12 × 3 × 1.5 ≈ 54 TB. (Attachments go to object storage: 30% of tickets with a 300 KB file is about 130 TB a year.) Cap a shard at 1.5 TB so a full copy at 200 MB/s takes about two hours. The pooled 80% of data, 43 TB, over 20 cells is 2.2 TB a cell: 2 shards per pooled cell. With 2 per enterprise cell and 1 per dedicated cell, that is 40 + 8 + 12 = 60 shards.

Connections — the number that kills database-per-tenant. Even 5 connections per tenant database is 50,000 × 5 = 250,000 PostgreSQL connections. Each is a server process using several megabytes: over a terabyte of RAM, mostly for free tenants who log in twice a week. Pooled, each cell’s 70 pods hold 10 connections each, which PgBouncer multiplexes onto about 100 server connections per shard: 60 × 100 = 6,000 connections. Migrations tell the same story: one schema change on 50,000 databases at 2 seconds each is about 28 hours of mixed schema versions; on 60 shards it takes minutes.

The directory. 50,000 tenants plus about 8,000 custom domains at roughly 200 bytes each is about 12 MB — small enough for every router to hold in memory.

Cost of a small tenant. Assume a pooled cell costs about ₹12 lakh a month (pods, two PostgreSQL shards with replicas, Redis, search, a share of Kafka). Twenty cells cost ₹2.4 crore. Free tenants are 3 of the pooled 70 points of traffic, about 4.3%: ₹10.3 lakh ÷ 30,000 ≈ ₹34 per free tenant a month. A minimal dedicated stack costs at least ₹50,000 a month; for 30,000 free tenants that is ₹150 crore a month.

High-level design

Build one requirement at a time. By the end of this section the platform works for a few thousand tenants; the deep dives make it work for 50,000 without leaking, melting or stalling.

Customers sign up and get a tenant in seconds

Sign-up calls the tenant API in the control plane. It validates the slug, writes a tenant row in the registry with state provisioning, and starts a provisioning workflow keyed by the operation ID. Every step is idempotent, so a crash halfway through resumes instead of leaving a half-made tenant (see multi-step workflows).

1. Place. Pick the open pooled cell in the tenant’s home region with the most headroom, and a shard in it.
2. Seed. In one transaction, insert the tenant row, default roles, ticket statuses, SLA policies and an inbound email address — about 30 small inserts.
3. Identity. Create the tenant’s realm in the regional identity service and invite the first admin.
4. Publish. Add the directory entry and push a new snapshot to the routers. Only now does acme.helpdesk.example resolve.

A pooled tenant is just rows in shared tables, provisioned in about five seconds. An enterprise that needs its own cell gets one from infrastructure-as-code first. Building a cell takes tens of minutes, so keep one or two empty warm cells per region, hand one over in minutes, and build a replacement in the background.

Every request reaches its tenant’s data, and only that

A request arrives for acme.helpdesk.example. An edge router terminates TLS, reads the host, looks up the tenant and its cell in an in-memory copy of the directory, and forwards the request to that cell. The cell’s gateway verifies the JWT, checks that its tid claim is the same tenant the host named, applies per-tenant limits and builds a tenant context object. Services then open a database transaction that first sets app.tenant_id, and PostgreSQL’s row-level security filters every row against it.

Login is separate. A user types an email at login.in.helpdesk.example; the identity service finds the tenant’s connection and redirects to its own identity provider (Okta, Entra ID, Google Workspace) over SAML or OIDC, or checks a password for small tenants. It then issues a short-lived JWT — 15 minutes, refreshed silently — whose tenant claim is signed, so no client can change it. See authentication, sessions and JWTs.

Small and large tenants get different isolation behind one codebase

Every cell runs the same build, the same schema and the same API. What differs is how many tenants share it. Pooled cells hold about 2,500 free, standard and business tenants. Enterprise cells hold about seven large tenants on bigger machines. Dedicated cells hold one tenant — say, a bank that needs its data in India and its own encryption keys.

This is the AWS whitepaper’s bridge model and Azure’s “vertically partitioned deployments”: share by default, silo where compliance or a contract demands it, but keep the silo a clone of the pooled environment so one team can run both. The concepts are covered in tenancy models: silo, pool and bridge, and cells in cell-based architecture, deployment stamps and geodes. Dedicated cells are priced to cover their cost and offered sparingly; if half your tenants had one, you would be back to running 25,000 separate products.

Each tenant configures limits, flags, custom fields and SSO

Configuration is layered. Platform defaults apply to everyone, the plan overrides them, the tenant’s contract overrides the plan, and feature flags sit on top. A config service in the control plane resolves the layers into one versioned document per tenant and pushes changes to every pod in the tenant’s cell, which keeps them in memory. No request ever queries the plan tables.

Custom fields need care, because every tenant wants different ones — an order_id for a food brand, a policy_number for an insurer:

• Columns per tenant (ALTER TABLE per field) means a schema per tenant, already ruled out.
• Entity–attribute–value rows (ticket_id, field_id, value) are flexible but turn every ticket read into a many-row join and make filtering slow.
• A custom jsonb column, with field definitions stored as tenant metadata, is the practical choice in PostgreSQL. A GIN index answers “tickets where order_id is X” with a containment query, and a per-plan cap (50 fields on standard, 300 on enterprise) bounds row size.

Salesforce describes the metadata-driven idea taken to its end: one shared schema where tenant objects and fields exist only as metadata, values sit in generic “flex columns”, and indexed fields are copied into typed pivot tables. No tenant ever changes the physical schema. SSO settings, workflows and SLA policies are tenant metadata in the same way: versioned, validated by the config service, cached in the cell.

Operators move tenants and roll out releases safely

The last requirement adds two workflows to the control plane: a tenant mover and a release orchestrator. Together with the registry and config service they make up a control plane that is strongly consistent and allowed to be slow, while the routers and cells form a data plane that must keep serving from its last snapshot even if the control plane is down. AWS calls this static stability; see control plane vs data plane.

The mover changes a tenant’s directory entry; the release orchestrator ships a new build one cell at a time. You now have a working platform. The deep dives cover what breaks at 50,000 tenants.

Deep dives

How do we keep one tenant’s data away from another’s?

This is the question the whole design is judged on: no bug may ever show one tenant’s tickets to another.

Bad: shared tables, filtered only in application code. Every query carries WHERE tenant_id = ?, added by the developer or an ORM scope. Cheap and simple, until the one query that forgets: a new report, an admin script, a raw SQL fast path, a join that filters one table but not the other. With 300 endpoints and a release every day, “nobody ever forgets” is not a control. The AWS isolation whitepaper is blunt: it is unrealistic to expect developers never to cross a tenant boundary by accident, so enforcement should not be left to them.

Good: a database (or schema) per tenant. The boundary is physical, so a missing filter cannot leak, and backup, restore, export and deletion are per tenant — offboarding is DROP DATABASE. You pay at scale: 250,000 connections, 28-hour migrations and 50,000 databases to monitor, most of them idle free tenants. A schema per tenant shares a server but creates 50,000 × 40 = 2 million tables, bloating the system catalogue, and still needs 50,000 migrations. Excellent for dozens or hundreds of large tenants; wrong for 50,000.

Great: pooled tables, tenant_id first in every key, and row-level security under the application filter. Keep the app’s WHERE tenant_id = ? — it helps the planner — but add a second layer, enforced by the database, that does not depend on anyone remembering:

-- Migrations run as table_owner. The app connects as app_rw: not the owner, no BYPASSRLS.
CREATE TABLE tickets (
  tenant_id   uuid        NOT NULL,
  ticket_id   bigint      NOT NULL,
  contact_id  bigint      NOT NULL,
  subject     text        NOT NULL,
  status      text        NOT NULL,
  custom      jsonb       NOT NULL DEFAULT '{}',
  created_at  timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (tenant_id, ticket_id),
  FOREIGN KEY (tenant_id, contact_id) REFERENCES contacts (tenant_id, contact_id)
);

ALTER TABLE tickets ENABLE ROW LEVEL SECURITY;
ALTER TABLE tickets FORCE ROW LEVEL SECURITY;   -- the owner obeys the policy too

CREATE POLICY tenant_isolation ON tickets
  USING      (tenant_id = current_setting('app.tenant_id', true)::uuid)
  WITH CHECK (tenant_id = current_setting('app.tenant_id', true)::uuid);

The shared data-access library opens every transaction the same way:

BEGIN;
SELECT set_config('app.tenant_id', '7c1e4f2a-3b9d-4e61-a0f5-2d8c9b6e1a47', true);  -- = SET LOCAL
SELECT ticket_id, subject FROM tickets
 WHERE tenant_id = '7c1e4f2a-3b9d-4e61-a0f5-2d8c9b6e1a47' AND status = 'open'
 ORDER BY created_at DESC LIMIT 50;
COMMIT;   -- the setting disappears here

The details are where leaks hide, so check each against the PostgreSQL documentation:

• Who bypasses RLS. Superusers and BYPASSRLS roles always do; table owners do unless the table has FORCE ROW LEVEL SECURITY. So the app connects as a role that owns nothing, and migrations, the mover and backups use separate, audited roles.
• Transaction scope, not session scope. SET LOCAL (or set_config(…, true)) lasts only until the transaction ends, committed or not. With PgBouncer in transaction-pooling mode your next transaction may run on a server connection another tenant just used — and session-level SET is not supported there anyway. A transaction-scoped setting cannot leak to the next borrower.
• Fail closed. If code forgets to set the tenant, current_setting('app.tenant_id', true) returns NULL (or an empty string that fails the uuid cast), so the query sees nothing or errors. A visible bug, not a leak.
• Keys include the tenant. Unique and foreign-key checks bypass RLS, which the docs call a covert channel. So contact_id references (tenant_id, contact_id), and emails are UNIQUE (tenant_id, email) — otherwise a uniqueness error could tell one tenant that another has a contact with that email.
• Indexes lead with tenant_id, so the policy predicate is an index prefix and costs almost nothing. Large tables can also be partitioned by tenant, as Salesforce partitions its shared tables by org ID.

For enterprises add per-tenant encryption keys: attachments and sensitive fields are encrypted with a data key per tenant, wrapped by a master key in a key management service (encryption and key management), and dedicated cells encrypt whole volumes with the tenant’s own key. That is the bridge model at the data layer: pooled rows for most, separate databases for the few who need them.

How do we carry the tenant context through every hop?

The database is one place data lives; caches, search indexes, queues, object storage and logs are others, each a new chance to mix tenants. Resolve the tenant once, from something trustworthy, then let shared code — not product code — carry it everywhere.

Resolve it once. The router knows the tenant from the host: a subdomain (acme.helpdesk.example) or a custom domain (support.acme.com, verified with a DNS record and given its own TLS certificate through an ACME issuer). API keys also map to one tenant. But a host name is a routing hint anyone can type; the authority is the signed tid claim in the token. The gateway requires the two to match, and nothing in the body or query string can override them — as Azure’s guidance puts it, a tenant identifier in a request does not by itself authorise access to that tenant. A user in two tenants, such as an agency handling support for three brands, gets a token for one tenant at a time.

Carry it everywhere. The gateway builds a context object — tenant ID, cell, plan, user, roles, request ID — and the service-to-service layer forwards the original token (or a short-lived internal token minted by the gateway), never a plain X-Tenant-Id header that any internal caller could forge. Then one shared library applies it:

• Database: the transaction-scoped setting from the previous dive.
• Caches: every key starts with t:<tenant>:, built by the library, so two tenants’ ticket 1041 never collide. Nothing tenant-specific is cached under a global key.
• Search: each document carries tenant_id, and the search client refuses a query without a tenant filter. The largest tenants get their own index so their volume doesn’t skew everyone’s relevance or latency.
• Queues and jobs: the tenant travels in a message header and in the job record. Consumers restore the context from the header before any handler runs; a job scheduled for next Tuesday keeps its tenant ID in its row.
• Object storage: attachments live under a per-tenant prefix, fetched with credentials scoped to that prefix for this request. The AWS whitepaper calls the issuer a token vending machine: it fills a policy template with the tenant ID and returns short-lived credentials that cannot read any other prefix.
• Logs and traces: every line and span carries tenant and cell, so support can answer “what happened to acme at 10:42?” But don’t make the tenant a label on every metric — 50,000 tenants × hundreds of metrics is a cardinality explosion. Compute per-tenant SLIs from logs and traces (observability).

The whitepaper suggests hiding all this in shared libraries or interceptors, “outside the view of developers”. In practice: one internal SDK per language, a lint rule banning raw database and cache clients outside it, and a test harness that runs every endpoint as two tenants.

How do we route 60,000 requests a second to the right cell?

A cell is a complete copy of the stack — gateway, services, workers, PostgreSQL shards, Redis, search, a Kafka cluster or a share of one — that shares no state with other cells. If it fails, only its tenants notice; AWS’s guidance puts it as 10 cells serving 100 requests leaving 90% unaffected when one fails. Each cell has a tested limit (here 4,000 req/s) and runs well below it, at about 2,500 tenants and 2,500 req/s at peak. The router in front decides which cell gets each request.

Bad: hash the tenant ID modulo the number of cells. No lookup, no state. But adding cell 21 remaps almost every tenant, you cannot put a big tenant somewhere deliberate, and you cannot pin a tenant to India. Consistent hashing reduces the reshuffle but still gives you no control over placement.

Good: look up a directory database on every request. Full control: any tenant can live anywhere. But now 60,000 lookups a second hit one database, which becomes the single shared dependency the cells were meant to remove. Caching lookups with a TTL helps until a tenant moves and stale routers keep sending writes to the old cell for a minute.

Great: a full mapping, published as versioned snapshots that every router holds in memory. The registry is the source of truth. On every change, a publisher writes a new snapshot of all 58,000 entries (12 MB) and pushes it to the routers, which also poll every few seconds in case a push is missed. Lookup is a hash-map read in memory. If the control plane, the publisher or object storage is down, routers keep serving from the last good snapshot.

Three refinements make it safe:

• Cells check ownership. Each cell knows which tenants it owns. A request for a tenant it does not own — a stale router, a bug — gets a “wrong cell, need directory v919 or later” reply, and the router refreshes and retries. A misroute can never serve from the wrong database.
• The router stays dumb. It is shared by every cell, so a bug there is a bug everywhere. Keep business logic out of it: no rate limiting by plan, no feature flags, just host → cell. AWS calls it “the thinnest possible layer”.
• Placement is a policy. New pooled tenants go to the open cell in their home region with the most headroom; a cell stops accepting new tenants once its peak load reaches 70% of its limit, and a fresh cell is opened. Filling cells one by one (Azure calls it bin packing) uses hardware better but piles all brand-new, unknown tenants into one cell. Spreading them keeps an unexpectedly huge newcomer from landing next to another one. Enterprises are placed by hand.

This is the full-mapping approach from the AWS cell guidance and the tenant mapping layer from Azure’s tenancy-models guide; the concept article is routing requests to tenants.

How do we stop one tenant from hurting the others?

A noisy neighbour is rarely malicious: a business tenant whose new integration polls the API in a tight loop, a bulk import of five years of tickets, a report that scans 40 million rows. Treat it as resource governance — decide, for every shared resource in the cell (the API, the database, the job workers), how much one tenant may take.

Bad: one shared pool and autoscaling. Autoscaling reacts in minutes and scales the whole cell for one tenant’s mistake. And the database does not autoscale: one tenant’s runaway queries slow every tenant on the shard.

Good: per-tenant rate limits at the gateway, by plan. A token bucket per tenant, and one per user inside it — 600 requests a minute by default, 3,000 on business, the contract number for enterprises — read from the resolved config. Over the limit, 429 with Retry-After; the rate limiter design covers the mechanics. This protects the front door cheaply, but not against a few very expensive requests, nor against background work, which never passes the gateway.

Great: govern every shared resource, in layers.

• Concurrency caps on expensive work. At most 2 report exports per tenant at once, and at most 10 of a shard’s 100 database connections held by one tenant, enforced by a per-tenant semaphore in the data-access library. A tenant at its cap queues behind itself, not behind everyone.
• Time limits per query. The library sets SET LOCAL statement_timeout alongside the tenant: 2 seconds for interactive requests, 60 seconds for reports, which also run on a replica.
• Fair queues for background jobs. SLA timers, automations, emails, imports and exports all run as jobs. In one FIFO, a 200,000-job import from one tenant delays everyone’s SLA reminders by about 40 minutes: at roughly 0.1 s a job on 8 workers, 200,000 × 0.1 ÷ 8 = 2,500 s. Give each tenant its own logical queue and let a scheduler pick round-robin across tenants with work, weighted by plan, with a cap on workers per tenant.

• Shuffle-sharded worker groups. Fair queues share time; they don’t protect against a poison job — an import file that crashes the parser, an automation that loops. Split the cell’s workers into 16 groups and give each tenant two, chosen pseudo-randomly. A tenant whose jobs crash workers takes down at most its two groups, and every other tenant keeps at least one healthy group unless it drew exactly the same pair.

The arithmetic is from the Amazon Builders’ Library: eight workers in fixed pairs make four shards, so one bad customer takes out a quarter of the service; the same eight workers in any pair make 28 combinations, a scope of impact of 1/28. With 16 groups there are 16 × 15 ÷ 2 = 120 pairs, so a cell’s 2,500 tenants sit about 21 to a pair. A poison tenant fully takes down about 20 others, not 2,500.

• Hot-tenant detection and moves. Track per-tenant requests, database time and job seconds from the logs as a top-K per cell. A tenant using more than 15% of its cell for three days running is a placement problem, not a limits problem: the control plane proposes moving it to an enterprise cell (next deep dive). The same numbers give cost per tenant for pricing and the metering pipeline.

You pay in complexity: every shared resource needs a per-tenant knob, all driven by the same resolved config so support can explain why a customer was throttled. Noisy neighbours and fairness goes deeper on weighted fair queuing.

How do we move a tenant without downtime?

Tenants move when they outgrow a pooled cell, buy a dedicated one, or when a cell is retired or a shard rebalanced. The largest business tenants hold about 300 GB.

Bad: stop, dump, restore, start. Maintenance mode, export, import, flip the directory. Obviously consistent, but 300 GB at 100 MB/s is 50 minutes of downtime, plus search reindexing.

Good: copy, stream changes, then a short freeze. Copy a consistent snapshot of the tenant’s rows while it keeps working, then apply every later change using change data capture from the source shard’s write-ahead log, filtered to this tenant. When the target is seconds behind, pause writes, drain, flip. Downtime drops to seconds; the risk is flipping onto a copy that is quietly wrong.

Great: copy, catch up, verify, freeze, flip, and keep a way back. The AWS cell guidance lists the phases — clone as a non-authoritative copy, flip it to authoritative, redirect, forget the old copy. Add verification before the flip and reverse replication after it:

1. Copy. Read the tenant’s rows from a replica inside one consistent snapshot, recording its WAL position. Throttle to about 100 MB/s so the shard’s other 1,249 tenants don’t notice: 300 GB takes about 50 minutes. Rebuild the tenant’s search index in the target, and copy attachments if the region changes.
2. Catch up and verify. Stream changes from that position until lag is under a second. Compare row counts and per-table checksums on both sides, and shadow-read a sample of live queries against the target.
3. Freeze and flip. Set the directory state to moving: gateways answer the tenant’s writes with 503 and Retry-After: 10, reads continue from the source, job queues pause. Once the target has applied everything up to the source’s current position, run a final checksum on the hottest tables, publish acme → in-d12 and unpause. About 10 seconds.
4. Watch, and roll back if needed. Stream changes in reverse, target to source, for seven days, so rollback is another flip. Then delete the source rows in small batches to avoid long locks and bloat.

The control plane refuses moves that break a residency rule — in tenants only move between Indian cells — and runs a few moves per shard at most. Design change data capture covers the streaming part in depth, and online schema and data migrations the general copy–sync–cut-over pattern.

How do we ship one codebase to 50,000 tenants safely?

Cells limit the blast radius of failures; release waves limit the blast radius of your own changes. AWS’s cell guidance and Azure’s deployment stamps both say the same thing: deploy cell by cell, and stop at the first sign of trouble.

The waves. Wave 1 is a canary cell holding your own support team’s tenant and synthetic tenants that exercise every feature around the clock. Then one pooled cell, then three, then the rest of the pooled cells, then enterprise cells, then dedicated cells in each customer’s agreed maintenance window. Each gate checks the cell’s SLIs per tier and per tenant, not just the cell average: a bug that breaks custom-field search for business tenants is invisible in a cell-wide error rate. If the new build burns error budget at more than twice the normal rate, the orchestrator halts and rolls the wave back automatically. Hotfixes take the same path, with shorter bakes. The mechanics of the pipeline itself are in the CI/CD deployment system design.

Deploy is not release. New behaviour ships dark behind a feature flag and is turned on later, per tenant or per plan, from the config service — first for internal tenants, then a beta cohort, then everyone (canary, blue-green and feature flags). Turning a flag off is faster and safer than any rollback.

Schema changes are expand, then contract. Waves mean cells run two builds at once for days, so every migration is backward compatible. Expand — add the nullable column, the new table, the new index (built concurrently) — on all 60 shards before the code wave starts. Backfill in small batches. Contract — drop the old column — only after every cell has run the new build for a week. A rule that keeps this tractable: every cell must be on version N or N−1, and a dedicated cell may defer a release to its window but never skip one.

Per-tenant error budgets. A 99.95% monthly target allows about 22 minutes of downtime per tenant; 99.99% allows about 4. Track them per tenant from request logs. When a cell has burned its budget, it gets no releases except fixes — the error-budget policy, applied per cell.

The final design

Put the pieces together. The request path is the router, one cell and its stores; everything that knows about all tenants at once lives in the control plane, off that path.

Follow one request, and the work around it:

1. Route. A user opens acme.helpdesk.example or support.acme.com. The edge terminates TLS and the router maps the host to tenant acme and cell in-p07 from its in-memory directory.
2. Check. The cell gateway verifies the JWT, requires its signed tid to equal the host’s tenant, confirms the cell owns that tenant, and applies its rate limit: 403 on a mismatch, 429 over the limit.
3. Serve. Services get a tenant context from middleware. The shared library opens each transaction with a transaction-scoped app.tenant_id so row-level security filters every row, and scopes cache keys, search filters and storage credentials the same way.
4. Work later. Events and jobs go to Kafka with a tenant header; fair workers pull from per-tenant queues on two shuffle-sharded worker groups per tenant. Every request and job emits a usage event for metering and billing and a log line tagged with tenant and cell.
5. Log in. The regional identity service sends users to their company’s identity provider over SAML or OIDC and issues short-lived tokens for one tenant.
6. Place and move. Control-plane workflows provision, move (copy, CDC, verify, ten-second freeze) and offboard tenants, publishing a new directory snapshot each time.
7. Configure and release. The config service pushes resolved per-tenant config and flags to the cells; the release orchestrator ships each build in six waves with per-tenant health gates.

Check it against the requirements. A pooled tenant is live in about five seconds, a dedicated one in minutes from a warm cell. Every request is resolved from a signed claim, checked against the host, and filtered by the database even if the application forgets. One build runs in pooled, enterprise and dedicated cells, with layered, versioned configuration. Tenants move with seconds of write pause, a cell failure or bad deploy touches about 5% of tenants, and a free tenant costs about ₹34 a month.

What interviewers look for by level

Follow-up questions

What if a bank demands its data never leaves India and wants its own keys?

Give it a dedicated cell in an Indian region, with its disaster-recovery replica and backups in a second Indian region. Logs and traces with personal data stay in the regional observability stack; the global directory holds just “bharat-bank → in-d05”. For bring-your-own-key, the cell’s database volumes, backups and object storage are encrypted under a key the bank controls in its own key-management account or HSM, using envelope encryption: data keys are wrapped by the bank’s key, cached for minutes. If the bank revokes the key, its data becomes unreadable — which also means a key outage on their side is an outage for them, so put that in the contract. Support engineers get access only through in-region, just-in-time, audited sessions.

What if a tenant churns and asks for a full export and then deletion?

Offboarding is a lifecycle state, not a script. On cancellation the tenant goes read-only and an export job writes its tickets, contacts, configuration and attachments to an encrypted archive, available through a signed link for 30 days. Then a deletion workflow removes its rows by tenant_id in small batches, its search documents, object-storage prefix, cache keys, identity realm and directory entry, and records a deletion certificate. Rows still sit in backups until those age out, say after 35 days; for tenants with their own keys, destroying the key makes backups unreadable at once (crypto-shredding). Privacy laws such as the EU’s GDPR and India’s DPDP Act expect personal data to be erased once it is no longer needed, and your customers will hold you to their deadlines.

What if one tenant wants to import 50 million tickets?

At 8 KB each that is 400 GB — over a quarter of a 1.5 TB pooled shard — so treat the import request as a placement signal: move the tenant to an enterprise cell first, while it is still small and the move takes minutes. Then run the import as low-priority jobs in the tenant’s own fair queue, capped at a few workers, writing in batches of 1,000 with COPY, and slowing down automatically when replica lag or shard p99 rises. At about 2,000 tickets a second it finishes in roughly seven hours, with search indexing following asynchronously. Nobody else in the cell notices.

What if the product team wants cross-tenant benchmarks?

Say, “your first-response time versus similar companies”. Never query the production shards across tenants. Stream changes through CDC into a separate analytics warehouse per region, where a pipeline with its own audited access computes aggregates. Show a benchmark only when a bucket contains enough tenants — say 20 or more — so no single tenant’s numbers can be inferred, and let enterprise contracts opt out. EU and Indian data is aggregated in-region; only the aggregates travel. The OLTP cells never depend on this pipeline (see OLTP vs OLAP).

What if a tenant-ID bug ships and one tenant sees another’s data?

Detect it fast: canary tenants seeded with unique marker strings, and a checker that alerts if a marker ever appears in another tenant’s response, log or export. The gateway can also sample responses and verify that every embedded resource belongs to the requesting tenant. Contain it: a kill-switch flag turns off the offending feature everywhere in seconds, and the release orchestrator rolls back the wave. Then size it from logs that record both the token’s tenant and the tenant of each object served, so each affected customer learns exactly what leaked. Your customers have regulator clocks — GDPR expects a controller to notify its regulator within 72 hours where feasible — so they must hear from you quickly. Afterwards, add the case to the two-tenant test suite.

What if we’re starting from one deployment per customer?

Many SaaS products begin as an installed product run per customer. Don’t migrate everyone at once. First put the router and directory in front of the legacy installs, treating each as a “dedicated cell”, so every customer already has a URL the router owns. Build the pooled cells next to them. Then move customers one at a time with the same mover — export, add tenant_id to every row and key, import, verify, flip — smallest and friendliest customers first. Keep the biggest or most regulated ones dedicated if that suits the business. This is the strangler fig pattern applied to tenancy.

How to say it in the interview

“I’ll design the platform under a B2B helpdesk: 50,000 tenants, two million seats, 60,000 requests a second at peak, and very skewed — the biggest tenant is about 4% of traffic, and free tenants have three seats. The hard requirements are zero cross-tenant access, a blast radius of about 5% of tenants, and a free tenant that costs almost nothing.

So I pool by default and silo where it pays. The product runs in cells — complete copies of the stack. Pooled cells hold about 2,500 small tenants each; enterprises go into shared enterprise cells or their own dedicated cell, with the same build everywhere. A thin router maps the host name to a tenant and a cell from a 12 MB directory it holds in memory, published as versioned snapshots by the control plane, so routing survives a control-plane outage.

Isolation is layered. The cell gateway verifies a JWT and requires its signed tenant claim to match the host. Data lives in shared PostgreSQL tables with tenant_id leading every key, and row-level security filters every query using a tenant setting scoped to the transaction, so a forgotten WHERE clause returns nothing. A shared library carries the tenant into cache keys, search filters, queue headers, storage credentials and logs, so product code never does it by hand.

For noisy neighbours I use per-tenant rate limits by plan, concurrency caps and query timeouts, a queue per tenant with a fair scheduler for background jobs, and shuffle-sharded worker groups — 16 groups, two per tenant, 120 combinations. Tenants that outgrow a cell are moved with a snapshot copy, CDC catch-up, checksums, a ten-second write freeze and a directory flip, with reverse replication for rollback.

Finally, releases go out in six waves, from a canary cell to dedicated cells in their maintenance windows, gated on per-tenant SLOs, with feature flags per tenant and expand/contract migrations across all 60 shards.”

Concepts used in this design

• What is a tenant — the unit everything here is keyed by.
• Tenancy models: silo, pool and bridge — pooled cells for most, dedicated cells for a few.
• Multi-tenant data — tenant-first keys, row-level security, per-tenant keys.
• Multi-tenancy and isolation — defence in depth against cross-tenant access.
• Routing requests to tenants — host names, claims and the directory.
• Cell-based architecture, deployment stamps and geodes — cells as the unit of scale and blast radius.
• Control plane vs data plane — snapshots and static stability.
• Noisy neighbours and fairness — limits, fair queues and shuffle sharding.
• Design a rate limiter — per-tenant token buckets at the gateway.
• Authentication, sessions and JWT — the signed tenant claim.
• Encryption and key management — per-tenant data keys and BYOK.
• Change data capture — moving live tenants.
• Online schema and data migrations — expand/contract across every shard.
• Canary, blue-green and feature flags — release waves and per-tenant flags.
• Metering and usage-based billing — the usage events every request emits.

References and further reading

1. SaaS Tenant Isolation Strategies (AWS whitepaper)docs.aws.amazon.comSilo, pool and bridge models, tier-based isolation, why enforcement should not be left to service developers, PostgreSQL RLS for pooled data, and a token vending machine that issues tenant-scoped credentials.
2. Reducing the Scope of Impact with Cell-Based Architecture (AWS)docs.aws.amazon.comCells and a thin cell router, full-mapping partitions, a statically stable data plane, cell sizing and placement, the clone–flip–redirect–forget migration phases, and deploying cell by cell.
3. Workload isolation using shuffle-sharding (Amazon Builders’ Library)aws.amazon.comEight workers in pairs give 28 shuffle shards, so one bad customer affects 1/28 of the service instead of 1/4 with plain sharding.
4. Tenancy models for a multitenant solution (Azure Architecture Center)learn.microsoft.comTenants versus deployments, a tenant-to-deployment mapping layer, bin-packing placement, and mixing single-tenant and shared deployments.
5. Deployment Stamps pattern (Azure Architecture Center)learn.microsoft.comStamps as scale units serving a set number of tenants, deployment rings, deploying at least two stamps, and why moving tenants between stamps needs custom logic.
6. Map requests to tenants in a multitenant solution (Azure Architecture Center)learn.microsoft.comHost names, token claims and API keys as tenant identifiers, and still checking that each request is authorised for the tenant it names.
7. PostgreSQL documentation: Row Security Policiespostgresql.orgDefault deny, owners bypassing policies unless FORCE ROW LEVEL SECURITY is set, BYPASSRLS roles, and referential-integrity checks as a covert channel.
8. PostgreSQL documentation: SETpostgresql.orgSET LOCAL lasts only until the end of the current transaction, whether it commits or rolls back.
9. PgBouncer: features and pooling modespgbouncer.orgTransaction pooling hands a server connection to a client only for one transaction and does not support session-level SET.
10. Salesforce Platform Multitenant Architecturearchitect.salesforce.comA metadata-driven shared schema keyed by an org ID, with flex columns, pivot-table indexes and partitioning by tenant.