Design an email service (Gmail)

Email is the oldest messaging system you will ever design, and the only one where you do not control the other end. Anyone on the internet can send your users mail over SMTP, and your users expect to write to anyone: a colleague on your service, a recruiter on Outlook, a college friend on Rediffmail. Each mailbox is also an archive nobody cleans up — the IRCTC e-ticket from 2016, every Diwali-sale blast — and a live feed that must show a new OTP within seconds. You will design a Gmail-scale service: SMTP in and out, spam and spoofing, an exabyte-class mailbox store, conversations and labels, real-time sync, and search over a decade of one person’s mail.

Understand the problem

An email service is two systems glued together: a federated transfer system that exchanges mail with thousands of providers over SMTP, with senders who may be lying, and a personal database of billions of mailboxes kept for years. Weak answers design only the second and treat “receive email” as an HTTP POST. Strong answers say early that SMTP’s promises shape the whole write path.

Functional requirements

1. Users should be able to receive email from anyone on the internet, with spam, phishing and malware kept out of the inbox.
2. Users should be able to read and organise their mailbox — conversations (threads), labels such as Inbox, Starred and their own, read and unread — and see new mail on every device within seconds.
3. Users should be able to send email to users on our service and on any other provider, with attachments up to 25 MB, the limit Gmail documents for personal accounts.
4. Users should be able to search all of their own mail, including messages from years ago, by words and by operators such as sender, date and “has an attachment”.

Non-functional requirements

• Scale. Google reported 1.5 billion monthly Gmail users in 2019. Design for 2 billion accounts and 1 billion daily users — my rounding, not a published figure.
• Durability above everything. Once we say 250 OK to a sending server, the message is never lost: in three zones before the reply, 11 nines in the blob store after.
• Availability. 99.99% for reading and receiving. SMTP senders retry for days, so a short inbound outage delays mail rather than losing it — but nobody wants a two-hour-late OTP.
• Latency. Inbox page: p50 under 200 ms, p99 under 500 ms. Search: p50 under 300 ms, p99 under 1 s. Mail between our own users: delivered within 2 s at p50, 10 s at p99.
• Consistency. Read-your-writes for a user’s own actions (sent mail is in Sent at once); search and other devices may lag by seconds.
• Storage. Each account gets 15 GB free, shared with Drive and Photos, and most users never delete anything.

Out of scope

• Calendar, chat and video. Separate products; see Design Slack or Discord.
• Ads and smart features such as Smart Reply. They read the same data but do not change the core design.
• Admin consoles and enterprise policy (retention, legal hold). Configuration on top of this design.
• End-to-end encryption. It stops server-side spam filtering and search; see the follow-up questions.
• POP3. IMAP and our own API cover every modern client.

The set-up

Plan the approach

Build one requirement at a time — receive, read, send, search — then spend most of the time in the deep dives. This problem is won on details no box diagram shows: when to say 250 OK, where to reject spam, how a mailbox sits on disk, and what to do when a remote server says “try again later” for three days. Say the shape early: “an SMTP edge, a mailbox core keyed by user, and clients fed by a per-user change log.”

Core entities

• Account. A user ID, addresses, a quota and the bytes used.
• Message. An internal ID (time-ordered within the account), the RFC 5322 Message-ID header, a thread ID, a set of label IDs, a header summary, its size, and blob references for the body and attachments.
• Thread. A conversation: an ID, a normalised subject and its messages. Its labels are the union of its messages’ labels.
• Label. System labels (INBOX, UNREAD, STARRED, SENT, SPAM, TRASH) and user labels (“Travel”), with total and unread counts.
• Blob. Immutable bytes — a compressed body or an attachment — stored under the SHA-256 of its content.
• Change record. A strictly increasing history ID and what changed.
• Delivery job. Outbound mail for one destination: recipients, attempts, next try.

The metadata is easiest to picture as three tables that share one key prefix, the user ID:

-- One account's rows are one contiguous key range: user_id first.
CREATE TABLE messages (
  user_id     bigint,
  message_id  bigint,          -- time-ordered, unique within the account
  thread_id   bigint,
  label_ids   set<int>,        -- INBOX, UNREAD, STARRED, user labels ...
  from_addr   text, subject text, snippet text, sent_at timestamp,
  size_bytes  int,
  body_blob   text,            -- sha256 of the compressed body
  attachments list<frozen<attachment_ref>>,
  PRIMARY KEY ((user_id), message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);

CREATE TABLE label_index (     -- "INBOX, newest conversation first"
  user_id bigint, label_id int, last_activity bigint, thread_id bigint,
  PRIMARY KEY ((user_id, label_id), last_activity, thread_id)
) WITH CLUSTERING ORDER BY (last_activity DESC);

CREATE TABLE history (         -- the per-user change log
  user_id bigint, history_id bigint, change blob,
  PRIMARY KEY ((user_id), history_id)
);

API

There are two very different surfaces: SMTP for other mail servers, and an HTTPS API plus a push channel for our own apps.

SMTP, inbound. Another provider delivering to Priya looks like this. The reply to the final dot is the moment that matters:

S: 220 mx1.ourmail.example ESMTP ready
C: EHLO mta7.bank.example
S: 250-mx1.ourmail.example  250-STARTTLS  250-SIZE 52428800  250 8BITMIME
C: STARTTLS   ...TLS handshake, then EHLO again...
C: MAIL FROM:<alerts@bank.example>
S: 250 2.1.0 OK
C: RCPT TO:<priya@ourmail.example>
S: 250 2.1.5 OK                       (550 5.1.1 if no such user)
C: DATA
S: 354 Go ahead
C: ...headers, body, attachments...  .
S: 250 2.0.0 OK queued as 7f3a9c      (only after a durable, replicated write)

Read the mailbox. List conversations under a label, newest first, with a cursor:

GET /v1/threads?label=INBOX&limit=50&pageToken=eyJ0cyI6...

{
  "threads": [
    { "threadId": "t_77", "subject": "Your e-ticket for 12951 Rajdhani",
      "snippet": "PNR 4521...", "from": "IRCTC", "messageCount": 1,
      "labels": ["INBOX", "UNREAD"], "lastActivity": "2026-10-27T09:14:03Z" }
  ],
  "nextPageToken": "eyJ0cyI6MTcy...",
  "historyId": "1047"
}

GET /v1/threads/t_77 returns the messages with sanitised HTML and attachment metadata; attachments download from short-lived signed URLs.

Change things. Read state and stars are label edits too:

POST /v1/messages:batchModify
Content-Type: application/json

{ "ids": ["m_9001", "m_9002"], "addLabels": ["STARRED"], "removeLabels": ["UNREAD"] }

The response carries the new historyId, the version of the mailbox that includes this change.

Send. Attachments upload first (resumable); an idempotency key makes a retried tap on Send produce one email, not two:

POST /v1/messages:send
Idempotency-Key: 6f1c2e0a-91b4-4c55-a1f7-0d9e2b7c4a11
Content-Type: application/json

{ "to": ["rahul@ourmail.example", "meera@outlook.example"],
  "subject": "Re: Goa trip", "inReplyTo": "m_8800",
  "bodyHtml": "<p>Booked! Sharing the tickets.</p>",
  "attachments": ["att_sha256_9f3c..."] }

{ "messageId": "m_9010", "threadId": "t_40", "historyId": "1048" }

Sync and push. A device that knows historyId 1041 asks for everything since:

GET /v1/history?since=1041&limit=500

{ "changes": [
    { "id": "1042", "type": "messageAdded",  "messageId": "m_9001" },
    { "id": "1045", "type": "labelsChanged", "messageId": "m_8990",
      "added": [], "removed": ["UNREAD"] } ],
  "historyId": "1047", "hasMore": false }

Open apps hold a WebSocket to a push gateway, which sends only a hint: {"type":"mailbox","historyId":"1047"}; phones get it through FCM or APNs. Search is GET /v1/search?q=from:irctc has:attachment older_than:1y, returning thread IDs and snippets.

Back-of-envelope estimates

Show the arithmetic (method: back-of-envelope estimation).

Inbound mail. Assume each of 2 billion accounts receives 25 messages a day that we accept. 2B × 25 = 50 billion messages a day; 50B ÷ 86,400 s ≈ 580,000 messages/s, about 1.2M/s at peak. Assume the edge rejects as many again during the SMTP conversation, so the MTAs handle about 1.2M transactions/s on average.

Concurrent SMTP sessions. A session lasts about 2 s, so by Little’s law 1.2M/s × 2 s ≈ 2.4 million concurrent sessions; at 20,000 per host that is 120 MTA hosts, about 250 with headroom.

Sent mail. 1B daily users × 5 = 5B a day ≈ 58,000/s. If 40% go to other providers, outbound SMTP is 2B a day ≈ 23,000 messages/s; the other 3B are local deliveries, already inside the 50B.

Reads. 1B × 20 mailbox views = 20B a day ≈ 230,000 reads/s, about 500,000/s at peak. Search: 1B × 0.5 = 500M a day ≈ 6,000 searches/s — few, but each one heavy.

Mail bytes. Assume 50 KB per message, attachments included. Per account: 25 × 50 KB = 1.25 MB a day, × 365 ≈ 456 MB a year. In total: 50B × 50 KB = 2.5 PB a day, × 365 ≈ 912 PB — about 0.9 exabytes of new mail a year. An average account would take 15 GB ÷ 456 MB ≈ 33 years to fill its quota; heavy users fill it in a few. Inbound bandwidth: 2.5 PB ÷ 86,400 s ≈ 29 GB/s ≈ 230 Gbit/s.

Metadata. About 1 KB per message with its index entries: 50B × 1 KB = 50 TB a day, × 365 ≈ 18 PB a year, 55 PB with three replicas. Each new message is about four row writes (message, two label-index entries, a change record): 580k × 4 ≈ 2.3M row writes/s.

What the numbers decide:

• Bytes, not requests, dominate, so compression, deduplication and cold tiers are the biggest cost levers.
• Everything a user does touches one user’s data, so partition by user and almost every request is single-partition.
• Even metadata is sized by bytes. At about 10 TB a node, 55 PB is roughly 5,500 nodes per year of mail — far more than 2.3M writes/s needs.
• Search is low QPS and high cost per query, which favours small per-user indexes.

High-level design

Build one requirement at a time. By the end it works for one region at moderate traffic.

Users should be able to receive email from anyone on the internet

A sending server finds us through DNS. Our domain publishes MX records such as MX 5 mx1.ourmail.example and MX 10 mx2.ourmail.example; RFC 5321 tells senders to try the lowest preference first and to randomise among equal ones, which spreads load for free. Each name resolves to load balancers in several regions, in front of a fleet of inbound MTAs (mail transfer agents).

Follow one message from a bank to Priya:

1. Connect. The bank’s MTA looks up our MX records, connects to an inbound MTA and upgrades to TLS.
2. Check the envelope. At RCPT TO the MTA checks a cached directory: does Priya exist, is she under quota? If not, it answers 550 or 452 there, so the sender’s server writes any bounce.
3. Take the message. After DATA, it checks SPF, DKIM and DMARC and runs cheap spam checks.
4. Make it durable, then reply. It writes the message to the inbound queue, replicated across three zones, and only then answers 250 OK. MTAs keep no state.
5. Deliver. Delivery workers run full spam, phishing and virus checks, pick thread and labels, store bytes in the blob store, and write the message row, label index, counts and a change record into Priya’s partition of the mailbox store in one transaction.

A message to 40 of our users is one transaction with 40 RCPT TO lines: one queue entry, one body, 40 mailbox writes.

Users should be able to read and organise their mailbox

Every read is about one user, so the mailbox store is partitioned by user ID: an account’s messages, label index, counts and change log are one contiguous key range on one shard. Listing the inbox is one range scan of the label index — (priya, INBOX), newest first, 50 entries — plus one batched read of those thread summaries. Opening a conversation reads its rows, then fetches bodies from the blob store through a cache.

Organising is just editing labels: “Archive” removes INBOX, “Mark as read” removes UNREAD, “Star” adds STARRED. Each edit updates the message rows, label index and unread counts in one transaction and appends a change record with the next history ID.

That change log makes mail appear in real time. A push gateway holds WebSockets for open sessions and uses FCM or APNs for phones (notification system). When Priya’s log moves to 1047, each device gets a tiny hint and asks GET /v1/history?since=1041 for exactly what changed; a lost hint breaks nothing.

Users should be able to send email

Sending is receiving in reverse, with one extra rule: the sender sees their own message immediately.

1. Upload attachments first (resumable, so a dropped connection on a Bengaluru metro ride resumes); each gets an ID from its content hash.
2. Save first. messages:send carries an idempotency key. The send service writes the message into the sender’s partition with the SENT label, in one transaction with a change record and an outbox entry — the transactional outbox pattern. The returned history ID gives read-your-writes on every device.
3. Route. Our recipients go straight to the delivery workers, the inbound pipeline minus SMTP; others go to an outbound queue per destination.
4. Deliver outside. Outbound MTAs sign with our DKIM key and deliver over SMTP, retrying for days and bouncing permanent failures.

Users should be able to search all of their own mail

An indexer tails each user’s change log, extracts text from headers, bodies and some attachments, and adds it to a search index for that user. The search service runs a query against that one index, gets back message IDs ranked mostly by date, and hydrates the top 20 from the mailbox store, so labels and read state are current even if the index is seconds behind.

You now have a working email service; the deep dives make it survive 50 billion messages a day.

Deep dives

How do we accept inbound mail without ever losing it?

RFC 5321 is blunt: a 250 reply to DATA means “accepting responsibility for delivering or relaying the message”, which must not be lost “for frivolous reasons, such as because the host later crashes”. It also asks the receiver to reply quickly, because a sender that times out sends again. So the reply must be durable and fast.

Bad: reply 250, then process. Buffer in memory, say 250, hand off. Fast, but every deploy or crash loses acknowledged mail; a rolling restart of 250 hosts would drop thousands of messages that senders have already deleted.

Good: write to local disk with fsync, then 250. The classic MTA spool survives crashes and reboots. But the message lives on one disk until delivered, so a dead disk loses it, and every MTA is stateful: you must drain spools before replacing hosts.

Great: replicated write, then 250. The MTA appends envelope and message to the inbound queue, which acknowledges once two of three zones have it. Large messages go to the (multi-zone) blob store first, with a pointer in the queue. A cross-zone write costs a few milliseconds inside a two-second session, and the MTAs keep no state: kill any of them, and the worst case is a sender that retries.

Three more details make the promise hold:

• Validate before you accept. Check recipient and quota at RCPT TO and reject there. After a 250, RFC 5321 obliges you to bounce failed deliveries — and a bounce to a forged sender is spam you sent.
• De-duplicate in the consumer. The queue is at-least-once and lost 250s cause resends, so the worker keys each delivery on recipient plus a hash of Message-ID and body, keeps keys for a week, and skips repeats — an idempotent consumer. Message-ID alone is optional and spammers reuse it.
• Tempfail sparingly. If the queue cannot confirm a write in, say, 30 seconds, answer 451 4.3.0 and the sender keeps the message: backpressure built into SMTP. But the RFC suggests senders wait at least 30 minutes to retry, so a 451 is an incident tool, not load balancing.

Three zones cover everything except losing a whole region before its queue drains. A synchronous cross-region replica would close that gap, but it ships all 230 Gbit/s of inbound mail across regions; I would replicate asynchronously, accept seconds of exposure in a regional disaster, and say so.

How do we keep spam, phishing and spoofing out?

The internet sends more junk than mail, some of it claiming to be your user’s bank. You need authentication (is the sender who it claims to be?) and classification (does this user want it?). Authentication rests on three standards:

• SPF (RFC 7208): a domain lists in DNS which IPs may send with its domain in the envelope MAIL FROM. Evaluation may trigger at most 10 DNS-querying terms. SPF checks the envelope, not the From: users see, and breaks on forwarding.
• DKIM (RFC 6376): the sender signs chosen headers and a body hash, naming its domain (d=); the public key sits in DNS, such as s1._domainkey.bank.example. The signature travels with the message, so it usually survives forwarding.
• DMARC (RFC 7489) ties both to the visible From: domain: a message passes when SPF or DKIM passes and the passing domain aligns with the From: domain. The owner’s policy at _dmarc.bank.example says none, quarantine or reject.

Authentication only proves who sent a message; authenticated mail can still be spam. So classification follows, ordered by cost: cheapest checks first, on the most messages.

• Connection, envelope, message. Reputation, reverse DNS, TLS and per-IP rate limits cost almost nothing; unknown recipients get 550 5.1.1 (many from one IP means harvesting: throttle); SPF, DKIM, DMARC and a fast first-pass model run while the sender waits. Reject the obvious; tempfail the suspicious — real servers retry, spam cannons rarely do.
• Full classifier. After the 250, a model scores the message. Google’s 2019 write-up says Gmail blocks more than 99.9% of spam, phishing and malware, weighs thousands of signals per email, and personalises per user. Signals: sender, IP and link reputation (brand-new domains are a classic tell), authentication, content, and whether the sender is a known contact.
• Attachments. Block executables, scan with several antivirus engines, and detonate risky types (macro documents, archives) in a sandbox — seconds to minutes, so hold only those messages or lock the attachment until the verdict.

The arithmetic explains the order: a 20 ms classifier on every peak-time message needs 1.2M × 0.02 s = 24,000 cores. Every message a 0.1 ms IP check rejects is 20 ms you never spend.

Report spam clicks become training labels and update sender reputation within minutes. And big providers now make bulk senders share the cost: since February 2024 Google requires anyone sending more than 5,000 messages a day to Gmail to use SPF, DKIM and aligned DMARC, to support one-click unsubscribe on marketing mail (RFC 8058’s List-Unsubscribe-Post), and to keep user-reported spam below 0.3%, ideally below 0.1%. We would enforce the same rules.

How do we store billions of mailboxes?

The data has two shapes. Metadata — labels, read state, threads, header summaries — is small (about 1 KB a message), changes constantly, and is read one user at a time in sorted order. Content — bodies and attachments — is large, immutable, and mostly read in its first few days. Storing them the same way is the classic mistake.

Bad: one table sharded by message ID, bodies inline. Listing one inbox is a scatter-gather across every shard, and marking a message read rewrites a 50 KB row.

Good: shard by user ID, bodies inline. An inbox is now one range read. But shards fill with cold bytes, metadata competes with bodies for cache, rebalancing a user copies gigabytes, and the largest accounts — FastMail describes one with over half a million messages — become huge, hot partitions.

Great: metadata by user, content in a blob store. Keep metadata in a sharded store keyed by user ID, and bodies and attachments in blob storage, referenced by hash.

The metadata store. Key rows by user_id, then time-ordered message_id, with label index and change log under the same prefix. In a range-partitioned store (Bigtable or Spanner style) a huge account just spans several tablets; in a hash-partitioned one such as Cassandra or DynamoDB, bucket the largest accounts by year to keep partitions small.

Transactions stop at the account boundary. Google’s Megastore paper uses exactly this example: each email account is an entity group, transactional inside — a user who sends or labels a message sees it even after failover — while mail between accounts travels asynchronously. A message to 40 colleagues is 40 independent transactions.

Read-your-writes, cheaply. Writes go to the partition leader and return a history ID; reads carry the last one seen as minHistoryId, and a follower serves them only if it has caught up. Per-user consistency, no global strong consistency (strong vs eventual consistency).

The blob store. Split each message along its MIME structure. Store large parts — Dovecot’s single-instance attachment storage defaults to a 128 KB threshold — under the SHA-256 of their content, and pack headers, small parts and the body into one compressed blob. Then:

• Deduplicate where it happens. A circular to 40 colleagues is stored once; forwards re-attach the same PDF. Outside bulk mail is personalised (a tracking link per person), so whole-message deduplication rarely helps. Assume 15% of attachment bytes are duplicates.
• Compress bodies. HTML and text shrink about 4×; PDFs and JPEGs are already compressed.
• Tier by age. Metadata is always hot. Bodies stay on replicated SSD for 30 days, then move to erasure-coded HDD, as Dovecot’s “alternate storage” does on one server. A 2019 email opens tens of milliseconds slower.
• Delete lazily, shred instantly. Encrypt each blob with its own data key; when the last reference goes, destroy the key (crypto-shredding) and let garbage collection reclaim bytes later. Never let a client skip an upload by sending only a hash — that turns deduplication into an oracle for “does anyone have this file?”.

The arithmetic: assume bodies are 40% of bytes and shrink 4×, attachments 60% and do not: 0.4 ÷ 4 + 0.6 = 0.7, so 912 PB × 0.7 ≈ 638 PB. Deduplicating 15% of attachment bytes: 0.1 + 0.6 × 0.85 = 0.61, so 912 × 0.61 ≈ 556 PB. Three replicas need 556 × 3 ≈ 1,668 PB of raw disk; erasure coding at 1.5× (six data plus three parity fragments) needs 556 × 1.5 = 834 PB. Keeping the newest 30 days triple-replicated adds little: 556 × 30 ÷ 365 ≈ 46 PB, about 137 PB raw.

One more rule: charge quota on logical bytes, what the user sees, not deduplicated physical bytes — otherwise free space would depend on what others sent. Check it at RCPT TO and answer 452 4.2.2 when full.

How do we group messages into conversations, and model labels?

Threads belong to a mailbox — Priya and Rahul may group the same messages differently — so threading runs in the delivery worker, per recipient.

Bad: thread by subject. Strip Re: and Fwd: and group equal subjects. Every “Invoice” and “Hello” from different people collapses into one conversation, and a reply with an edited subject splits away.

Good: thread by headers. RFC 5322 defines the links: a reply’s In-Reply-To holds its parent’s Message-ID, and References holds the chain of ancestors. Join the thread of any ID you recognise. Precise, but real mail is messy: clients drop References, replies arrive before the message they answer, and people reply to an old email just to reuse the address, then change the subject.

Great: headers and subject together. RFC 8621 (JMAP for Mail) suggests this rule: two messages share a thread if a message ID appears in both (in Message-ID, In-Reply-To or References) and their subjects match after stripping prefixes. Gmail’s API documents the same shape for adding a message to a thread.

Implement it with a per-user table mapping each Message-ID to a thread ID: look up all referenced IDs in one batched read, check the subject, join or start a thread, then record the new ID. When messages arrive out of order — C references A and B, which started separate threads — merge them and emit a “thread merged” change so devices regroup. Cap threads at a few hundred messages so a self-replying alert cannot grow one forever.

Labels, not folders. A folder is a place: a message lives in exactly one. A label is a tag: Gmail’s API calls labels many-to-many, and JMAP lets an email sit in several mailboxes, “often exposed in a user agent as labels rather than folders”. Treat Inbox, Unread, Starred, Spam, Trash and user labels alike, as one set on the message row.

A thread’s labels are the union of its messages’ labels, and “archive this conversation” removes INBOX from each message. All in one partition, that is one transaction that also updates the label index and unread counts — so the app badge is right, not “eventually right”. Recompute counts in the background to repair drift from bugs.

How do clients stay in sync and see new mail in real time?

A browser tab, a phone back from an hour in a tunnel, and Outlook on a laptop must all converge without re-downloading the mailbox, and show new mail within seconds. Three protocols do this differently.

IMAP (RFC 9051) syncs one folder at a time. Each message gets a 32-bit, strictly ascending UID; if the server must renumber, it changes UIDVALIDITY and clients drop their cache. IDLE pushes updates for the selected folder, re-issued at least every 29 minutes, and CONDSTORE and QRESYNC let a reconnecting phone fetch only what changed in one round trip. The cost: a long-lived connection per watched folder.

JMAP (RFC 8620 and RFC 8621) is JSON over HTTPS, built for this. Each data type has a state string; Email/changes with sinceState returns the IDs created, updated and destroyed since, or cannotCalculateChanges (“resync”). Push sends only StateChange objects over SSE or web push, and batched calls let a phone query and fetch in one round trip.

A history API is what Gmail’s public API does: a full sync first, then history.list from a startHistoryId. Google says history is typically kept for at least a week; an expired ID returns 404 and forces a full sync.

All three are one idea underneath: a per-user change log and a cursor. Every mailbox write appends a record with the next history ID in the same transaction, so the log never disagrees with the data. A device stores the last ID it applied, push tells it the latest, and it pulls the difference. The same log feeds the search indexer.

The recommendation: our apps use a JMAP-shaped API on the change log; IMAP is a gateway for third-party clients that maps labels to folders and keeps a UID counter per label. Four details make it scale:

• Push is a hint, pull is the truth. Hints may be lost or duplicated harmlessly; phones get them through FCM or APNs, browsers over a WebSocket or SSE connection.
• Coalesce bursts. Fifty messages in a second should produce one hint.
• Size the fleet. If 300 million devices are connected at peak and a host holds about 200,000 idle WebSockets, that is 300M ÷ 200k = 1,500 hosts, sharded by user ID.
• Bound the log. Keep 30 days of history — at 50 changes a day of 100 bytes, 30 × 50 × 100 B = 150 KB per user. Older cursors expire and resync, newest conversations first.

How do we search ten years of one person’s mail in under a second?

Every query is scoped to one user and usually wants newest first; each corpus is small, the total is trillions. That decides the index.

Bad: one global inverted index. Partition by term, and the posting list for “invoice” holds everyone’s invoices — billions of entries thrown away to find one user’s 40. Partition by document, and every query fans out to every shard. Either way, work grows with the whole corpus, not one mailbox.

Good: a shared cluster, routed by user. Use Elasticsearch with the user ID as routing key, so a query touches one shard. It works up to a point, but each shard mixes millions of users: every query depends on a user filter (one bug is a privacy incident), and deleting an account is millions of small deletes.

Great: one small index per user. FastMail’s engineers described this in 2014: one Xapian database per user, stored with the mailbox. It is opened when the user searches and costs only disk otherwise. Isolation becomes physical — a query never opens another user’s index — and deleting an account deletes its files.

How the per-user index is built and kept fresh:

• Index from the change log. The indexer tails each user’s history: on “added” it extracts text from headers, body and (capped) attachments; on “deleted” it removes the document. Labels and read state are not indexed — the search service filters and hydrates them from the mailbox store — so a label change never rewrites the index.
• Tier the writes. FastMail found that indexing each message straight into the main database overloaded its servers, since one email causes many index updates. Their fix: index into a small in-memory database, compact it to disk hourly if more than half full (otherwise in quiet hours), and search all tiers at once — a log-structured merge tree in disguise.
• Read-your-writes on the tail. The index lags by under 30 seconds at p99. Each index records the last history ID it applied, and the search service matches the few newer messages directly from metadata, so a two-second-old OTP email is searchable.
• Warm it early. A cold index on HDD may take a few hundred milliseconds for its first query, so start loading it when the app opens.

The cost is real. FastMail’s post shows about 4.4 GB of compacted index for an account of over half a million messages — under 10 KB per message. At 9 KB × 50B messages a day, that is 450 TB a day, about 164 PB of index a year, roughly 30% on top of the 556 PB of stored mail; so cap attachment text and compress postings.

How do we deliver outbound mail reliably to other providers?

Now we are the sender: about 23,000 messages a second to tens of thousands of destinations, each with its own limits, outages and opinion of us.

Bad: one FIFO queue. When a big provider throttles us or a small ISP is down, its messages block everything behind them, and retries hammer the struggling host.

Good: one queue per destination. Each gets its own queue and concurrency, so a slow provider delays nobody else. Key queues by receiving MX host, not recipient domain: thousands of company domains share a few providers, and Google’s guidelines apply limits by MX domain.

Great: per-destination queues, adaptive pacing, long and polite retries. The scheduler keeps state per destination — open connections, recent reply codes, host reachability:

• Add slowly, back off fast. One more connection after a run of successes; halve them after a 421 or throttling 4.7.x. For its 4.7.28 rate-limit reply, Google advises pausing at least 10 minutes, then reopening one connection and adding them one at a time.
• Track hosts, not messages. RFC 5321 says to keep a list of unreachable hosts: when a domain is down, one probe per interval tells you, instead of 10,000 messages each waiting out a timeout.
• Retry for days. The RFC wants at least 30 minutes between retries and a give-up time of at least 4–5 days; failures are usually transient. Schedule: now, after 30 minutes, then every 3 hours with jitter until day 5 — about 41 attempts.
• Bounce properly. A 5xx is permanent: post a delivery status notification (RFC 3464’s multipart/report) to the sender, from the empty reverse path <> so bounces never bounce. After 4 hours, send a “still trying” notice.

Size the deferred queue with Little’s law: if 2% of outbound messages wait 6 hours on average, that is 2B × 2% = 40M a day × 6 ÷ 24 ≈ 10 million messages waiting at any moment — a durable store, not memory.

Delivery also depends on reputation, judged per IP and per domain: publish SPF, DKIM-sign everything (rotate keys via new selectors), publish DMARC p=reject for our domain, keep reverse DNS valid, and use TLS (MTA-STS enforces it where receivers ask). The biggest threat is our own users — hijacked accounts and bulk sign-ups — so cap sending per account, run an outbound spam classifier, and split IPs into pools so an abuse burst damages a small pool, not everyone’s mail. Warm new IPs slowly; Google warns that suddenly doubling volume can trigger rate limits.

The final design

Mail enters and leaves through the SMTP edge at the top; each user’s state lives in one partition in the middle; one per-user change log drives push and search.

Follow the main paths:

1. Receive. A sender finds our MX records and connects to an inbound MTA, which upgrades to TLS, checks recipient and quota at RCPT TO, verifies SPF, DKIM and DMARC, and rejects obvious junk inside the session.
2. Commit. The MTA appends the message to the inbound queue, waits for two of three zones, and only then replies 250 OK.
3. Deliver. Workers de-duplicate, classify, scan and thread the message, store bytes in the blob store by hash, and write row, label index, counts and change record into the recipient’s partition in one transaction.
4. Notify. The change log tells the push gateway the new history ID; devices get a hint and pull /history from their cursor.
5. Read and organise. Clients list, open and label through the mailbox API, one partition per request; attachments come from signed blob URLs; history IDs give read-your-writes.
6. Send. The API saves to SENT with an outbox record in one transaction and hands the message to the same queue; local recipients are delivered as in step 3, in about a second.
7. Deliver outside. External recipients go to a queue per destination MX; outbound MTAs sign with DKIM, pace by reply codes, retry for five days and bounce permanent failures.
8. Search. The indexer tails the change log into each user’s tiered index; queries hit that one index plus the unindexed tail and hydrate from the mailbox store.

Check it against the requirements. Durability: nothing is acknowledged before it is in three zones, and every later step is idempotent and replayable. Availability: stateless MTAs, workers and APIs; MX records span regions, and SMTP retries cover short outages. Latency: an inbox page is one range scan; internal delivery is a queue hop plus one transaction; search touches one small index. Consistency: read-your-writes per user through history IDs; search and other devices converge in seconds. Cost: about 0.9 EB of new mail a year needs roughly 0.83 EB of raw disk instead of 2.7 EB.

What interviewers look for by level

Follow-up questions

What if one mailbox receives a huge volume, like a support address?

A support@ address receiving 100 messages a second makes one hot partition. Group-commit its deliveries every few hundred milliseconds, bucket its rows and search index by year, and rate-limit per recipient at the edge with 451, which real senders retry. If a team works one inbox, it is really a ticketing product: give it its own store.

What if a whole region goes down?

MX records span regions, so senders fail over on their own, and unacknowledged mail waits in their queues. Each mailbox has a home region and an asynchronous copy elsewhere; failover promotes it, losing at most seconds. Devices’ cursors may now point past the new leader’s log, so give each partition a log epoch — like IMAP’s UIDVALIDITY — and when it changes, devices re-fetch recent state.

What if a newsletter goes to 10 million of our users at once?

RFC 5321 requires servers to accept at least 100 recipients per transaction, so the sender needs about 100,000 transactions. Throttle each sender with 451s; their queue absorbs the rest. Recipients spread across all shards, so the writes are wide, not hot. Cache the classifier’s verdict per campaign and adjust per user, coalesce hints, and file it under a promotions category so it does not ring 10 million phones.

What if users want end-to-end encryption?

The server then sees only ciphertext, so content spam filtering (only authentication and reputation remain), server-side search and previews break, and key management and interoperability (S/MIME, OpenPGP) are hard. A practical design offers it per message as opt-in, keeps routing headers in the clear, and builds a client-side index for those messages.

What if a user deletes their account, and some of their attachments are shared with other mailboxes?

Deleting the account removes its rows, index files and change log, and drops one reference from every blob it used. Blobs other mailboxes still reference stay — those users got their own copy. Blobs that reach zero references have their keys destroyed at once (crypto-shredding makes backups unreadable too), and garbage collection reclaims the space later. Publish a deletion window, such as 30 days, that covers backups.

How to say it in the interview

“I’ll design email at Gmail scale: about 2 billion accounts, 50 billion delivered messages a day, and one rule — once we say 250 OK to a sending server, we never lose that message.

Senders find our inbound MTAs through MX records. The MTAs check the recipient at RCPT TO, verify SPF, DKIM and DMARC alignment, run cheap reputation checks, and reject junk inside the session so the sender owns the bounce. They write the message to a queue replicated across three zones and only then reply 250, so MTAs stay stateless and a lost reply only causes a duplicate the workers drop.

Workers classify, scan and thread each message by reply headers plus subject, then write it into the recipient’s partition: the store is keyed by user ID, so a delivery is one transaction updating the message row, label index, counts and a per-user change log. Labels are a set; Inbox and Unread are just labels. Bodies and attachments go to blob storage by content hash — deduplicated, compressed, erasure-coded after 30 days — because the real cost is 0.9 exabytes of new mail a year.

Clients read one partition per request, with read-your-writes through history IDs. The change log drives the rest: push gateways send hints over WebSockets or mobile push, and devices pull changes since their cursor — the idea behind JMAP state strings and IMAP UIDs, which I’d offer through a gateway. Search is a small per-user index built from the same log, with the newest messages matched directly.

Sending saves to Sent and an outbox in one transaction. Local recipients go straight to the workers; external ones go to a queue per destination MX, paced by reply codes, retried for five days, bounced with DSNs, and protected by DKIM, our DMARC policy, sending limits and separate IP pools.”

Concepts used in this design

• Blob storage — bodies and attachments stored once by content hash, hot and cold.
• Database sharding — every mailbox partitioned by user ID.
• Cassandra and DynamoDB — wide-column stores for per-user metadata rows.
• Message queues — the inbound queue and per-destination outbound queues.
• Delivery guarantees — why SMTP and our pipeline are at-least-once.
• Idempotent consumers and the inbox pattern — dropping the copy when a 250 is lost.
• Idempotency — one email per tap on Send.
• Transactional outbox — saving to Sent and queueing delivery together.
• Change data capture — the change log behind sync, push and search.
• Real-time connections — WebSockets and SSE for push hints.
• How search engines work — the inverted index in each per-user index.
• Strong vs eventual consistency — read-your-writes for mailboxes, eventual for search.
• Domain Name System — MX, SPF, DKIM and DMARC records live in DNS.
• Timeouts, retries and backoff with jitter — the five-day retry schedule.
• Encryption and key management — per-blob keys and crypto-shredding.
• Design a notification system — getting hints onto phones.

References and further reading

1. RFC 5321: Simple Mail Transfer Protocolrfc-editor.orgA 250 reply to DATA transfers responsibility for the message (section 6.1); retry at least 30 minutes apart and give up after 4–5 days (section 4.5.4.1); MX preference rules.
2. RFC 5322: Internet Message Formatrfc-editor.orgMessage-ID, In-Reply-To and References (section 3.6.4): the headers that threading is built on.
3. RFC 7489: Domain-based Message Authentication, Reporting and Conformance (DMARC)rfc-editor.orgAlignment of the visible From: domain with an SPF- or DKIM-authenticated domain, and the none, quarantine and reject policies.
4. RFC 9051: Internet Message Access Protocol (IMAP) version 4rev2rfc-editor.org32-bit, strictly ascending UIDs per mailbox, UIDVALIDITY, and IDLE re-issued at least every 29 minutes.
5. RFC 8621: The JSON Meta Application Protocol (JMAP) for Mailrfc-editor.orgAn email may sit in several mailboxes at once (labels), keywords such as $seen and $flagged, and the suggested threading rule: a shared message ID plus the same normalised subject.
6. Email sender guidelines (Google)support.google.comThe 2024 rules: SPF or DKIM for everyone; SPF, DKIM and DMARC, one-click unsubscribe and a spam rate below 0.3% for senders of 5,000+ messages a day; backing off after a 4.7.28 reply.
7. Gmail API: Synchronize clientsdevelopers.google.comFull versus partial sync with a historyId cursor; history records typically kept for at least a week; an expired cursor returns 404 and forces a full sync.
8. Spam does not bring us joy: ridding Gmail of 100 million more spam messages with TensorFlow (Google, 2019)workspace.google.com1.5 billion monthly Gmail users in 2019; more than 99.9% of spam, phishing and malware blocked; thousands of signals per email and per-user personalisation.
9. Megastore: Providing Scalable, Highly Available Storage for Interactive Services (Baker et al., CIDR 2011)cidrdb.orgUses an email account as the model entity group: transactional within one account, so a user who sends or labels a message sees the change even after failover.
10. The FastMail email search system (FastMail, 2014)fastmail.comOne Xapian search database per user; new mail indexed into a small in-memory tier, then compacted into cheaper disk tiers; an account with over half a million messages.