Design Google Docs

Google Docs looks like a text box with a save button you never press. Underneath, it is one of the hardest small problems in system design: several people type into the same sequence of characters at the same moment, over networks with 30 to 300 ms of delay, and every screen must end up showing exactly the same document — without locking anyone out and without “conflicted copy” files. Think of four students finishing a college report at 1 a.m. before the deadline, a Bengaluru startup’s founders editing an investor memo together, or a newsroom live-blogging an IPL final while 20,000 readers watch the page update. In this problem you design the editor’s backend: how concurrent edits are merged, where a document “lives” while people edit it, how it is stored so it loads fast and keeps its full history, and how cursors, offline edits and huge audiences fit in.

Understand the problem

The prompt sounds like CRUD, and the trap is to design CRUD and stop. The real problem is concurrent edits to one shared sequence. Millions of documents spread easily over many machines; one document with 30 people typing in it cannot be split, and all 30 must see the same final text. Say this early: it tells the interviewer where you will spend your time.

Functional requirements

1. Users should be able to create a document and open one they have access to — quickly, even if it has three years of history.
2. Users should be able to edit the same document at the same time and see each other’s changes within about 100–200 ms, with every copy converging to the same content.
3. Users should be able to see who else is in the document, with live cursors and selections.
4. Users should be able to browse version history and restore an earlier version.

Editing while offline and merging later is a natural fifth requirement; we treat it as a deep dive, because it reuses the same merge machinery.

Non-functional requirements

• Scale. About 100M daily active users, 20M of them connected at peak, with roughly 15M documents open at once. Most open documents have a single editor; a few have dozens.
• Collaboration limits. Up to 100 simultaneous editors on one document — Google Drive documents the same cap of 100 tabs or devices actively editing a file — and many thousands of read-only viewers.
• Latency. Your own keystroke appears in under 16 ms (one frame), never waiting for the network. Someone else’s edit appears in about 100 ms at p50 within a region and under 500 ms at p99 across continents.
• Durability. Once the editor shows “Saved”, the edit survives any single machine or zone failure. Losing an acknowledged keystroke is a serious bug.
• Consistency. All copies of a document must converge to identical content once everyone has received every edit, and each user’s intent should be preserved (text typed into a sentence stays in that sentence). Within a document there is one total order of edits; across documents nothing needs ordering.
• Availability. 99.95% or better for editing. If a server dies, typing continues locally and sync resumes within seconds.
• Size. Documents up to about a million characters, and years of history.

Out of scope

• Comments and suggestion mode. They anchor to text ranges much as cursors do, but add their own workflow; see the follow-ups.
• Spreadsheets and slides. Different data models (a grid, a tree of shapes) with their own merge rules.
• Embedded images and drawings. Uploaded to blob storage and referenced from the document by ID; the merge logic treats each as one item.
• Full-text search across documents. A separate indexing pipeline fed from snapshots.
• The sharing UI and folders. We assume an access-control service that answers “what role does this user have on this document?”

The set-up

Plan the approach

Meet each requirement simply, then spend most of the time in deep dives — this problem is won there. Say the shape early: “a stateless HTTP API for opening documents and history, a stateful real-time path where each open document has exactly one owner, and a durable operation log behind it.” The deep dives follow from that sentence.

Core entities

• Document. ID, title, owner, created time, latest revision number, pointer to the latest snapshot.
• Operation (op). One small change: insert text at a position, delete a range, or apply a style to a range. Stored with the document ID, a revision number assigned by the server (1, 2, 3 … with no gaps), the author, the client that sent it and that client’s sequence number.
• Snapshot. The full document content as of revision N, serialised and compressed, kept in blob storage.
• Session. One connected tab: user, document, connection, the last revision it has seen.
• Presence. A user’s colour, name, cursor and selection. Ephemeral — never stored.
• Access-control entry. Document, principal (user, group or “anyone with the link”) and role: owner, editor, commenter or viewer.

The op log is the heart of the data model. As a table, keyed so that one document’s ops are stored together in revision order:

CREATE TABLE doc_ops (
  doc_id      BYTES     NOT NULL,  -- partition key: one document's ops live together
  rev         BIGINT    NOT NULL,  -- 1, 2, 3 ... assigned by the document's owner
  epoch       BIGINT    NOT NULL,  -- owner's fencing token (see deep dives)
  author_id   BYTES     NOT NULL,
  client_id   BYTES     NOT NULL,  -- one per open tab
  client_seq  BIGINT    NOT NULL,  -- lets the server drop a resent duplicate
  op          BYTES     NOT NULL,  -- e.g. insert "2026 " at 4, in a compact binary form
  created_at  TIMESTAMP NOT NULL,
  PRIMARY KEY (doc_id, rev)
);

API

There are two surfaces: a REST API for everything that is not live typing, and a WebSocket per open document for the live part.

Create and open.

POST /v1/docs
Content-Type: application/json

{ "title": "IPL final live blog" }

{ "doc_id": "d_7Kq2", "rev": 0 }

GET /v1/docs/d_7Kq2

{
  "doc_id": "d_7Kq2",
  "title": "IPL final live blog",
  "role": "editor",
  "rev": 18342,
  "content": "<document body at revision 18342>",
  "session_url": "wss://collab.example.com/v1/docs/d_7Kq2/session",
  "session_token": "eyJhbGciOi..."
}

The live channel. After opening, the client connects to session_url and says which revision it has, so the server can send anything newer. Then frames flow both ways. Every op the client sends carries the revision it was typed against (base_rev):

{ "type": "op", "client_id": "c_91", "client_seq": 57, "base_rev": 18342,
  "ops": [ { "insert": "Kohli ", "at": 120 } ] }

The server answers the sender with an acknowledgement carrying the revision it assigned:

{ "type": "ack", "client_seq": 57, "rev": 18344 }

Everyone else receives the op as the server stored it. Notice the position: another user’s 6-character insert earlier in the text became revision 18343, so the server shifted this insert from 120 to 126 before broadcasting it:

{ "type": "op", "rev": 18344, "author": "u_asha", "ops": [ { "insert": "Kohli ", "at": 126 } ] }

Presence travels on the same socket but is never acknowledged or stored:

{ "type": "presence", "user": "u_ravi", "rev": 18344, "cursor": 132, "selection": [132, 140] }

History and restore.

GET  /v1/docs/d_7Kq2/revisions?before=18344&limit=50   grouped edits: who, when, how much
GET  /v1/docs/d_7Kq2?rev=12000                         read-only content at an old revision
POST /v1/docs/d_7Kq2:restore   body: rev = 12000       appends a NEW revision that recreates rev 12000

Restore never rewrites history. It computes the edit that turns today’s content back into revision 12,000’s and appends it as the next revision, so collaborators see it arrive like any other change, and the restore can itself be undone.

Back-of-envelope estimates

These numbers decide whether one server per document is enough, what dominates storage, and whether cursors can share the op log. (For the method, see back-of-envelope estimation.)

Edits. Assume each daily user types or deletes about 1,500 characters a day across all documents: 100M × 1,500 = 150 billion character edits a day. The client sends one op at a time and batches whatever is typed while it waits for the acknowledgement, so an op carries about 5 characters on average: 150B ÷ 5 = 30 billion ops a day. Spread over a day: 30B ÷ 86,400 s ≈ 350k ops/s on average, and about 3× that at the weekday-morning peak, ≈ 1M ops/s.

Per document. That total is spread across about 15M open documents. Even a busy document with 100 editors, each sending about 3 ops a second while typing, produces 100 × 3 = 300 ops/s. If transforming and applying an op takes about 50 µs, that is 300 × 50 µs = 15 ms of CPU per second — under 2% of one core. One server per document is plenty.

Fan-out. If each op reaches on average two other open tabs, the servers push about 1M × 2 = 2M op messages/s at peak, plus 1M acknowledgements.

Presence. Suppose 10% of connected users, 2M people, share a document with someone else, and each cursor moves about twice a second: 2M × 2 = 4M presence updates/s, each delivered to about three others, so ≈ 12M messages/s. Presence traffic is about four times the op traffic — a strong hint that it must not touch the log.

Op log storage. An op record is about 100 bytes (IDs, revision, timestamps and a few characters of payload): 30B × 100 B = 3 TB a day, or 3 TB × 365 ≈ 1.1 PB a year before compression. Keystroke-level history is the biggest cost in the system, so keep it fine-grained for 30 days (30 × 3 TB = 90 TB) and compact older history into coarser revisions.

Snapshots. One snapshot per 1,000 ops, average document about 30 KB: 30B ÷ 1,000 = 30M snapshots a day × 30 KB = 0.9 TB a day, about 0.3 TB compressed. Old snapshots can be thinned to a few per day per document.

Session servers. An open document in memory costs about 150 KB: 30 KB of content, about 100 KB for its last 1,000 ops (kept to transform late arrivals) and per-tab bookkeeping. 15M × 150 KB = 2.25 TB of RAM. At about 24 GB of document state per server, that is 2.25 TB ÷ 24 GB ≈ 94, so about 100 session servers, doubled to 200 for headroom and failover.

Connections. 20M WebSockets at about 50k per gateway machine = 20M ÷ 50k = 400 gateway servers.

Opens. Each user opens about 5 documents a day: 100M × 5 = 500M opens ÷ 86,400 s ≈ 5,800 opens/s, about 17k/s at peak. Each open reads one snapshot and at most 1,000 ops, so it is cheap and bounded.

What the numbers decide: the global rate is large but the per-document rate is tiny, so partition by document and give each one a single owner; presence outweighs edits, so it gets its own lossy path; and history dominates storage, so snapshots and compaction are part of the design, not an afterthought.

High-level design

Build it one requirement at a time. By the end of this section it works for a single region at modest traffic.

Users should be able to create and open documents

Creating a document is a metadata write: a row in the metadata DB with the owner as the first access-control entry, and an empty snapshot at revision 0. Opening one is a read: check the caller’s role, then assemble the content at the latest revision.

1. The editor calls GET /v1/docs/d_7Kq2 on the document service, a stateless REST service behind a load balancer.
2. The service reads the document row and checks the caller’s role in the metadata DB.
3. It fetches the latest snapshot, here revision 18,000, from the snapshot store.
4. It reads the ops after the snapshot from the op log and replays them to produce revision 18,342, which it returns with a session URL and a short-lived session token.

Users should be able to edit the same document at the same time

The tempting first design is what the original Google Docs did: clients send the whole document (or a diff) on save, and the server compares versions and merges them. Google’s engineers explained in 2010 why they abandoned it. Say the text reads “Final at Wankhede”: Asha bolds “at Wankhede” while Ravi replaces “Wankhede” with “Eden Gardens”. Comparing versions, the server cannot tell whether the answer is “at Eden Gardens” in bold, “Wankhede Eden Gardens” or “Eden Gardens Wankhede” — the intent behind each change is gone. Locking a paragraph per editor avoids the merge but hampers collaboration, and two people can still start typing before either sees the other’s lock.

So send intent, not state. Every edit becomes a small op — delete 8 characters at 9, insert “Eden Gardens” at 9, bold 6 to 17 — and the document is whatever you get by replaying its ops in order. Three new pieces make that live:

• A WebSocket gateway fleet holds the long-lived connections (see real-time connections) and forwards each document’s frames to the server that owns it.
• A session server owns each open document: it keeps the content and recent ops in memory, puts every incoming op into one order, assigns revision numbers, appends to the op log and broadcasts.
• Optimistic clients. The editor applies your keystroke locally at once and sends the op in the background, so typing never waits for the network.

Ops typed at the same moment were typed against different versions, so the server must sometimes rewrite an op’s position before applying it — operational transformation, the first deep dive. Here is the happy path, with nothing concurrent:

The server appends to the log before it acknowledges or broadcasts. That costs about 5 ms per op, and buys a simple promise: nobody ever sees an edit that could later vanish.

Users should be able to see who else is in the document

Presence — avatars, coloured cursors, selections — rides the same WebSocket but follows different rules. The session server keeps an in-memory map from each tab to its user, colour, cursor and selection, broadcasts changes to the other tabs, and drops an entry when its tab disconnects. Nothing is acknowledged or written to disk: a lost cursor update is fixed by the next one a moment later.

Users should be able to browse and restore versions

Because the op log is the document’s history, version history is a read path over data you already have. A background snapshotter follows the log and writes a snapshot every 1,000 ops. The history list groups consecutive ops by author and pause — “Ravi, 10:42–10:57, 1,204 edits” — and viewing an old revision loads the nearest earlier snapshot and replays forward. Restoring appends a new revision, as described in the API, so nothing is ever deleted.

This works: edits are ordered, durable and broadcast; presence is live; history is replayable. It does not yet explain how the server merges edits typed at the same time, how a gateway finds a document’s owner, what happens when that owner dies, or how it survives a 20,000-reader audience. The deep dives do.

Deep dives

How do we merge concurrent edits so every copy converges?

Asha and Ravi both see “IPL FINAL” at revision 41. Asha inserts “2026 ” at position 4; at the same moment Ravi deletes the first four characters, “IPL ”. Each applies their own op at once, and each op travels to the other. The design must guarantee two things: convergence — once both ops have arrived everywhere, both screens show the same text — and intention preservation — each op still does what its author meant, even though the text moved under it.

Bad: locks or last-write-wins. Paragraph locks stop people typing where they want, and two editors can both start before either sees the other’s lock. Last-write-wins on a paragraph silently throws away someone’s sentence. Figma uses last-writer-wins per object property, ordered by its server, and it works well for design objects — but Figma’s own write-up notes the consequence for text: if a text value is “B” and one person changes it to “AB” while another changes it to “BC”, the result is “AB” or “BC”, never “ABC”. Fine for a design tool, fatal for a word processor.

Good: operational transformation (OT) with a central server. Ops refer to positions, and a concurrent op may have moved those positions. So before applying an op that was typed against an older version, transform it: shift it over whatever happened in between. Run the example both ways.

The rule that makes this work was set out in the Jupiter paper by Nichols, Curtis, Dixon and Lamping at Xerox PARC in 1995: a function xform(c, s) = (c′, s′) such that applying c then s′ gives the same result as applying s then c′. Every pair of op types needs a rule. For plain text, they fit in a table:

The last row is from Google’s own description: bolding characters 10–20 transformed against an insert of “ABC” at 15 becomes bolding 10–23. Here are the text rules as code — short enough to read, which is the point: the core of OT fits on a screen.

from dataclasses import dataclass, replace

@dataclass(frozen=True)
class Op:
    kind: str        # "ins" or "del"
    pos: int
    text: str = ""   # inserts: the text
    n: int = 0       # deletes: how many characters

def transform(op: Op, prior: Op) -> list[Op]:
    """Rewrite op so it applies after `prior`, which the server ordered first."""
    if prior.kind == "ins":
        k = len(prior.text)
        if op.kind == "ins":           # tie at the same spot: prior stays on the left
            return [replace(op, pos=op.pos + k)] if prior.pos <= op.pos else [op]
        if prior.pos <= op.pos:        # text inserted before our range
            return [replace(op, pos=op.pos + k)]
        if prior.pos >= op.pos + op.n: # inserted after our range
            return [op]
        left = prior.pos - op.pos      # inserted inside: delete around it
        return [replace(op, pos=prior.pos + k, n=op.n - left), replace(op, n=left)]
    d, m = prior.pos, prior.n          # prior deleted [d, d + m)
    if op.kind == "ins":
        if op.pos <= d:
            return [op]
        return [replace(op, pos=max(d, op.pos - m))]
    overlap = max(0, min(op.pos + op.n, d + m) - max(op.pos, d))
    start = op.pos if op.pos <= d else max(d, op.pos - m)
    return [replace(op, pos=start, n=op.n - overlap)] if op.n > overlap else []

A real editor has dozens of op types — paragraphs, list items, tables, images, styles, suggestions — and every pair needs a rule that keeps the xform property. That is where OT gets hard: proofs for even the two-primitive case are notoriously error-prone, which is one reason Figma chose not to use OT.

The protocol is what makes OT tractable. In Jupiter, clients never talk to each other. The server puts every op into one order and each client runs a two-party protocol with the server, so an n-person problem becomes n two-person problems. Google Wave and Google Docs added one more rule: a client has at most one op in flight. While it waits for the acknowledgement, it buffers new keystrokes and composes them into a single op. Because every op the client sends is based on a revision the server really had, the server keeps one history and nothing per client: an op with base_rev 41 arriving when the head is 44 is transformed against revisions 42, 43 and 44, in order, and becomes revision 45.

The client mirrors this. It tracks the last revision it has seen, the op in flight, the buffer and its local document. A remote op is transformed against the in-flight and buffered ops (and they against it) before it is applied, and each acknowledgement releases the buffer as the next op. Wave’s whitepaper names the cost: other people’s edits arrive in chunks, about one round trip apart. The gain is a simple server with one history per document and work that does not grow with the number of editors.

Great for offline-first: CRDTs. A sequence CRDT (conflict-free replicated data type) avoids transforming positions by never using positions. Every inserted character gets a globally unique ID — in Yjs, a (client ID, counter) pair — and an insert says “put this after the item with ID x”, not “put this at 4”. RGA, the Replicated Growable Array of Roh and others, orders concurrent inserts after the same item by comparing their IDs, so every replica picks the same order. A delete only marks the item as a tombstone, because someone else’s concurrent insert may refer to it.

You gain merging without a central order: ops can arrive in any order (as long as an insert arrives after the item it refers to), which makes offline editing and peer-to-peer sync natural. Libraries such as Yjs and Automerge make it practical; many newer editors run a CRDT in the browser with a server that just relays and stores updates. You pay in metadata: in Yjs’s worst case, one item per character, each item costs about 88 bytes of memory. Real typing is kinder: on an editing trace of 182,315 inserts and 77,463 deletes, Yjs needed only 10,971 items and encoded the 104,852-character result in about 160 KB — roughly 53% more than the text itself. Tombstones grow forever unless you can prove every replica has seen the delete, which is easy with a server and hard without one.

Recommendation. For Google Docs, use OT with a central server, which is what Docs itself does. You need servers anyway for access control, storage and search, and in exchange you get compact ops, no per-character metadata and a history that is simply revisions 1, 2, 3 … Choose a CRDT library instead if offline-first or peer-to-peer editing is a core requirement, or if you would rather adopt a proven library than write transform rules for every new op type.

How does every editor of a document reach the same server?

The OT protocol only works if exactly one server puts a document’s ops in order. With 200 session servers, deploys every day and machines that sometimes pause, “one owner” has to be enforced, not assumed.

Bad: stateless servers, with the database as the sequencer. Any server accepts an op, reads the latest revision, transforms, and inserts revision + 1 only if nobody else did, retrying on a clash. It is correct, but every op becomes a 5–10 ms read-transform-conditional-write round trip, servers fight over one row on a busy document, and each must keep its own warm copy and relay ops to clients connected elsewhere. You have rebuilt a single owner, badly, inside the database.

Good: consistent hashing on the document ID. Each gateway hashes doc_id onto a ring of session servers, so finding the owner needs no lookup, and adding a server moves only about 1/N of the documents (see consistent hashing). The flaw is membership. During a deploy or a network blip, two gateways can briefly disagree about the ring and send editors of one document to two servers — and both will happily assign revision 18,343. That is a forked history, and no transform can repair it.

Great: an explicit owner per document, with a lease and a fencing token. Keep a small assignment table in a coordinator such as etcd or ZooKeeper: d_7Kq2 → S1, epoch 7, lease 10 s. When the first editor opens a document, a session server acquires the lease (placement can still use the hash ring as a hint, or pick the least-loaded server), renews it every 3 seconds, and releases it when the document has been idle for a few minutes. Gateways cache the mapping and re-check it on any error. The detail that makes it safe is the epoch, a fencing token: every append to the op log carries the owner’s epoch, and the log refuses an append whose epoch is older than the newest it has seen. Figma’s 2024 write-up describes the same pattern: an ownership row per file in DynamoDB, and journal writes that succeed only if the lock’s ID still matches. (More on fencing in distributed locking.)

What happens to ops in flight? Nothing is lost, because of two earlier choices:

• Acknowledged ops are already durable, since the owner acknowledges only after the log append. The new owner rebuilds the document from the latest snapshot plus the log tail, exactly like a cold open.
• Unacknowledged ops are still on the clients, in the in-flight slot and the buffer. On reconnect each client sends hello with the last revision it saw, receives what it missed, rebases, and resends its in-flight op with the same client_seq. If that op had actually been appended before the crash — the append succeeded but the acknowledgement was lost — the new owner finds (client_id, client_seq) in the log tail and simply acknowledges it again instead of applying it twice. That is idempotency applied to keystrokes.

The cost is a pause. After a crash, sync stops for roughly the lease length, about 10 seconds; typing never stops, because clients apply edits locally. A shorter lease fails over faster but triggers false failovers during ordinary GC pauses and network hiccups. Planned deploys avoid it: the old owner stops accepting ops, flushes, releases the lease, and the new owner takes it immediately — Figma reports that flushing unsaved changes before closing a file takes under a second at p99.

How do we store documents so nothing is lost and old ones open fast?

Bad: overwrite the document on every change. Rewriting a 30 KB document per op is 1M × 30 KB = 30 GB/s of writes at peak, and the history is gone. Keeping documents only in memory and saving them periodically is the opposite trade: Figma’s multiplayer service used to write a checkpoint of each open file to object storage every 30–60 seconds, so a server crash could lose up to a minute of work — the reason it added a journal.

Good: an append-only op log. Every op is a 100-byte append, and the document is whatever replaying the log produces — the idea behind a write-ahead log and event sourcing. It is durable and keeps full history for free. But opening means replaying everything: a three-year-old document with 2M ops is 2M × 100 B = 200 MB to read and apply, on every open.

Great: the op log plus periodic snapshots. A snapshot every 1,000 ops, or whenever the document goes quiet, caps every open at one blob read plus at most 1,000 ops, about 100 KB — tens of milliseconds.

The details that matter:

• The log store. Partition by doc_id, sort by rev, make appends conditional on the epoch, and replicate each append synchronously to three zones so an acknowledged op survives losing a zone. A wide-column store such as Bigtable, Cassandra or DynamoDB fits: the global write rate is high (1M ops/s) but each partition is calm (at most a few hundred ops/s), so there are no hot partitions.
• Group commit. The owner batches ops for one document that arrive within about 5 ms into a single append, so 100 typists cost one write, not 100.
• Ack after append, or before? Figma writes its journal asynchronously: it persists 95% of changes within about 600 ms, handles more than 2.2 billion changes a day, and aims to lose under a second of edits on a crash. Acknowledging after a synchronous append costs about 5 ms per op and loses nothing acknowledged. For a word processor that shows “Saved”, pay the 5 ms.
• Snapshots are off the hot path. A snapshotter follows the log and writes (doc_id, rev) → blob to object storage, then updates the document’s pointer. If it falls behind, opens replay a few thousand ops instead of a few hundred — slower, never wrong.
• History and compaction. Keep keystroke-level ops for 30 days (90 TB), then collapse each run of ops into one revision per author per editing session and thin old snapshots to a few per day. Named versions pin their snapshot forever.

How do we show live cursors without flooding the system?

The estimates showed presence at about four times the op traffic. It is also worthless a second later: nobody needs to know where Ravi’s cursor was at 10:41:07.

Bad: treat cursor moves as ops. That means 4M extra durable writes a second, a history full of cursor noise and — worse — cursor moves occupying the single in-flight slot, delaying real edits.

Good: a separate, lossy channel through the session server. Presence is held only in the owner’s memory, never acknowledged or stored, and expires when a tab disconnects or misses heartbeats for about 30 seconds. A cursor is “position p at revision r”, so when an op arrives the cursor is transformed exactly like an insert at p. With a CRDT you anchor the cursor to a character’s ID instead, and it moves with that character automatically.

Great: throttle at the sender, coalesce at the server. Clients send at most one presence update every 100 ms, latest value wins. The server does not forward each update as it arrives; every 100 ms it sends each tab one frame listing the cursors that changed. Under load, presence is the first thing to degrade — down to one frame a second — because an edit must never wait behind a cursor.

How do we handle reconnects and offline editing?

Connections drop all the time — a Mumbai local entering a tunnel, a laptop lid closing, a flight. The client keeps its in-flight op and buffer in IndexedDB with their base revision, so even closing the tab loses nothing. Reconnecting is then the same conversation as any late op, just with a longer gap:

Rebasing m local ops over n remote ops costs about m × n transforms. For 40 over 60 that is 2,400 — trivial. The trouble is long absences. Three days offline with 5,000 local ops against 50,000 remote ones is 5,000 × 50,000 = 250M transforms, minutes of CPU at about a microsecond each, and the result may be nonsense if others rewrote the paragraphs Ravi was editing. So set a product limit: if the gap is beyond about 10,000 remote ops, or older than the 30 days of fine-grained log the server keeps, do not auto-merge. Save the offline work as a copy and show a side-by-side comparison.

Figma reconnects by downloading a fresh copy and reapplying offline edits on top, which works because its edits are last-writer-wins per property; for text, that reapplying is the rebase. CRDTs shine here: offline edits are just ops with IDs that merge in any order with no transform. If long offline sessions are central to the product, that is the strongest argument for a CRDT.

How do we serve thousands of viewers and check access on every op?

Google Drive’s help pages set the shape of the answer: a file can be shared with up to 600 individual addresses, and at most 100 tabs or devices can actively edit it at once — beyond that, only the owner and some editors can make changes. So: cap editors per document, and give readers a separate, read-only path.

Take a college results notice or an IPL live blog: 5 editors producing about 20 ops a second, and 20,000 readers.

Bad: everyone connects to the owner. 20 ops/s × 20,000 viewers = 400,000 messages a second from one server for one document, 20,000 sockets pinned to one machine, and all of them reconnecting at once when it fails over.

Good: publish snapshots to a cache. Every few seconds, render the content to a static page behind a CDN and let viewers poll. It scales without limit and shields the owner, but readers see updates seconds late — right for a “publish to the web” page.

Great: a fan-out tree for live viewers. The owner publishes each op once to a per-document pub/sub topic. Relay servers subscribe and push to the viewers connected to them, batching into one frame every 200 ms. Viewers never send ops, so relays never transform anything; a viewer who starts editing is moved to the editor path if the document is under its cap.

Access checks on every op, without a database call per keystroke. At 1M ops a second, asking an ACL service per op is out of the question. Instead:

• At connect, the gateway verifies a short-lived signed session token (user, document, role, about 15 minutes to expiry) issued by the document service after a real ACL check.
• Per op, the owner checks the role it holds in memory for that connection: viewers and commenters cannot send text ops — a hash lookup, not a network call.
• On a permission change, the ACL service publishes an event and the owner applies it in the document’s order: “from revision 18,400, Ravi is a viewer”. Any op from Ravi that arrives after that point is rejected, even if he typed it a moment earlier, and his client reloads read-only. Short token lifetimes bound the damage if an event is ever missed.

The final design

The live path is gateway → owner → op log; everything else either prepares it (open, ownership) or follows it (snapshots, viewers).

Follow Asha’s morning from opening the live blog to her edit reaching 20,000 readers:

1. Open. The doc service checks her role in the metadata DB and returns the content at the latest revision — nearest snapshot plus a short replay — with a session token.
2. Connect. Her browser opens a WebSocket to a nearby gateway, presenting the token.
3. Find the owner. The gateway asks the coordinator (usually from its cache) which session server holds the document’s lease. If nobody does, one acquires it with a new epoch and loads the snapshot and log tail.
4. Edit. Her ops reach the owner one at a time. It checks her role in memory, transforms each op against anything newer than its base revision, and assigns the next revision. Presence updates are merged into 100 ms frames and never stored.
5. Persist, then share. The owner appends the op to the log, fenced by its epoch and replicated across three zones; only then does it acknowledge Asha and broadcast to the other editors.
6. Fan out. It publishes each op once to the document’s topic; relays push batched updates to viewers.
7. Snapshot. The snapshotter follows the log and writes a snapshot every 1,000 ops, which keeps opens fast and serves version history and restores.

Check it against the requirements. Your keystrokes are instant because clients are optimistic; others’ edits arrive in about 55 ms in one region (25 + 5 + 25), a few hundred milliseconds across continents. Convergence comes from one order per document plus transformation. Acknowledged edits survive a zone loss because acknowledgement waits for a three-zone append. An owner failure pauses sync for about 10 seconds and loses nothing; a planned deploy pauses it for under a second. The 1M ops/s peak spreads over about 15M documents, 100 session servers and 400 gateways; presence and viewers never touch the log.

What interviewers look for by level

Follow-up questions

What if we need comments and suggestion mode?

Both attach to a range of text that keeps moving. Store a comment in its own table with an anchor — start and end positions at a revision — and let the session server transform the anchor with every op, exactly like a cursor; if the anchored text is deleted, the comment shows as “on deleted text” rather than vanishing. Suggestion mode is harder: a suggestion is an op that is recorded but not yet applied to the accepted text. Model it as ordinary ops carrying a “suggested by Ravi” attribute, so they merge and transform like everything else, and make accepting or rejecting a suggestion just another op. The cost is more op types, and therefore more transform rules.

What if the region hosting a document’s owner goes down?

Inside a region, the three-zone log means any server can take over. For a whole-region failure, replicate the op log to a second region asynchronously and let the coordinator there grant a lease with a higher epoch. Ops acknowledged in the last second or so may be missing there. To recover them, clients keep acknowledged ops until the server reports them replicated to the second region, then resend anything above the new owner’s head; duplicate suppression handles the rest. The expensive alternative is synchronous cross-region replication, which adds 30–100 ms to every keystroke’s acknowledgement. Most products keep the owner near its editors and accept a small recovery-point window, an availability-over-consistency choice in the sense of CAP and PACELC.

What if customers want end-to-end encryption?

Then the server cannot read ops, so it cannot transform them. The usual answer is a CRDT: clients encrypt each update, the server only orders, stores and relays ciphertext, and clients merge on their side. Snapshots must also be made by clients, since the server cannot build them. You lose everything that needs the server to read the document — search, previews, server-side exports, malware scanning, smart features — and key management for shared documents becomes its own design problem. Offer it as a mode for sensitive workspaces, not the default.

What if someone pastes a 300-page document, or a script floods ops?

A paste is one large insert, so the problem is size, not rate: cap a single op at a few hundred KB, upload bigger pastes as a blob and insert a reference, and snapshot immediately afterwards so later opens do not replay a huge op. For floods, rate-limit ops per connection at the gateway — no human types 500 ops a second — and cap a document’s total op rate so one runaway client cannot starve its own collaborators. The rate limiter design applies directly, with the connection as the key.

How to say it in the interview

“I’ll design a collaborative editor: about 100 million daily users, up to 100 editors on one document, your own keystrokes instant, other people’s visible in about 100 milliseconds, and no acknowledged edit ever lost.

The key decision is the data model: a document is an ordered log of small operations — insert, delete, apply style — not a file. I merge concurrent edits with operational transformation and a central server, the Jupiter model Google Docs uses. Clients apply edits locally and keep one op in flight, tagged with the last revision they saw. The server transforms each late op against everything after that revision, assigns the next revision, appends it to the log and broadcasts it. One order per document means every copy converges. If offline-first or peer-to-peer were central, I’d use a CRDT like Yjs instead and pay in per-character metadata and tombstones.

That needs exactly one owner per open document. A session server holds a lease with an epoch in a coordinator, gateways route to it, and the op log rejects appends from older epochs, so a paused server can never fork the history. After a failover, clients resend unacknowledged ops and the server drops duplicates by client sequence number.

Numbers: about a million ops a second at peak, but only a few hundred on the busiest document, so about a hundred session servers holding documents in memory, and around 3 terabytes of ops a day. A snapshot every thousand ops makes any open one blob plus at most a thousand ops; old history is compacted.

Presence is about four times the op traffic, so it never touches the log: it lives in memory and is coalesced into one frame per tab every 100 milliseconds. Offline clients rebase on reconnect, within a bound. Big audiences go through a read-only relay tier, and access is checked in memory on every op.”

Concepts used in this design

• Clocks, ordering and conflict resolution — why concurrent edits need one order or commutative merges.
• Real-time connections: long polling, SSE and WebSockets — the live channel between editors and the owner.
• Real-time updates — the general pattern behind pushing ops and presence.
• Stateful vs stateless design — why the session tier is deliberately stateful and the REST tier is not.
• Consistent hashing — placement hints for documents, and why it is not enough on its own.
• Distributed locking — leases and fencing tokens for single ownership.
• ZooKeeper and etcd — where the ownership leases live.
• Write-ahead logging — the op log is a document’s WAL.
• Event sourcing and CQRS — state as a replay of events, with snapshots.
• Blob storage — where snapshots live.
• Idempotency — resending unacknowledged ops without applying them twice.
• Publish/subscribe — fanning ops out to relays and viewers.
• Heartbeats — expiring presence and detecting dead owners.
• Strong vs eventual consistency — one order per document, converging replicas on every screen.
• Design Dropbox / Google Drive — the file-sync sibling, where whole files, not keystrokes, are the unit.

References and further reading

1. What’s different about the new Google Docs: Working together, even apart (Google Drive Blog, 2010)drive.googleblog.comWhy the old diff-and-merge approach and paragraph locking fail, and the switch to storing a document as a chronological list of changes.
2. What’s different about the new Google Docs: Conflict resolution (Google Drive Blog, 2010)drive.googleblog.comThe three change types (insert, delete, apply style), the revision log replayed to display a document, and example transformations.
3. What’s different about the new Google Docs: Making collaboration fast (Google Drive Blog, 2010)drive.googleblog.comWhat each client and the server track, one unacknowledged change in flight at a time, and the server transforming late changes before logging them.
4. High-Latency, Low-Bandwidth Windowing in the Jupiter Collaboration System (Nichols, Curtis, Dixon, Lamping, UIST 1995)lively-kernel.orgThe central-server OT model: the server serialises all changes, each client runs a two-party protocol with it, and xform(c, s) fixes up crossing messages.
5. Google Wave Operational Transformation (Wang, Mah, Lassen, 2010)svn.apache.orgClients wait for an acknowledgement before sending more, so the server keeps a single history; pending client operations are composed into one.
6. How Figma’s multiplayer technology works (Figma, 2019)figma.comOne server process per document, server-ordered last-writer-wins per object property, why that cannot merge concurrent text edits, and the offline reconnect strategy.
7. Making multiplayer more reliable (Figma, 2024)figma.comCheckpoints every 30–60 seconds plus a sequence-numbered journal, an ownership lock with conditional writes, and 95% of changes persisted within about 600 ms.
8. Yjs internals (Yjs repository)github.comUnique (clientID, clock) IDs per inserted item, origin and originRight for concurrent inserts, deletions as flags, and state-vector sync.
9. Are CRDTs suitable for shared editing? (Kevin Jahns, 2020)blog.kevinjahns.deMeasured CRDT overhead on a real editing trace of 182,315 inserts and 77,463 deletes, and the 88-bytes-per-item worst case.
10. Share and manage access to files in Google Drive (Google Drive Help)support.google.comSharing limits: up to 600 individual addresses per file, and up to 100 tabs or devices actively editing a file at once.