Design live comments

Live comments are the stream of messages and reactions that scroll beside a live video — a cricket final, a product launch keynote, a stand-up special. Anyone watching can post, and everyone watching should see new comments within a couple of seconds. The design is interesting because the numbers are lopsided. Writing a comment is a tiny HTTP request; reading it means pushing it to every viewer, over connections that must stay open for hours. A stream with 10 viewers and a stream with 10 million share one code path, and at the top end the naive maths breaks: according to LinkedIn’s Akhilesh Gupta, the 2019 World Cup semi-final between India and New Zealand drew more than 25 million concurrent viewers to a single stream. You will design how connections are held, how a comment reaches only the servers that need it, what to do when one stream is too big to deliver everything, and how late joiners catch up.

Understand the problem

Separate two things early: the video (big, cacheable, delayed by several seconds of segments and buffering) and the interaction channel (tiny messages that must reach everyone quickly). This design is only the second. The video pipeline is in design YouTube.

Functional requirements

1. Viewers should be able to post a text comment on a live video they are watching.
2. Viewers should be able to send reactions — a clap, a heart, a laugh — as often as they like.
3. Viewers should see other viewers’ comments, reaction counts and the viewer count update live, within about two seconds.
4. Viewers who join late or reconnect should see the recent comments first, then continue live without gaps or duplicates.

Non-functional requirements

• Freshness. A comment reaches other viewers within 2 seconds at p95, about half a second at p50 on small streams. Your own comment appears instantly. Reactions may lag up to 3 seconds.
• Scale. Up to 10 million concurrent viewers on one stream, and 50 million across the platform during a big match, plus a long tail of about 100,000 live streams with a handful of viewers each.
• Availability. 99.95% for the live channel. Under stress, degrade by showing fewer comments, never by breaking the video.
• Durability. An acknowledged comment is stored — for replay, for the history late joiners see, and for moderation audits. Individual reactions are not; per-second counts are.
• Delivery and order. Best effort: a viewer on a bad network may miss a comment, and that is acceptable. Within a stream, comments appear roughly in posting order with no duplicates.
• Safety. Comments that are obviously abusive must never reach the audience; moderation sits before fan-out.
• Efficiency. Phones on mobile data and batteries: few, small, bundled frames.

Out of scope

• Video ingest, transcoding and delivery. A separate pipeline; we only assume a stream ID.
• Threads, replies and likes on individual comments. Live comments are a flat, fast-moving feed.
• Private chat and group messaging. See design WhatsApp and design Slack and Discord.
• The internals of the spam and toxicity models. We design where they plug in and their latency budget.
• Paid comments and payments. One follow-up covers how they change delivery.

The set-up

Plan the approach

Build the four requirements with the simplest design that works for one modest stream: HTTP to post, a persistent connection to receive, a pub/sub layer in between. Then spend the deep dives on what changes as viewers grow by six orders of magnitude: the transport, routing events to the right servers, the one stream that is too hot to deliver in full, ordering and resume, the write-path budget, viewer counts, and many regions with slow clients.

Core entities

• Live video (stream): stream ID, owner, state (live or ended), start time, home region, delivery mode (every comment, or selected).
• Comment: stream ID, sequence number within the stream, comment ID, author, text, language, created at, moderation state.
• Reaction tally: stream ID, second, count per reaction type. Individual taps are never stored.
• Connection: connection ID, gateway, user, stream watched, last sequence delivered.
• Subscriptions, at two levels: on each gateway, stream → its local connections (in memory); in a shared store, stream → the gateways that have at least one viewer of it.

API

Post a comment. Plain HTTP. client_msg_id makes a retried post idempotent; the response carries the sequence number so the client can place its own comment.

POST /v1/live/ipl-final/comments
Authorization: Bearer <token>

{ "client_msg_id": "a7c9e2", "text": "What a catch by the boundary rope!" }

{ "comment_id": "c_91f3", "seq": 18233, "state": "published" }

Send reactions. The client counts taps locally and flushes at most once every 2 seconds.

POST /v1/live/ipl-final/reactions

{ "counts": { "clap": 7, "heart": 2 } }

Receive live events. One server-sent events stream per viewer. Last-Event-ID is sent automatically by the browser on reconnect; the gateway replays anything newer from its buffer.

GET /v1/live/ipl-final/events
Accept: text/event-stream
Last-Event-ID: 18232

id: 18233
event: comments
data: {"items":[{"seq":18233,"id":"c_91f3","user":"Aarav","text":"What a catch by the boundary rope!"}]}

event: tally
data: {"reactions":{"clap":41200,"heart":22100},"viewers":9870000}

: keep-alive

Recent history. For first paint, and for a client whose gap is too old to replay. Identical for every viewer, so it is cacheable for a second.

GET /v1/live/ipl-final/comments?limit=50

Back-of-envelope estimates

Size the hot stream first; it breaks things the long tail never will. (For the method, see back-of-envelope estimation.)

Connections. LinkedIn reports holding about 100,000 persistent connections per frontend machine in production. The hot stream: 10M ÷ 100k = 100 gateways. The platform at peak: 50M ÷ 100k = 500 gateways, about 750 with 50% headroom for failures and reconnect storms.

Comments on the hot stream. Assume that at a wicket 0.2% of 10M viewers post within the same second: 20,000 comments/s at peak, a few thousand per second on average.

Naive fan-out. Every comment to every viewer: 20,000 × 10M = 200 billion deliveries/s. At about 150 bytes each that is 30 TB/s. Even if the network could carry it, no human can read 20,000 comments a second; five to ten is already a fast scroll.

Selected fan-out. Show about 10 comments a second, bundled with the tallies into one frame per viewer per second: 10M frames/s. At about 1.5 KB per frame (10 × 150 B, plus tallies), that is 15 GB/s ≈ 120 Gbps of egress for the stream, or 150 MB/s ≈ 1.2 Gbps per gateway before compression. Bandwidth, not CPU, sets how many hot-stream viewers one gateway can carry.

Reactions. Assume 1M viewers react during a big moment and each client flushes once every 2 seconds: 1M ÷ 2 = 500,000 requests/s — 25 times the comment rate. They must be counted, never stored or delivered one by one.

Dispatch. Platform-wide, 500M comments a day ÷ 86,400 ≈ 5,800 comments/s, plus one tally event per second for roughly 20,000 streams with active reactions: about 26,000 events/s. At LinkedIn’s measured ~5,000 events/s per dispatcher node, that is 6 dispatchers; run 12 for headroom.

Heartbeats. 50M connections, one keep-alive every 30 s: 50M ÷ 30 ≈ 1.7M tiny frames/s fleet-wide, about 3,300/s per gateway — cheap, but not free.

Storage. 500M comments a day × ~300 B (text, metadata, index) = 150 GB/day. History for late joiners — the last 500 comments of each of 100,000 live streams — is 100,000 × 500 × 300 B = 15 GB of cache.

High-level design

Build it one requirement at a time. By the end of this section, it works for streams of a few thousand viewers in one region.

Viewers should be able to post a comment

Posting is ordinary request–response. The only rule that matters: store and check a comment before fanning it out, so nothing reaches viewers that cannot be replayed or taken back.

1. The API gateway authenticates the viewer and applies a per-user rate limit — say one comment every 2 seconds on big streams — so one script cannot flood a stream.
2. The comment service asks a per-stream sequencer for the next number (INCR seq:ipl-final in Redis) and writes the comment to a store partitioned by stream and time bucket, clustered by sequence, so “the last 50 comments” is one partition read.
3. A fast moderation step — a blocklist plus a small spam and toxicity model, with a hard 150 ms timeout — decides to publish, hold for review, or show the comment only to its author.
4. Passing comments go to the dispatchers. The viewer got a 202 as soon as the comment was stored, and their app showed it immediately.

Viewers should be able to send reactions

Reactions are 25 times more frequent than comments and individually meaningless — nobody needs to know who clapped at 20:41:07. So they take a different path: count, don’t store.

The phone counts taps and animates them locally at once, then flushes the count every 2 seconds. The reaction API adds counts to in-memory counters, sharded so no single counter takes all 500,000 requests a second. Once a second an aggregator sums the shards per stream, publishes a tally event, and appends the per-second totals to storage, which later draws the “reactions over time” graph on the replay. The receiving phone turns “+1,840 claps this second” into a burst of floating icons; it does not need 1,840 messages to do that.

Viewers should see new comments and reactions live

This is the core of the design: getting one event from a dispatcher to every connection watching that stream, and to no one else. LinkedIn’s real-time platform, described in Akhilesh Gupta’s QCon talk, does it with subscriptions at two levels.

1. Connect. The viewer’s app opens one long-lived connection — server-sent events here, discussed below — to any gateway behind the load balancer, and says which stream it is watching.
2. Subscribe locally. The gateway records ipl-final → this connection in an in-memory table. In memory is safe, as Gupta points out: the table only describes connections to this machine, and if the machine dies those connections die with it.
3. Subscribe globally. If this is the gateway’s first viewer of the stream, it registers ipl-final → gw1 in a shared key–value subscription store (with a TTL it keeps refreshing), and removes it when its last viewer leaves.
4. Publish. A dispatcher receiving a comment looks up which gateways watch the stream and sends it to each — 3 sends, however many viewers those gateways hold.
5. Deliver. Each gateway looks up its local table and writes the event to every matching connection.

This is publish–subscribe with the subscriber being a server, not a person. Any dispatcher can serve any stream, because the subscription data lives in the shared store; dispatchers hold no connections and can be added freely.

Viewers who join late should catch up, then continue live

A viewer who opens the stream mid-match should see the last few dozen comments, then live ones, with nothing missing in between and nothing twice. The same problem appears every time a phone switches from Wi-Fi to 4G and reconnects.

• Join. Every gateway already receives every event for the streams it serves, so it keeps the last 500 per stream in a ring buffer. A new connection gets a snapshot of the last 50, then live events, down one connection, in order. Only a gateway’s first viewer of a stream makes it warm the buffer from the history cache; at match start, when five million people join in a few minutes, almost none of those joins touch storage.
• Resume. Each event carries its sequence number in the SSE id field, so the browser sends Last-Event-ID when it reconnects — the HTML standard builds this in. The gateway replays anything newer from its buffer, skipping nothing and repeating nothing.
• Too far behind. If the gap is older than the buffer, or the client was offline for minutes, the gateway sends a reset event and the client fetches the latest 50 from the history API, cached at the CDN for a second — then continues live.

When a separate history call is needed, subscribe first and fetch second, then drop buffered live events at or below the snapshot’s highest sequence. Reddit’s r/place used exactly this order so that no update fell between the snapshot and the live feed.

You now have a working system for streams of a few thousand viewers. The deep dives are about the other six orders of magnitude.

Deep dives

Which transport should carry live comments to viewers?

Four options, all covered in depth in long polling, SSE and WebSockets; here is how they score for this workload.

Bad: short polling. Ask every 2 seconds, “anything new since 18233?”. Simple and cache-friendly, but 10M viewers ÷ 2 s = 5M requests a second, nearly all of them returning nothing on a quiet stream and each repeating headers, cookies, auth and a load-balancer hop.

Okay: long polling. The server holds each request open until there is news, then the client immediately asks again. Quiet streams become cheap, but a busy one returns once per batch, so at one batch a second you are back to 10M requests a second — plus a gap between response and next request where events must be buffered.

Good: server-sent events. One ordinary HTTP response that never ends; the server writes id:/event:/data: chunks into it. It passes through proxies and corporate firewalls as normal HTTP, resumes for free with Last-Event-ID, and the server can set the client’s reconnect delay with a retry: field. It is one-way, which suits us: comments and reactions go up as normal HTTP posts, multiplexed on the same HTTP/2 connection. LinkedIn chose SSE for instant messaging precisely for that compatibility. The catch on HTTP/1.1 is the browser’s per-host connection limit when a user opens several tabs.

Good too: WebSockets. An HTTP Upgrade to a full-duplex, framed connection (RFC 6455), with ping/pong frames for keep-alives. Better if the client sends a lot — typing indicators, games, reactions over the same socket — and Twitch’s chat edge speaks WebSockets (and IRC over raw TCP). The costs: you build resume yourself (send the last sequence in a subscribe message), and some networks and proxies still mishandle upgrades.

Pick SSE for the downstream channel, with WebSockets as an equally good answer if the interviewer pushes on two-way traffic. Three details make either work in production:

• Heartbeats. Proxies and load balancers silently drop idle connections. Send something every 15–30 seconds — the HTML standard suggests an SSE comment line (: keep-alive) about every 15 seconds; WebSockets have ping frames. Two missed heartbeats on the server side means the connection is dead, so its slot can be freed.
• Mobile radios. Android’s documentation describes a typical radio staying at high power for about 18 seconds after every transfer. While the video plays, the radio is busy anyway, so one bundled frame a second costs almost nothing extra; ten unbundled comment messages a second would. When the app goes to the background, close the connection rather than keep a heartbeat ticking.
• Connection limits. LinkedIn’s write-up on reaching 100,000 connections per machine lists the walls you hit in order: the kernel’s accept backlog (128 by default), per-process file descriptors, memory (about 20 KB per TLS connection on their JVM, so they raised the heap to 16 GB), and ephemeral ports on the load balancer. Budget for all four before you promise 100,000.

How do we route each comment only to the servers that need it?

There are 750 gateways and 100,000 live streams, most watched by a handful of people. The routing layer’s job is to make a comment on a 12-viewer stream cost a few messages, not 750.

Bad: broadcast every event to every gateway. Simple and stateless: each gateway drops what it has no viewers for. But a small stream’s comment wakes all 750 machines, and total traffic becomes (events per second × number of gateways), which grows with every gateway you add.

Bad, differently: a log topic per stream that gateways consume. Durable and replayable, which is tempting. But a gateway cannot know in advance which streams its next viewer will watch, so in practice each gateway consumes every live topic — Gupta’s answer to “why not Kafka?” is exactly this: adding frontend machines stops helping because each must still read everything. Keep the log for storage of comments, not for last-mile routing.

Great: subscriptions at two levels. As built in step 3: gateways register the streams they serve in a shared store; dispatchers send only to those gateways; gateways fan out in memory. The dispatcher’s work for one event is bounded by the number of gateways watching, and the big multiplication — up to 100,000 connections per gateway — happens in memory. LinkedIn measured about 5,000 published events a second per dispatcher node and 75 ms at p90 from dispatcher to bytes on the connection.

Make it robust:

• Subscriptions are soft state. Each gateway refreshes its entries every 30 seconds with a TTL of 90, so a crashed gateway’s entries expire on their own and no one has to clean up after it.
• Shard the store and the dispatchers by stream ID with consistent hashing, so one stream’s lookups and publishes stay on one shard and can be cached in the dispatcher’s memory for a second.
• Accept at-most-once delivery. If a dispatcher dies mid-publish, some viewers miss a comment. That is the same guarantee Redis Pub/Sub documents — a message is delivered once if at all — and Redis 7’s sharded Pub/Sub, which hashes channels to cluster slots like keys, is a reasonable off-the-shelf version of this tier. Gaps are healed by sequence numbers on the client, not by retries in the core.

What do we do when one stream has 10 million viewers?

The routing above makes delivery possible; the hot stream makes it pointless to deliver everything. Remember the arithmetic: 20,000 comments a second × 10M viewers = 200 billion deliveries a second, for a feed no one can read.

Bad: deliver everything, and let the phone drop what it cannot show. Every byte still crosses the network; 30 TB/s is not an engineering problem, it is a budget fantasy.

Good: rate-limit the stream. Cap the stream at, say, 10 comments a second and drop the rest at random. Delivery becomes affordable — but random selection shows spam as often as wit, and drowns the moment everyone types “OUT!!!” at once.

Great: select, summarise and batch. Switch the stream into selected mode once it passes a threshold (say, 20 comments a second); small streams keep receiving every comment individually.

• A selector per hot stream reads the stream’s moderated comments and, every second, picks the best ~10 by a cheap score: spam and toxicity scores, the author’s history, the broadcaster or verified accounts, novelty against the last few seconds. Twenty thousand candidates a second is a top-K problem small enough for one process, or a few shards merged each second.
• Collapse the chorus. Near-identical comments (“OUT!!!”, “out”, “OUTTT”) become one line — “4,210 people said OUT” — which is both cheaper and more fun than 4,210 lines.
• Select per audience. Run the selection per language, so Hindi, Tamil and English viewers each get ten readable comments a second, and gateways subscribe to ipl-final:hi or ipl-final:ta rather than one channel.
• Batch per second. The selected comments and the latest tally travel as one frame per second. Every viewer of a language channel receives identical bytes, so each gateway serialises and compresses the frame once and writes the same buffer to up to 100,000 sockets.
• Personal touches at the edge. Your own comment is shown by your app the moment you post it; it does not need to win selection to appear on your screen.

What you pay: most comments on a hot stream are seen only by their authors — make that honest in the product (“top comments”, with a tab to slow-scroll everything from history). The selector is a per-stream component that must fail over quickly; if it dies, the dispatchers forward the first ten moderated comments of each second until a replacement takes over. And the threshold needs hysteresis, so a stream hovering around 20 comments a second does not flap between modes.

How do we keep comments in order without duplicates or gaps?

Viewers forgive a missing comment; they notice comments that jump around or appear twice. Order only matters within a stream, and only roughly.

Bad: order by client timestamps. Phone clocks disagree by seconds, and a comment “from the future” sticks to the top of the feed.

Good: order by arrival at each gateway. Free, but two gateways can receive two comments in different orders, so two friends watching side by side see different sequences, and a reconnect has no way to say “where was I?”.

Great: a per-stream sequence number assigned at write time. The sequencer’s INCR gives every stored comment a number that increases within its stream — 20,000 increments a second on one key is comfortably within a single Redis shard. Everything downstream uses it:

• Order. The client inserts by sequence, not by arrival, and holds an out-of-order item for up to about 300 ms waiting for the one before it.
• Duplicates. A retry or a replay after reconnect delivers a sequence the client already has; it drops it. The same client_msg_id makes the post idempotent, so a double-tapped send stores one comment (see idempotency).
• Gaps. A missing number is a hint, not an error: the sequencer can skip numbers (an INCR whose write then failed), and moderation removes some. On a small stream the client asks the gateway to replay a range only if the gap persists. On a selected stream gaps are normal, so frames carry their own frame sequence, and that is what Last-Event-ID resumes from.

What you pay: the sequencer is on the write path for every comment, so it needs a fast failover. If it is unavailable, the comment service can fall back to a timestamp-based ID that is still roughly ordered (see design a unique ID generator), accepting a few out-of-order comments for a few seconds. Across different streams, no order is promised or needed.

How do we fit storage, moderation and fan-out into two seconds?

The write path has three jobs that pull against speed: store the comment (so it can be replayed and audited), check it (so abuse never reaches ten million screens), and publish it. Do them in that order, each with a budget.

Bad: fan out first, check later. Fastest to the screen — and then a slur has been shown to the whole audience before anyone can remove it. A “delete” event chasing the comment helps only those who have not read it yet.

Bad, the other way: queue every comment for thorough review. Safe but useless: by the time a heavy model or a human approves “What a catch!”, the over has ended.

Great: a fast inline check with a strict timeout, and slower checks after. Twitch’s chat architecture has a service, Clue, that runs user actions through business logic: is this user banned in this channel, are they behaving abusively? Do the same before fan-out:

• Inline, ≤ 150 ms: a blocklist, per-channel rules set by the broadcaster’s moderators, the author’s reputation and recent rate, and a small toxicity model. Outcomes: publish, reject, or shadow — the author sees their own comment, nobody else does, which gives a spammer no signal to adapt to.
• On timeout: fail towards caution on hot streams (hold the comment) and towards openness on small ones (publish, re-check asynchronously). It is a per-stream policy, decided in advance.
• After publish: a heavier model and user reports run asynchronously. A removal sends a delete event with the comment ID; clients remove it if shown, and the history store marks it hidden.

The selection window is the one deliberate delay: waiting up to a second for candidates is what lets the selector choose well and lets each viewer receive one frame a second. Everything else is small, and the 2-second p95 target leaves room for a slow phone network. Note also what this budget is not measured against: the video itself typically lags several seconds behind real time — Meta’s live player fetched three-second segments — so comments usually arrive ahead of the picture. That is a product problem with a design answer, covered in the follow-ups.

How do we show “9.8M watching” without counting every heartbeat centrally?

The viewer count looks like presence, and LinkedIn’s presence platform shows the pattern for that: periodic heartbeats every d seconds, entries that expire after d + ε, so a phone that drops and reconnects within d seconds never flickers offline. But a viewer count needs only a number per stream, not a status per person — so aggregate before you centralise.

Bad: a central increment on connect and decrement on disconnect. One hot counter takes millions of updates during a surge, and every lost disconnect (a crashed gateway, a phone that walked into a lift) leaks a viewer forever.

Great: each gateway reports its own counts, as soft state. A gateway already knows exactly how many of its connections watch each stream. Every 5 seconds it writes (stream, gateway) → count to a counter store with a 15-second TTL; an aggregator sums the live rows per stream and publishes the total in the next tally. A crashed gateway’s rows simply expire, and its viewers — who reconnect elsewhere — are counted by their new gateway.

Two refinements. Connections are not people — one viewer with the app open on a phone and a TV counts twice — so if the number must mean people, gateways also add user IDs to a HyperLogLog per stream, which estimates uniques in at most 12 KB with a 0.81% standard error in Redis’s implementation. And round what you show (“9.8M watching”): nobody needs the last digit, and a rounded number changes less often, so it compresses into fewer tally updates.

How do we serve many regions and protect the fleet from slow clients?

A final in Mumbai is watched from Singapore, Dubai, London and Toronto. Viewers should connect to a nearby region; comments should reach every region; and no single phone on a weak signal should be able to hurt a gateway.

Regions: subscribe locally, publish everywhere. Each region runs its own gateways, dispatchers and subscription store. There are two ways to cross regions: let gateways subscribe across regions, or let dispatchers publish to their peers. LinkedIn chose the second — a dispatcher forwards each event once to the dispatcher in every other data centre, and each peer delivers it using its own local subscriptions. That keeps the high-churn data (millions of viewers joining and leaving) inside each region and sends only the low-volume data (published events) across. For a selected stream it is cheaper still: the selector runs once in the stream’s home region, and each region receives one frame a second.

Slow clients: never let one socket hold memory hostage. LinkedIn’s gateways write to connections fire-and-forget — no thread waits on a slow client. But bytes still queue somewhere, so bound them:

• Give each connection a send queue of at most a few frames (about 5 seconds).
• When it is full, drop the oldest comment frame and keep only the newest tally — counts supersede each other, comments are best effort anyway.
• When a connection falls more than about 30 seconds behind, close it. The client reconnects with Last-Event-ID and gets a fresh snapshot instead of a stale backlog. This is backpressure applied per socket.

Reconnect storms. When a gateway dies, its 100,000 viewers reconnect at once; when a region fails, millions do. Clients must back off with jitter (the SSE retry: field lets the server set the base delay), the load balancer must spread new connections across healthy gateways, and the fleet needs the 50% headroom from the estimates — TLS handshakes make a reconnect storm a CPU storm. Deploys should drain a gateway gradually: stop accepting, then ask its clients to reconnect, spread over a minute, instead of cutting 100,000 connections at once. See timeouts, retries and backoff with jitter.

The final design

1. Connect. The app opens one SSE connection through the edge load balancer to a gateway, naming the stream. The gateway adds it to its in-memory table, registers the stream in the subscription store if it is new to this gateway, and sends a snapshot of the last 50 events from its ring buffer.
2. History, if needed. A client whose gap is older than the buffer fetches the latest 50 from the history API, served by the CDN with a one-second TTL and request coalescing in front of the history cache.
3. Post. Comments and reaction counts go up over HTTP to the write API, which authenticates and rate-limits.
4. Store and check. The comment service takes a per-stream sequence number, writes the comment store, and runs the inline moderation check within 150 ms. Small streams publish straight to the dispatchers.
5. Count reactions. Sharded counters sum reactions per stream and emit one tally a second.
6. Select. For hot streams, the selector picks about ten comments a second per language, collapses duplicates, and adds the latest tally into one frame a second.
7. Dispatch. A dispatcher looks up which gateways watch the stream, forwards the event once to peer regions, appends it to the history cache, and sends it to each subscribed gateway.
8. Deliver. Each gateway writes the event to every matching connection from memory, with a bounded send queue per connection, and keeps it in its ring buffer for resumes.

Check it against the requirements. Small-stream comments arrive in about 350 ms and hot-stream comments in under 1.5 s, inside the 2-second p95. Ten million viewers cost 100 gateways and 10 million frames a second, not 200 billion messages. Stored comments survive for replay and audit; reactions survive as per-second tallies. Order and resume come from per-stream sequence numbers. And every tier scales sideways: gateways with viewers, dispatchers with event rate, selectors with the number of hot streams.

What interviewers look for by level

Follow-up questions

What if comments must line up with the video?

The video usually lags real time by several seconds — segments must be encoded, delivered and buffered — and the lag differs per viewer, so comments about a catch can arrive before the catch. Stamp every frame with the server time, have the player report how far behind the live edge it is (say, 12 seconds), and let the client hold incoming frames until its video reaches that moment. The server stays the same; only the client delays rendering. The same timestamps let a replay show comments exactly where they happened.

What if viewers should also see their friends’ comments on a hot stream?

Selection would drop them, but the internal fan-out of every candidate to the gateways is affordable even when the last mile is not: 20,000 comments a second × 100 gateways is 2 million internal messages a second. Send the full candidate stream to gateways on a separate channel; when a viewer connects, load their friend list into a small Bloom filter on the gateway, and add any candidate whose author matches into that viewer’s next frame. Personalisation happens at the edge, where the per-viewer state already lives.

What if a paid comment must never be dropped?

Give it a priority lane. Paid comments bypass selection, are pinned for a few seconds in every frame, and are stored with their payment ID. Because their delivery has to be dependable rather than best effort, the client confirms it has rendered them, and anything not confirmed is replayed from history on the next resume. Moderation still runs first — paying does not buy the right to abuse.

What if a whole region fails in the middle of the match?

Its viewers reconnect to the next-nearest region, which must have headroom for them — size each region to absorb a neighbour’s share. Clients spread the reconnects with jittered backoff, gateways in the receiving region serve snapshots from their own buffers (they already carry the stream), and the stream’s selector fails over to another region, using the comment store’s replica to rebuild its recent window. Viewers see a few seconds of silence, then the feed resumes; comments posted during the gap are in the store and reappear in history. See multi-region and active-active.

How to say it in the interview

“I’ll design live comments and reactions for a live video, from ten viewers up to ten million on one stream, with comments visible to everyone within about two seconds.

Viewers post over plain HTTP and receive over one persistent connection — server-sent events, because it’s ordinary HTTP, resumes with Last-Event-ID, and our traffic down is far bigger than up. Gateways hold about a hundred thousand connections each, so a ten-million-viewer stream needs about a hundred of them.

Routing uses subscriptions at two levels. Each gateway keeps an in-memory table of which of its connections watch which stream, and registers the streams it serves in a shared store as soft state. A dispatcher sends each event only to gateways watching that stream, and each gateway fans out in memory. So a dispatcher’s cost depends on gateways, not viewers.

The write path is persist, moderate, publish: a per-stream sequence number from a Redis counter, a write to the comment store, and an inline moderation check with a 150-millisecond timeout. Sequence numbers give ordering, deduplication and resume.

The hard part is the hot stream. Twenty thousand comments a second to ten million viewers is two hundred billion deliveries a second, and nobody can read that. So above about twenty comments a second, a per-stream selector picks about ten good comments a second per language, collapses duplicates into counts, adds reaction tallies, and sends one frame per viewer per second — ten million frames a second, encoded once per gateway.

Viewer counts are soft state reported by gateways every five seconds. Across regions, the home region publishes each batch once to peer dispatchers, and every connection has a bounded send queue, so slow phones lose old frames instead of costing memory.”

Concepts used in this design

• Long polling, SSE and WebSockets — the downstream transport and its trade-offs.
• Real-time updates — the general pattern of pushing server changes to clients.
• Publish–subscribe — dispatchers routing events to subscribed gateways.
• Push vs pull — why viewers hold a connection instead of polling.
• Stateful vs stateless design — stateful gateways with in-memory subscriptions, stateless dispatchers.
• Redis — the per-stream sequencer, counters and sharded Pub/Sub.
• Consistent hashing — sharding dispatchers and subscriptions by stream.
• Backpressure and load shedding — bounded queues per connection, selection on hot streams.
• Idempotency — retried posts that store one comment.
• Count-min sketch and HyperLogLog — counting unique viewers cheaply.
• Heartbeats — keep-alives, dead-connection detection and soft-state expiry.
• Timeouts, retries and backoff with jitter — surviving reconnect storms.
• Multi-region and active-active — publishing across regions and failing over mid-match.
• Content delivery network — caching history for late joiners.

References and further reading

1. Akhilesh Gupta, “Streaming a Million Likes/Second: Real-Time Interactions on Live Video” (QCon London 2020, InfoQ transcript)infoq.comLinkedIn's real-time platform: SSE connections, in-memory subscriptions on frontends, dispatchers with a KV subscription store, cross-data-centre publish, 100K connections per machine, ~5,000 events/s per dispatcher, 75 ms p90.
2. LinkedIn Engineering, “Instant Messaging at LinkedIn: Scaling to Hundreds of Thousands of Persistent Connections on One Machine” (2016)linkedin.comWhy SSE was chosen over WebSockets for compatibility, and the limits hit on the way to 100K connections per node: accept backlog of 128, file descriptors, ~20 KB of memory per TLS connection, ephemeral ports.
3. LinkedIn Engineering, “Now You See Me, Now You Don't: LinkedIn's Real-Time Presence Platform” (2018)linkedin.comPresence from periodic heartbeats every d seconds, entries that expire after d + ε, and delayed triggers that absorb jittery mobile reconnects.
4. Meta Engineering, “Under the hood: Broadcasting live video to millions” (2015)engineering.fb.comThree-second HLS segments, edge caches serving over 98% of requests, 1.8% leakage, and request coalescing to protect the origin from a thundering herd.
5. Twitch Engineering: An Introduction and Overview (2015)blog.twitch.tvChat as Edge nodes speaking IRC over TCP and WebSockets, a Pubsub tier forming a hierarchical fan-out, Clue for ban and abuse checks, and Room for viewer lists.
6. RFC 6455: The WebSocket Protocolrfc-editor.orgThe HTTP Upgrade handshake (101 Switching Protocols) and Ping/Pong control frames usable as keepalives.
7. HTML Living Standard, Section 9.2: Server-sent eventshtml.spec.whatwg.orgThe EventSource interface, id and retry fields, the Last-Event-ID header on reconnect, and a comment line every ~15 s to keep proxies from dropping idle connections.
8. Android Developers, “Optimize network access”developer.android.comThe radio state machine: each transfer keeps a typical 3G radio drawing power for about 18 s, so bundling transfers saves battery.
9. Redis documentation: Pub/Subredis.ioAt-most-once delivery for Pub/Sub, and sharded Pub/Sub (Redis 7+) that hashes channels to slots like keys.
10. Reddit Inc., “How we built r/place”redditinc.comClients subscribe to live updates before fetching full state and replay what arrived meanwhile; the WebSocket tier peaked at over 4 Gbps across 24 instances.