Design Twitter

Twitter looks like a smaller cousin of the news feed: people post short messages, follow each other, and read a timeline. The differences are what make it a hard interview. The follow graph is one-way and wildly lopsided — a pop star can have tens of millions of followers while following a few hundred. Everything is public and about now, so a tweet must be searchable within seconds and a sudden burst of tweets about a final over must surface as a trend within a minute or two. And traffic is shaped by the world’s attention: in 2013 one moment in a TV airing of a Japanese film drove 143,199 tweets in one second, about 25 times the normal rate. In this problem you design the core of Twitter: posting, the home timeline, tweet IDs, real-time search, trends, engagement counts, and how all of it survives a spike.

Understand the problem

Read the prompt as five products sharing one stream of tweets: posting, the follow graph, the home timeline, search and trends. The home timeline is the classic fan-out problem, and the news feed article covers its general theory — push versus pull, cost per author–reader pair, ranking. Here, spend your time on what is specific to Twitter: an asymmetric graph with extreme accounts, IDs, real-time search, trends and spikes.

Functional requirements

1. Users should be able to post tweets of up to 280 characters with optional media, reply to tweets, retweet them and like them.
2. Users should be able to follow any account without approval, and unfollow it.
3. Users should be able to read their home timeline — tweets and retweets from accounts they follow, newest first — and any user’s profile timeline.
4. Users should be able to search recent tweets by keyword or hashtag and find a tweet within seconds of it being posted.
5. Users should be able to see trending topics for their country or city, updated every minute or so.

Non-functional requirements

• Posting is never blocked. A tweet is acknowledged in under 500 ms at p99 and is durable once acknowledged — even at 25× normal traffic.
• Timelines are fast. The home timeline returns in under 50 ms at p50 and 200 ms at p99 on the server.
• Delivery is quick. A tweet reaches followers’ home timelines within about 5 seconds at p50; under a big spike, delivery may lag but must catch up within minutes.
• Search is real time. Tweets are searchable within about 10 seconds; queries return in under 100 ms at p50.
• Trends are timely. A topic that takes off appears within a couple of minutes and disappears when it fades.
• Availability over consistency. 99.99% for reading and posting. Timelines, counts and trends are eventually consistent; an author always sees their own tweet immediately.
• Scale. 300M daily active users, 500M tweets a day, follower counts from zero to about 100M.

Out of scope

• Direct messages — a chat system; see design WhatsApp.
• Notifications and push — see design a notification system.
• Media upload, transcoding and delivery — tweets reference media stored in a blob store behind a CDN; see design Instagram and design YouTube.
• The ranked “For You” timeline in depth — covered briefly as a follow-up.
• Ads, moderation and spam classifiers — we consume their verdicts as filters.

The set-up

Plan the approach

Walk the life of one tweet. It is given an ID, written once, and published to a stream. Several consumers read that stream independently: fan-out builds home timelines, the search ingester indexes it, the trends pipeline counts its terms, and counters track its engagement. Each consumer can fall behind without blocking the others or the author. Build each requirement simply, then go deep on timelines, IDs, search, trends, spikes and counters.

Core entities

• Tweet. tweet_id (64-bit, time-sortable), author, text, media IDs, reply_to, quote_of, creation time. A retweet is a small tweet that points at the original.
• User. Handle, display name, avatar, follower and following counts, flags such as verified or protected.
• Follow edge. Follower → followee, stored in both directions so “whom do I follow?” and “who follows me?” are each one read.
• Home timeline entry. A reference in a reader’s cached timeline: tweet ID, author ID, flags, and for a retweet the original tweet’s ID.
• Engagement. Like and retweet edges (user → tweet) plus per-tweet counts.
• Trend. A term or hashtag, a location, a score and a window.

API

Posting returns the new ID immediately:

POST /2/tweets
Authorization: Bearer <token>
Idempotency-Key: 9b3f6c1e-2d4a-4f7e-8a1b-5c6d7e8f9a0b
Content-Type: application/json

{ "text": "What a finish! #IPLFinal", "media_ids": ["1849302211400040448"] }

{ "id": "1850125436912345088", "created_at": "2026-10-27T14:03:12.418Z" }

Timelines page with ID cursors, which works because IDs sort by time:

GET /2/timelines/home?count=50&max_id=1850125436912345087
GET /2/users/{user_id}/tweets?count=50&max_id=…

{
  "tweets": [
    { "id": "1850125430011022336", "author_id": "44196397", "text": "…",
      "like_count": 18234, "retweet_count": 2210, "reply_count": 940 }
  ],
  "next_max_id": "1850125430011022335"
}

Engagement, follows, search and trends:

POST   /2/tweets/{id}/retweets
PUT    /2/tweets/{id}/likes/me
PUT    /2/users/me/following/{user_id}
GET    /2/search/recent?q=%23IPLFinal&since_id=1850125000000000000
GET    /2/trends?location=mumbai

max_id means “older than this ID” and since_id means “newer than this ID”. Because a tweet ID encodes its creation time, those cursors need no separate timestamp and never skip or repeat when new tweets arrive (see pagination: offset vs cursor).

Back-of-envelope estimates

Tweets. 500M a day — the volume Twitter itself has cited. 500M ÷ 86,400 ≈ 5.8k tweets/s on average (Twitter quoted 5,700/s in 2013). The record one-second peak was 143,199/s, about 25× normal. Plan to accept 10× for minutes and 25× for seconds.

Home timeline reads. 300M daily users × 30 timeline requests a day (opens, refreshes, scrolls) = 9B a day. 9B ÷ 86,400 ≈ 104k reads/s, about 300k/s at peak — the same order as the 300k timeline queries a second Twitter reported in 2012.

Fan-out deliveries. Assume each tweet lands in about 100 cached home timelines, once you skip inactive users and the biggest accounts. 500M × 100 = 50B inserts a day; 50B ÷ 86,400 ≈ 580k inserts/s, about 1.7M/s at a 3× peak. (In 2012 Twitter reported 30B deliveries a day, about 300k a second.)

Timeline cache memory. Keep the newest 800 entries per active user at 20 bytes each: 800 × 20 B = 16 KB per user. 300M × 16 KB = 4.8 TB; with about 1.5× data-structure overhead, 7.2 TB, and ≈ 22 TB with three copies.

Tweet storage. About 1 KB per tweet with metadata: 500M × 1 KB = 500 GB a day, about 180 TB a year before replication. Media is far larger and lives in a blob store.

Search index. A real-time index of the last 7 days holds 7 × 500M = 3.5B tweets. If one index server holds about 100M tweets (12 segments of about 8.4M, the layout described below), you need 3.5B ÷ 100M ≈ 35 partitions, each with several replicas for query load.

Trend events. About 3 countable terms per tweet (hashtags, names, phrases) gives 1.5B term events a day ≈ 17k/s, and over 400k/s at a 25× spike.

High-level design

Build each requirement in its simplest working form; the deep dives then make each one survive Twitter’s extremes.

Users should be able to post tweets

The client uploads any media first, then calls POST /2/tweets. The tweet service validates the text, gets a new ID from an ID service, writes the tweet once to the tweet store, and publishes a tweet.created event. It returns the ID straight away. Everything else — timelines, search, trends — happens downstream of that event.

The tweet store is a partitioned key-value store keyed by tweet ID, plus a second index of (author ID, tweet ID) for profile timelines. A retweet is a tiny tweet that points at the original, so it gets its own ID, its own place in timelines and its own author. Likes are edges (user → tweet) with a per-tweet count, written by an engagement service — the counting deep dive covers how those counts survive a viral tweet.

Users should be able to follow any account

A follow is an edge in the social graph, written twice — under the follower and under the followee — so “whom does Asha follow?” and “who follows this account?” are each a single range read on one shard. Twitter’s graph store, Flock, ran on sharded MySQL and was described in 2017 as handling peaks of tens of millions of queries a second. Follower and following counts are kept next to the lists. Unfollowing deletes the edge; tweets already in the reader’s cached timeline are filtered out when it is next read, as the news feed design describes.

What makes Twitter’s graph unusual is not its size but its shape. Following needs no approval and is one-way, so a handful of accounts collect tens of millions of followers while following very few. Twitter’s 2012 timelines talk listed the largest accounts at the time at 23M to 31M followers. That asymmetry drives the next requirement.

Users should be able to read their home timeline

The answer Twitter described in its 2012 “Timelines at Scale” talk is fan-out on write into Redis. When a tweet arrives, a fan-out service asks the social graph for the author’s followers and inserts the tweet’s ID into each follower’s home timeline, a Redis list keyed by the recipient. Reading the home timeline is then a range read of one list plus hydration of the IDs into full tweets and authors.

Twitter’s 2012 talk gives the details. Inserts are keyed by recipient, sent to Redis in pipelined batches of 4,000 destinations at a time, and replicated. Each timeline is a native Redis list whose entries are just a tweet ID (8 bytes), the author’s user ID (8 bytes) and 4 bytes of flags; a retweet entry carries the original tweet’s ID as well. Fan-out uses RPUSHX, which appends only if the list already exists, so users without a cached timeline cost nothing. The timeline service reads the list and hydrates it through the tweet and user services. In 2017 Twitter described this cache, Haplo, as a customised Redis written by the fan-out and timeline services.

That design reached 300k timeline reads a second at 1 ms p50 and 4 ms p99 for the timeline read itself. It has one glaring weakness, which the first deep dive takes apart: accounts with tens of millions of followers.

Users should be able to search recent tweets

Search is the mirror image of the home timeline. A home timeline does the work at write time so reads are cheap; search does almost nothing at write time beyond indexing, and does the work at read time. Twitter’s talk summed it up in two lines: the Redis timelines are O(1) to read and O(n) to write, while the search index is O(1) to write and O(n) to read.

An ingester tokenises each tweet, annotates it (language, entities) and hashes its ID to one index partition. A front end — Twitter calls it the Blender — sends each query to one replica of every partition, then merges and re-ranks what comes back. The Earlybird paper reported tweets searchable about 10 seconds after creation and queries answering in about 50 ms on average, at over two billion queries a day in 2011. How an index can accept writes and serve reads at the same time is the third deep dive.

Users should be able to see trending topics

Trends come straight from the stream. A term extractor pulls countable terms from each tweet — hashtags, names, short phrases — drops obvious spam, and counts each user at most once per term. Counts are kept per term, per location, per minute. A scorer compares each term’s recent count with its own normal level, and the top few per location are written to a small cache that GET /2/trends reads.

You now have a working Twitter: posting, a graph, Redis home timelines, a search index and trends. The deep dives are about the places where it breaks — big accounts, IDs, real-time indexing, counting at scale, spikes and hot counters.

Deep dives

How do we build home timelines when some accounts have 100 million followers?

The general trade-off — push costs a write per follower, pull costs a read per followee, and the cheaper choice depends on each author’s posting rate and each reader’s viewing rate — is worked through in design a news feed. Twitter’s version of the problem is sharper, because the graph is so lopsided. In 2012 the largest account had 31M followers; this design assumes the largest now have about 100M.

Do the arithmetic at the rate Twitter reported in 2012, about 300k deliveries a second across the whole system, and pretend one tweet had all of it:

And that is the optimistic case: every other tweet posted during those minutes is waiting in the same queue. Followers see the big tweet minutes apart, replies to it can reach timelines before the tweet itself, and ordinary users’ tweets are delayed behind it.

Bad: push everything. Simple and fast to read, but the delivery time and the queue backlog are set by the biggest account on the platform, and its tweets are exactly the ones everyone is waiting for.

Good: pull everything from an index. Build the home timeline at read time by asking the search index for recent tweets from the accounts you follow. Twitter ran search-based timelines too — over 30k queries a second in 2012 — but making every one of 300k home-timeline reads a second an O(n) query across every index partition costs far more than a list read.

Great: push for almost everyone, merge the giants at read time. Fan-out skips authors above a threshold — say a million followers; Twitter has not published its exact rule — and their tweets stay only in their own profile timelines. When you read your home timeline, the timeline service takes your cached list and merges in the recent tweets of the few big accounts you follow. Twitter’s talk described this as query expansion: your home timeline becomes “home_timeline:you OR user_timeline: each big account you follow”.

The read side stays cheap because there are few giants and they are shared. If a typical reader follows half a dozen accounts above the threshold, a timeline read is one LRANGE plus six reads of profile timelines — and those profile timelines are among the hottest keys in the system, so they sit in cache, replicated across many cache servers. Pick the threshold where a single tweet’s fan-out stops fitting the delivery target: at 300k deliveries a second, 5 seconds buys about 1.5M deliveries.

Four more details complete the Twitter version:

• Only active users get a cached timeline. RPUSHX appends only to lists that exist, so the 800-entry lists live only for people who have used the app recently. A returning user’s timeline is rebuilt once from the graph and their followees’ profile timelines, then kept current by fan-out.
• Entries are tiny. At 20 bytes — tweet ID, author ID, flags — an 800-entry timeline is 16 KB, which is how 300M timelines fit in about 22 TB of replicated memory.
• Retweets need deduplication. If five people you follow retweet the same tweet, you should see it once. Retweet entries carry the original tweet’s ID, so the timeline service can collapse them while reading.
• Deletes and protected accounts are checked at hydration, not retracted from millions of lists; the tweet service simply returns nothing for a deleted tweet.

How do we give every tweet a unique, time-sortable ID without coordination?

Every tweet needs an ID the moment it is posted — up to 143k a second in a spike — and that ID does three jobs: it is the storage key, it orders timelines, and it is the API cursor (since_id, max_id). Twitter’s 2013 write-up explains why this became urgent: tweets lived in a MySQL setup sharded by time with a single master, so every spike was serialised into one database and created read and write hot spots. Moving to storage that hashes tweets across many databases meant giving up MySQL’s auto-increment IDs, and Snowflake was built to replace them.

Bad: database auto-increment. Strictly ordered and compact, but one master hands out every ID, so the ID generator becomes the write bottleneck at exactly the moment traffic spikes.

Good: random UUIDs. No coordination at all, but 128 bits doubles the size of every key and every timeline entry, and random IDs do not sort by time — so you need a separate timestamp everywhere and lose ID-based cursors. Snowflake’s README is explicit that the IDs had to stay under 64 bits.

Great: Snowflake. Each ID worker composes a 64-bit integer from the current time, its own machine number and a per-millisecond counter. No worker talks to another.

The numbers fall out of the bit widths. Forty-one bits of milliseconds is 2^41 ms ≈ 69.7 years from a custom epoch. Ten bits allow 1,024 workers. Twelve bits allow 4,096 IDs per millisecond per worker — 4.1M IDs a second from one worker, against a record peak of 143k tweets a second. The README’s own targets were modest: at least 10k IDs a second per process and about 2 ms per response. You run a few workers per data centre for availability, not throughput.

Three operational rules keep Snowflake honest:

• Unique machine numbers. Two workers with the same 10-bit number can mint identical IDs in the same millisecond. Assign numbers from configuration or lease them from a coordination service such as ZooKeeper at start-up, and refuse to start without one.
• Clocks that never run backwards. If NTP steps a clock back, the worker could reissue old timestamps. The README’s rule is to refuse to generate IDs until the clock passes the last timestamp used — a short outage on one worker rather than duplicate IDs. Running NTP in a slewing mode avoids most steps (see clocks, ordering and conflict resolution).
• Sequence overflow waits. If a worker issues 4,096 IDs within one millisecond, it spins until the next millisecond.

Two side effects matter to API users. IDs sort only roughly across workers — the README calls them k-sorted, promising order within a second and aiming for tens of milliseconds — so since_id polling should allow a little overlap. And a Snowflake ID around 1.8 × 10^18 is far above 2^53 ≈ 9 × 10^15, the largest integer a JavaScript number holds exactly, so JSON responses should carry IDs as strings, as in the API above. The general problem is covered in design a unique ID generator.

How do we make a tweet searchable within ten seconds?

Web search can rebuild indexes in batches and accept hours of staleness. Twitter search is mostly about the last few minutes: what people are saying about the earthquake, the match, the outage. The index must take thousands of new tweets a second and answer queries in tens of milliseconds at the same time, newest results first.

Bad: batch indexing. Rebuild or merge the index every hour from the tweet store. Simple and efficient, but an hour of lag defeats the purpose.

Good: a general-purpose search engine with near-real-time refresh. Engines such as Elasticsearch make new documents visible by periodically opening small new segments and merging them in the background. That works for many products; at tens of thousands of writes a second, with spikes of 25×, the constant segment creation, merging and garbage collection compete with queries for the same CPU and memory.

Great: an index built for append-only, time-ordered documents. That is what Twitter built in Earlybird, on top of Lucene. The design exploits two facts about tweets: they are short, and they arrive in time order and are never updated in place.

• Partition by tweet, not by term. Tweets are hash-partitioned across index servers as they arrive, so a single hot hashtag spreads across every partition instead of melting one. The price is that every query fans out to all partitions; the Blender sends it to one replica of each, then merges and re-ranks.
• Segments with a single active one. Each server holds 12 segments of about 2^23 ≈ 8.4M tweets. New tweets fill one segment at a time; only that segment is ever written. When it fills, it is converted to a compact read-only form and a new one starts.
• Postings that are plain integers. Each posting is a 32-bit integer: 24 bits of document ID within the segment and 8 bits of position in the tweet. Eight bits suffice because a tweet is short — even at 280 characters it cannot hold more than about 140 words.
• Append forwards, read backwards. Postings for a term are appended in arrival order and traversed from the newest end, which gives “most recent first” without sorting. Postings grow through pools of slices of 2, 16, 128 and 2,048 integers, so a growing list is never copied and leaves no garbage for the JVM to collect.
• One writer, many readers, no locks. A single thread writes each segment. It publishes progress through one volatile variable, maxDoc, which acts as a memory barrier: readers only look at documents up to the last published maxDoc, so they never see a half-written tweet.

Size it with this design’s numbers. Hash-partitioned 35 ways, each partition receives 500M ÷ 35 ≈ 14.3M tweets a day — about 166 a second on average and about 4.1k a second at the 143k/s record — so one 8.4M-tweet segment fills in roughly 14 hours, and 12 segments hold about a week. That matches the 7-day real-time index from the estimates. In this design, older tweets go to a separate archive index on cheaper storage, queried only when the real-time index runs out of results.

Ranking happens inside the same fan-out. The paper describes three kinds of signals: static ones computed at indexing time, resonance signals such as retweet counts that an updater pushes to the index servers as they change, and the searcher’s own social graph, sent along with the query so tweets from people you follow score higher. Each partition returns its best, most recent matches; the Blender merges and re-ranks them. The paper reported tweets searchable within about 10 seconds, 50 ms average query latency, and more than two billion queries a day as of 2011. For the general machinery of inverted indexes, see how search engines work.

How do we find what is trending right now?

Start from Twitter’s own definition. Its 2010 explanation of trends says the algorithm looks for topics “being talked about more right now than they were previously”: the hottest emerging topics, not the most popular ones. A hashtag can be mentioned enormously and still not trend if its conversation is not growing fast enough relative to its normal level — which is why, the post explained, #wikileaks was not trending that week. So there are two problems: counting billions of term mentions cheaply, and scoring each term against its own baseline.

Counting. About 1.5B term mentions a day, 17k a second, and most distinct terms are one-off words and typos.

• Bad: increment a row per term in a database. 17k writes a second on average and over 400k in a spike, with the hottest term — the one that is trending — hammering a single row.
• Good: exact in-memory counts in stream workers, partitioned by term. Each worker owns a slice of the term space and keeps per-minute counts, emitting totals each minute. Memory grows with the number of distinct terms, though, and the trending term still lands on one worker; pre-aggregate upstream (each worker sends “term, count” per second instead of each mention), as described in design top-k trending.
• Great: bounded-memory heavy hitters. You only care about terms that are frequent, so use a heavy-hitter algorithm. Space-Saving keeps exactly m counters per place and window. If a term already has a counter, increment it; if not, evict the term with the smallest count, min, and give the newcomer min + 1 — remembering that it may be overcounted by up to min.

def space_saving(stream, m):
    counts, overestimate = {}, {}
    for term in stream:
        if term in counts:
            counts[term] += 1
        elif len(counts) < m:
            counts[term], overestimate[term] = 1, 0
        else:
            victim = min(counts, key=counts.get)   # the paper uses an O(1) structure
            low = counts.pop(victim); overestimate.pop(victim)
            counts[term], overestimate[term] = low + 1, low
    return counts, overestimate

The paper proves the guarantees that make this safe: the smallest counter never exceeds N/m for a stream of N items, and any term that really occurred more than that minimum is guaranteed to be held. Put numbers on it. If India produces 15% of term mentions, that is 225M a day, about 156k a minute. With m = 10,000 counters, N/m ≈ 16: every term mentioned more than about 16 times in a minute is tracked, and no count is off by more than about 16. At roughly 64 bytes a counter that is 640 KB per place per minute — 60 minutes for 500 places is about 19 GB across the cluster. A Count-Min sketch is the other common choice: a fixed grid of counters that estimates any term’s count, never undercounting, paired with a small heap of current leaders.

Scoring. Keep a baseline per term, place and hour of the week — for example the average for the same hour over the last four weeks — and score the last few minutes with a surprise measure such as (observed − expected) ÷ √(expected + 1). For #IPLFinal at 6,500 a minute against a baseline of 300, that is 6,200 ÷ 17.3 ≈ 357. For #GoodMorning at 8,000 against 7,900, it is 100 ÷ 88.9 ≈ 1.1. Brand-new terms get a small default baseline so they can trend on their first day.

Raw scores are easy to game, so add hygiene before ranking: count each account once per term per window, require a minimum number of distinct accounts, drop accounts flagged as spam, fold variants (#IPLFinal, #iplfinal, “IPL final”) into one topic, and keep a trend on the list for a few minutes once it appears so the list does not flicker. Counts roll up from city to country to worldwide by summing the same windows. Recomputing every minute puts a breakout topic on the list within about two minutes of its jump.

How do we survive a 25× spike in tweets?

On 3 August 2013, during a TV airing of the film Castle in the Sky in Japan, viewers took to Twitter at one moment and it took in 143,199 tweets in one second — about 25 times its 5,700-a-second average. Its engineering blog noted that users saw no blip. Three years earlier, the 2010 World Cup had repeatedly made Twitter unavailable for short periods. The same blog post explains what changed in between, and it maps neatly onto design principles: split the work by urgency, partition storage by hash, and give every service the same back-pressure behaviour.

Split work into “now” and “soon”. Only a few steps must finish before the author gets a response: validate, assign an ID, write durably, add the tweet to the author’s own timelines, acknowledge. Size that path for the 25× peak — about 150k tweets a second. Everything else reads the stream and may fall behind for a while.

Work the figure’s numbers. A minute at 10× normal is 58k tweets a second; at about 100 deliveries each, fan-out needs 5.8M inserts a second against a capacity of 3M. The backlog grows by 2.8M a second, reaching about 169M after 60 seconds. Afterwards it drains at 3M − 0.58M ≈ 2.4M a second, so it is gone in about 70 seconds, and the worst-delayed tweet waited 169M ÷ 3M ≈ 56 seconds. The record one-second peak is smaller: 143k × 100 = 14.3M inserts, an 11.3M backlog that clears in about 5 seconds. Drain smartly — deliver first to timelines of people who are online right now, since an offline reader will not notice a minute’s delay.

Partition storage by hash, not by time. Twitter’s earlier tweet storage filled one MySQL database at a time, in time order, behind a single master — so every spike landed on the newest database. Hashing each tweet across many databases spread writes and reads over the whole cluster, which is why it needed Snowflake IDs in the first place. The same rule applies to every store here: timelines by user, the index by tweet, counters by tweet.

Make back-pressure uniform. The 2013 post describes early service clients that retried slow services aggressively and caused thundering herds, and a shared RPC library (Finagle) that gave every service the same connection pooling, failover and load balancing. With bounded queues, timeouts and jittered retries everywhere, an overloaded component sheds or delays work instead of amplifying it — see backpressure and load shedding and retry storm.

Keep switches you can flip. Twitter described Decider, a runtime switch system that turns features on, off or to a percentage of traffic across many services in near real time. During a spike, the same switches let operators sample trends more coarsely, pause expensive recommendations, or delay non-essential fan-out — degrading gracefully rather than failing.

How do we store tweets and keep like and retweet counts at this scale?

Storage. A tweet is written once and read constantly, so the tweet store is a hash-partitioned key-value store keyed by tweet ID, replicated across data centres, with large caches in front. Twitter’s 2017 infrastructure overview describes cache as about 3% of its infrastructure but critical, with cache clusters handling 10M–50M queries a second each. Behind the caches sits Manhattan, which Twitter built as the backend for tweets, direct messages and accounts:

• Eventually consistent by default. Most of Twitter’s data favours availability, so replicas converge through continuous replica reconciliation, read repair and hinted handoff — the Dynamo-style toolkit.
• Strong consistency when you ask for it. A consensus-backed log provides check-and-set within one data centre (LOCAL_CAS) or across a quorum of data centres (GLOBAL_CAS). In this design, handle registration needs it — two people must never both get @asha — and almost nothing else does. See strong vs eventual consistency.
• An engine per access pattern. A log-structured merge tree for write-heavy data, a B-tree for read-heavy data, and a read-only format for datasets bulk-loaded from batch jobs. Twitter reported read-only clusters at tens of millions of queries a second and read-write clusters at millions.

Media lives in a separate blob store — hundreds of billions of objects by 2017 — behind a CDN; that pipeline is the subject of design Instagram.

Counts. Every tweet shows reply, retweet and like counts, and a tweet from a cricketer after a World Cup win can collect thousands of likes a second.

• Bad: increment a counter inside the tweet record. Every like becomes a read-modify-write of the same hot row — and of the record every reader is fetching.
• Good: an edge plus aggregated increments. Write the like as an edge (user → tweet) with a unique key, so retries and double taps are harmless, and emit “+1” only when the edge is new. A stream partitioned by tweet ID collects the increments and a consumer applies one “+N” per tweet per second. The news feed design walks through this pattern.
• Great: a dedicated counter service. Manhattan’s team built a time-series counters service for its observability platform, at millions of increments a second, after agreeing explicit trade-offs: how durable an increment must be, and how long before it becomes visible. Make the same choices for engagement counts. An increment may be lost in a crash, so a periodic job recounts edges for recently active tweets; a count may be a second or two stale, so serve it from cache and render it rounded (“18.2K”). The viewer’s own heart comes from their like edge, so it is always exact for them.

Counts are not just display. Earlybird’s ranking uses resonance signals such as retweet counts, which an updater pushes to the index servers as they change — so the counter stream feeds search ranking as well as the numbers under each tweet. Retweets also do double duty: a retweet is itself a tweet that fans out to the retweeter’s followers, and an increment on the original’s count.

The final design

Everything hangs off one stream. Synchronous work ends when the tweet is stored; each consumer owns its own read-optimised store, and each read path touches only that store plus the tweet cache.

Follow one tweet, then a timeline read:

1. Post. The client calls POST /2/tweets through the gateway, which authenticates and rate-limits.
2. ID. The tweet service takes a Snowflake ID — 41 bits of time, 10 of worker, 12 of sequence — with no coordination.
3. Store and acknowledge. The tweet is written to the hash-partitioned tweet store and added to the author’s own timelines; the author gets the ID back in well under 500 ms.
4. Publish. tweet.created goes onto the stream, partitioned by author so one author’s tweets stay in order.
5. Fan out. For an author below about a million followers, the fan-out service pages through followers from the social graph and appends 20-byte entries to cached home timelines with pipelined RPUSHX, 4,000 recipients a batch. Bigger authors are skipped.
6. Read. A home-timeline request takes one LRANGE of the reader’s list, merges the recent tweets of the few big accounts they follow by tweet ID, and hydrates the page through the tweet and user caches.
7. Index. The ingester tokenises the tweet and adds it to one of 35 Earlybird partitions, where it is searchable within about 10 seconds; the Blender queries one replica of every partition and merges.
8. Trend. Term counters keep Space-Saving summaries per place and minute; the scorer compares them with each term’s baseline and refreshes the trends cache every minute.

Check against the requirements. Posting does a handful of writes and is sized for 25×, so it never waits on fan-out, search or trends. Home timelines are one Redis range plus a few cached profile-timeline reads, comfortably inside 50 ms. Delivery is seconds for every author because the giants never fan out, and a 10× burst costs about a minute of delivery lag that drains on its own. Search is fresh within 10 seconds because the index is written in place, one active segment at a time. Trends need bounded memory per place however many distinct terms appear. Everything except handle registration is eventually consistent, which is what lets every component keep answering during failures.

What interviewers look for by level

Follow-up questions

What if the home timeline must be ranked, like “For You”?

Then the cached timeline becomes one candidate source among several. Twitter’s 2023 description of its For You pipeline pulls about 1,500 candidates per request — on average half from accounts you follow and half from accounts you don’t — scores them with a neural network of roughly 48M parameters that predicts ten kinds of engagement, then applies filters and heuristics such as author diversity and blocked accounts. It runs about 5 billion times a day, in under 1.5 seconds on average, using about 220 seconds of CPU per request. The same post says Twitter had recently stopped using its 12-year-old Fanout Service as the source of in-network tweets. The general lesson: once every request is ranked anyway, fetching followed accounts’ recent tweets from an index can compete with maintaining pushed lists. The ranking machinery itself is covered in design a news feed.

What if a tweet is edited or deleted after it has spread?

Timelines hold IDs, so nothing in 300M lists changes. Deletion marks the tweet as deleted in the tweet store, invalidates its cache entry, and removes it from the search index (mark the document deleted in its segment and filter at query time). Hydration then drops it from every timeline that still references it. Edits create a new version; Twitter’s 2023 post describes the feed checking whether tweets already on a device are stale and sending instructions to replace them with the edited version.

What if users want new tweets to appear live, without refreshing?

Keep a push channel open for active sessions — WebSockets or server-sent events through a connection gateway (see real-time connections). The fan-out service, after appending to a list, also publishes “new tweet for user X” to the gateway that holds X’s connection, if any; offline users simply read their list later. Twitter’s 2012 talk lists exactly this split: pull for the home timeline API, push for streaming connections and mobile notifications.

What if Twitter must run active-active in several regions?

Replicate the tweet stream and store between regions asynchronously; each region runs its own fan-out, index and trends from the replicated stream, so reads never cross regions. Snowflake keeps IDs unique as long as worker numbers are unique across regions — give each region its own range of the 1,024. The few operations that must be globally consistent, such as claiming a handle, use a cross-region check-and-set and pay the round trip. See multi-region and active-active.

What if a hashtag trends because of bots?

That is what the scoring hygiene is for: count each account once, require many distinct accounts, weight accounts by age and spam score, and hold back terms whose mentions come from a tight cluster of new or related accounts. Keep a human review queue for the top national list, and record why each trend was shown so it can be audited.

How to say it in the interview

“I’ll design Twitter for about 300 million daily users and 500 million tweets a day: roughly 6,000 tweets a second normally, spikes of 25 times that, and around 300 thousand home-timeline reads a second at peak.

Posting is short and synchronous: the tweet service takes a Snowflake ID — 41 bits of milliseconds, 10 of worker, 12 of sequence — so IDs are unique without coordination and sort by time. It writes the tweet to a hash-partitioned key-value store, adds it to the author’s own timelines, acknowledges, and publishes the tweet to a stream.

Everything else consumes that stream. Fan-out appends a 20-byte entry to each follower’s home timeline, a Redis list capped at about 800 entries, but only for users whose list is cached. Accounts above about a million followers are not fanned out at all; their recent tweets are merged in at read time, by ID. That keeps a timeline read to one Redis range plus a few hot cached reads, and keeps any single tweet’s delivery within seconds.

Search uses an index partitioned by tweet, with one writable segment per partition, postings appended in time order and read backwards, and a single writer so readers need no locks. Tweets are searchable in about 10 seconds and a query fans out to one replica of every partition.

Trends count terms per place per minute with a heavy-hitter algorithm like Space-Saving, so memory is bounded, and rank terms by how far they are above their own baseline, not by raw volume.

For spikes, only the posting path is sized for 25 times; fan-out, indexing, trends and counters are queues that may lag for a minute and drain. Storage is partitioned by hash, every service uses the same back-pressure, and runtime switches let us shed optional work.”

Concepts used in this design

• Push vs pull architecture — fan-out on write for most accounts, on read for the giants.
• Redis — home timelines as capped lists, RPUSHX, pipelining.
• Hot keys, cache stampede and invalidation — big accounts’ profile timelines and viral tweets.
• Clocks, ordering and conflict resolution — why Snowflake refuses backward clocks.
• ZooKeeper and etcd — leasing unique worker numbers.
• Pagination: offset vs cursor — since_id and max_id cursors.
• How search engines work — inverted indexes and postings.
• Search data structures — segments, postings lists and their layouts.
• Count-Min sketch and HyperLogLog — approximate counting for trends and views.
• Flink and stream processing — windowed counts over the tweet stream.
• Kafka — the partitioned tweet.created stream.
• Database sharding — hash, not time, as the partition key.
• Cassandra and DynamoDB — eventually consistent key-value storage like Manhattan.
• Strong vs eventual consistency — CAS for handles, eventual for everything else.
• Backpressure and load shedding — surviving 25× spikes.
• Real-time connections — live timeline updates.

References and further reading

1. Timelines at Scale (Raffi Krikorian, QCon San Francisco 2012, InfoQ)infoq.comSlides: 300k timeline QPS at 1 ms p50 and 4 ms p99, Redis lists of 20-byte entries filled with RPUSHX in pipelined batches of 4k, 30B deliveries a day, 3.5 s p50 to reach 1M followers, celebrity tweets merged at read time, Earlybird and Blender.
2. Snowflake README (Twitter, 2010)github.comThe 41-bit time, 10-bit machine and 12-bit sequence layout, 64-bit limit, k-sorting within about a second, and refusing to issue IDs when the clock goes backwards.
3. New Tweets per second record, and how! (Twitter Engineering, 2013, archived)web.archive.orgThe 143,199 Tweets-per-second peak, about 25 times the 5,700/s average, and the move from a temporally sharded single-master MySQL to hashed storage with Snowflake IDs.
4. Earlybird: Real-Time Search at Twitter (Busch et al., ICDE 2012)cs.uwaterloo.caHash-partitioned Lucene-based index, Blender merge, 10-second indexing latency, 50 ms queries, 12 segments of about 8.4M tweets, 32-bit postings and single-writer concurrency.
5. Manhattan, our real-time, multi-tenant distributed database for Twitter scale (Twitter Engineering, 2014, archived)web.archive.orgAn eventually consistent key-value store with opt-in strong consistency, LSM and B-tree engines, replica reconciliation and a counters service.
6. The Infrastructure Behind Twitter: Scale (Twitter Engineering, 2017, archived)web.archive.orgManhattan as the backend for Tweets, the Haplo timeline cache on customised Redis, cache clusters at 10M–50M QPS, and the Blobstore for media.
7. Twitter’s Recommendation Algorithm (Twitter Engineering, 2023, archived)web.archive.orgThe For You pipeline: about 1,500 candidates, half from outside your network, a roughly 48M-parameter ranker, 5B runs a day under 1.5 s, and retiring the old Fanout Service as a source.
8. To Trend or Not to Trend… (Twitter blog, 2010, archived)web.archive.orgTrends reward topics whose volume is rising sharply relative to their baseline, not the most popular topics overall.
9. Efficient Computation of Frequent and Top-k Elements in Data Streams (Metwally, Agrawal and El Abbadi, ICDT 2005)cs.ucsb.eduThe Space-Saving algorithm: m counters, replace the minimum, and the bound that any item more frequent than the minimum counter is kept.