Layer 3 · Go deeper · 11 min
Go deeper
Size it, build it, then make the calls an interviewer will push you on.
Back of the envelope
Size it up
Tap through each step of the arithmetic.
Does the live set of articles and stories fit in memory?
How many requests a second reach our servers on results day?
How many counter updates do clicks cause at peak?
Build-up · 5 steps
Build the high-level design
One functional requirement at a time, ending with an aggregator that works for one region at moderate traffic.
Step 1 of 5
See new articles in minutes
Poll, push and crawl differ only in how you learn of a URL; from the fetcher on, every article follows one path. The clean record lands on a Kafka topic, so later stages can replay.
6 interview moments
You’re the candidate
The interviewer pushes on the hard parts. Pick an approach and see how it holds up.
Deep dive 1 of 6
Interviewer
“Priority stories must show within 5 minutes, from 100,000 publishers that don’t want to be hammered. How do you find new articles?”
Your move. Pick an approach:
Deep dive 2 of 6
Interviewer
“At 18:07 an agency moves a 400-word report. By 18:30, fifty sites run it under their own headlines. How do you spot the copies?”
Your move. Pick an approach:
Deep dive 3 of 6
Interviewer
““ISRO’s lander touches down” and “What the landing means for India’s space budget” share almost no shingles. How do you group articles into stories?”
Your move. Pick an approach:
Deep dive 4 of 6
Interviewer
“An edition has 50,000 live stories and 30 slots. A landing breaks at 18:00. How do you rank so it tops the list within minutes?”
Your move. Pick an approach:
Deep dive 5 of 6
Interviewer
“The feed has about 10 ms of server time to pick 20 stories from tens of thousands. How do you personalise it?”
Your move. Pick an approach:
Deep dive 6 of 6
Interviewer
“Results day: traffic doubles to 120,000 requests a second within minutes, and everyone wants the same few pages. How do you serve it?”
Your move. Pick an approach:
Build-up · 9 steps
Follow one article to one reader
The write side is small and runs top to bottom; the read path only touches snapshots, profiles and caches, so nothing a reader does waits on ingestion.
Step 1 of 9
Discover
WebSub pushes arrive in seconds; adaptive pollers fetch feeds and news sitemaps with conditional GETs; a small crawler covers the rest. About 150 fetches a second, not 3,300.