Layer 3 · Go deeper · 11 min
Go deeper
Size it, build it, then make the calls an interviewer will push you on.
Back of the envelope
Size it up
Tap through each step of the arithmetic.
How big is the index, and does each shard fit in RAM?
How much CPU does one query cost?
How many replicas of each shard?
Build-up · 5 steps
Build the high-level design
One functional requirement at a time, ending with an engine that answers correctly over the whole web, just not yet fast or cheap enough.
Step 1 of 5
Turn pages into an index
The indexer parses, dedupes and tokenizes each crawled page, then sorts (term, docID) records into posting lists. It is off the query path: falling behind makes results staler, never slower.
6 interview moments
You’re the candidate
The interviewer pushes on the hard parts. Pick an approach and see how it holds up.
Deep dive 1 of 6
Interviewer
“Eight trillion postings, and the index must fit in RAM and decode millions of numbers per query. How do you store it?”
Your move. Pick an approach:
Deep dive 2 of 6
Interviewer
“42 TB won’t fit on one machine, and one CPU needs 2.5 s to decode a billion-posting list. How do you split the index?”
Your move. Pick an approach:
Deep dive 3 of 6
Interviewer
“Every query fans out to 1,000 leaves, and the p99 budget is 300 ms. How do you stop the slowest leaf setting the pace?”
Your move. Pick an approach:
Deep dive 4 of 6
Interviewer
““ipl score” matches hundreds of millions of pages, and you show ten. How do you rank them?”
Your move. Pick an approach:
Deep dive 5 of 6
Interviewer
“At 9:40 pm, “ipl score” wants pages updated minutes ago. How does a 20-billion-page index keep up?”
Your move. Pick an approach:
Deep dive 6 of 6
Interviewer
“A cricket-final spike is mostly a few hundred repeated queries. How do you stop paying full price for each?”
Your move. Pick an approach:
Build-up · 8 steps
Follow one query, then one page
Steps 1–5 must finish in a few hundred milliseconds. Steps 6–8 run continuously in the background and can fall behind without breaking a single query.
Step 1 of 8
Route and understand
Geo-aware DNS picks the nearest of three regions. The frontend normalises the query, detects language and script, fixes spelling, adds transliterated variants and classifies intent.