Layer 3 · Go deeper · 10 min
Go deeper
Size it, build it, then make the calls an interviewer will push you on.
Back of the envelope
Size it up
Tap through each step of the arithmetic.
With 5% serial work, how much can more workers buy you?
Add a small coherency cost. Where does throughput peak?
Going from 8 to 16 nodes, what happens to cost per request?
Build-up · 6 steps
Why more nodes stop helping
Start from the dream of N servers doing N times the work. Add a serial step, then cross-talk, and watch the curve bend, then fall.
Step 1 of 6
The dream: N nodes, N×
If scaling out were perfect, N servers would give N times the throughput of one. Two models explain why it doesn’t, and why it can even go backwards.
4 interview moments
You’re the candidate
The interviewer pushes on the hard parts. Pick an approach and see how it holds up.
Deep dive 1 of 4
Interviewer
“Traffic is heading for 10×. The interviewer asks why you don’t just add more app servers.”
Your move. Pick an approach:
Deep dive 2 of 4
Interviewer
“Steady peak writes have hit your single primary’s limit. A teammate suggests five read replicas. What do you do?”
Your move. Pick an approach:
Deep dive 3 of 4
Interviewer
“A Big Billion Days-style sale is on the calendar, and your stateless tier autoscales. How do you prepare?”
Your move. Pick an approach:
Deep dive 4 of 4
Interviewer
“Your one Postgres database runs on the biggest box you can afford, and data keeps growing. When do you start sharding?”
Your move. Pick an approach:
Build-up · 7 steps
From one box to cells
Most products climb the same steps. Each removes the bottleneck that ended the last one and adds moving parts, so climb only when a measured limit pushes you.
Step 1 of 7
One box
App and database share one machine, with nothing to coordinate and the lowest cost. They fight for CPU and memory, and any failure or deploy is an outage.