Layer 3 · Go deeper · 9 min
Go deeper
Size it, build it, then make the calls an interviewer will push you on.
Back of the envelope
Size it up
Tap through each step of the arithmetic.
How many workers does each order API instance need?
How fast can one TCP connection copy over a 100 ms, 1 Gbps path?
Build-up · 6 steps
Add up one HTTPS call
One call from a phone, row by row. Each delay starts when the previous one ends. Watch how little of it the server sees.
Step 1 of 6
Look up the name
Illustrative numbers for a phone with a 40 ms round trip. A fresh call first spends 20 ms on DNS, and nothing has reached your server yet.
4 interview moments
You’re the candidate
The interviewer pushes on the hard parts. Pick an approach and see how it holds up.
Deep dive 1 of 4
Interviewer
“The product manager says search “should be fast”. What latency requirement do you write down?”
Your move. Pick an approach:
Deep dive 2 of 4
Interviewer
“Each order request holds a database connection for two 4 ms queries, at 2,000 requests a second across 10 instances. How big is each pool?”
Your move. Pick an approach:
Deep dive 3 of 4
Interviewer
“Forcing the database log to disk takes 2 ms, and commits now arrive at 10,000 a second. How do you keep up?”
Your move. Pick an approach:
Deep dive 4 of 4
Interviewer
“A user request fans out to 100 servers that answer in 10 ms but take 1 second at p99. How do you tame the tail?”
63% of user requests now take over a second.
Your move. Pick an approach:
Build-up · 5 steps
Size, cap and queue with one law
Little’s law, L = λ × W, links in-flight work, arrival rate and time inside. Rearrange it to answer the capacity questions you meet in design.
Step 1 of 5
Draw any boundary
Little’s law holds for any stable system, a thread pool, a queue or a whole region. 2,000 requests a second, each inside for 150 ms on average, means about 300 in flight.