Layer 3 · Go deeper · 11 min
Go deeper
Size it, build it, then make the calls an interviewer will push you on.
Back of the envelope
Size it up
Tap through each step of the arithmetic.
How much disk does a week of records need?
How many brokers, and which resource runs out first?
How many position updates a second if consumers keep offsets?
Build-up · 5 steps
Build the high-level design
One functional requirement at a time, ending with a cluster that works at moderate load.
Step 1 of 5
Publish records durably
One file can’t take 3 GB/s or survive a dead disk, so each topic splits into partitions, led by one broker and copied by two. Hashing the key keeps each order’s events in one partition, in order.
6 interview moments
You’re the candidate
The interviewer pushes on the hard parts. Pick an approach and see how it holds up.
Deep dive 1 of 6
Interviewer
“3M records a second come in, and four groups read every one. How does a broker store them?”
Your move. Pick an approach:
Deep dive 2 of 6
Interviewer
“One partition, three replicas in three zones, carrying UPI payments. When may the leader tell the producer “got it”?”
Your move. Pick an approach:
Deep dive 3 of 6
Interviewer
“Four teams read
order-events, and each runs many workers. How do the workers share the work without breaking order?”Your move. Pick an approach:
Deep dive 4 of 6
Interviewer
“A settlement service reads
payment-eventsand writesledger-entries. How do you stop retries and crashes creating duplicates?”Your move. Pick an approach:
Deep dive 5 of 6
Interviewer
“A broker that led 570 partitions dies. Who picks the new leaders, and how fast?”
Your move. Pick an approach:
Deep dive 6 of 6
Interviewer
“Each order’s events must stay in order. One seller’s flash deal brings 30% of all orders. What do you key by?”
Your move. Pick an approach:
Build-up · 6 steps
Follow one ORDER_PAID event through it
The data path is producers, leaders, followers and consumers. The controller quorum and object storage sit beside it, never in front.
Step 1 of 6
Produce
The order service batches per partition for up to 5 ms, compresses, tags each batch with its producer ID and sequence number, and sends it to the leader from cached metadata. Clients over quota are slowed, not rejected.