Layer 3 · Go deeper · 8 min
Go deeper
Size it, build it, then make the calls an interviewer will push you on.
Back of the envelope
Size it up
Tap through each step of the arithmetic.
How much disk do 7 days of events take?
Build-up · 6 steps
Follow one write from produce to commit
One order event, keyed by order_id, on a topic with replication factor 3. Watch when the producer hears ok, and when a consumer may read it.
Step 1 of 6
Hash the key, find the leader
The producer computes murmur2(key) % partitions, so every event of this order lands in P1. It sends the batch to P1’s leader, which takes all of its writes.
4 interview moments
You’re the candidate
The interviewer pushes on the hard parts. Pick an approach and see how it holds up.
Deep dive 1 of 4
Interviewer
“Payment events must never be lost once Kafka acknowledges them. How do you set up the topic and the producer?”
Your move. Pick an approach:
Deep dive 2 of 4
Interviewer
“Each order’s state changes must be processed in sequence, and volume is high. How do you guarantee that?”
Your move. Pick an approach:
Deep dive 3 of 4
Interviewer
“In a UPI-style payments stream keyed by merchant, one giant merchant swamps its partition. What do you do?”
Your move. Pick an approach:
Deep dive 4 of 4
Interviewer
“One malformed record fails every time, and lag on its partition keeps climbing. What do you do?”
Your move. Pick an approach:
Build-up · 5 steps
Read, process, write, exactly once
A processor reads payments and writes ledger entries. Exactly-once holds for Kafka in, process, Kafka out. Watch where it stops.
Step 1 of 5
Read a batch
The processor polls a batch from the payments topic. Its producer has a transactional.id, so initTransactions fences any older instance with the same id, and a returning zombie can’t commit.