Layer 3 · Go deeper · 7 min
Go deeper
Size it, build it, then make the calls an interviewer will push you on.
Back of the envelope
Size it up
Tap through each step of the arithmetic.
How many writes a second can a three-member etcd cluster take?
Build-up · 6 steps
Elect a leader that can’t double-dispatch
Election picks one leader; a fencing token stops the old one, which may still be running. Followed in ZooKeeper, with etcd’s equivalents in the captions.
Step 1 of 6
Everyone takes a number
Each scheduler creates an ephemeral sequential node under /election, and the server appends a 10-digit counter. The lowest number leads. In etcd, each campaigns with a leased key, and the lowest create revision leads.
4 interview moments
You’re the candidate
The interviewer pushes on the hard parts. Pick an approach and see how it holds up.
Deep dive 1 of 4
Interviewer
“Your job scheduler runs three copies, and only one may dispatch jobs. How do you choose it and keep it safe?”
Your move. Pick an approach:
Deep dive 2 of 4
Interviewer
“Your etcd cluster must keep accepting writes when one availability zone fails. How many voters, and where?”
Your move. Pick an approach:
Deep dive 3 of 4
Interviewer
“Feature flags live in one etcd key, watched by 20,000 service instances. How do changes reach them all?”
Your move. Pick an approach:
Deep dive 4 of 4
Interviewer
“A nightly job rewrites rows in one Postgres database, and two runners must never overlap. Do you add etcd for the lock?”
Your move. Pick an approach: