Two data models: a znode tree, or flat keys
Kilobytes per value. An ephemeral znode dies with its session; etcd keys share a lease and carry revisions for ordering.
Four servers beat three?
Bad: Four: an extra copy
Four need 3 to commit, so they survive just 1 failure.
Good: Three or five
Three survive 1 with fewer machines; five survive 2. Kubernetes recommends five.
Writes are linearizable. Local reads can be stale.
Commit needs a majority on disk, so a follower may lag. ZooKeeper reads locally unless you sync(); etcd checks with the leader by default.
21k→8k
writes/s, from 3 to 13 ZooKeeper servers
2010 paper: each write passes one leader and a majority of fsyncs. Reads rose 87k → 460k, from memory. Scale writes by splitting clusters.
Who leads? The lowest number. Each waiter watches one node.
A leader’s crash deletes its ephemeral node and wakes exactly one successor. In etcd: a leased key, lowest create revision.
A 40-second GC pause: the leader’s session expires and a successor takes over.
The catch
The old leader wakes still believing it leads. Only the resource can stop it, by rejecting an older fencing token.
When to pick it, when not
- Pick: elections, membership, config, fenced locks
- New work: etcd. Where already running: ZooKeeper
- Not: application data, big values or heavy writes
- Not: queues, or what a row lock covers
Say it like this
I’d use etcd for elections, locks and config: three members across zones, linearizable writes that don’t scale. The leader’s create revision fences every write. Never a database or a queue.