Two master nodes for safety?
Bad: Yes, one spare
Lose one: one of two is no majority, so nothing is decided.
Good: No: three, by majority
Any two decide; a split can’t give both sides a majority.
Which shard holds P-48213? Hash it, modulo 4.
The primary count sits in the formula, so it’s fixed: 4 → 5 primaries would move 80% of documents. Reindex behind an alias instead.
25 GB
per primary: Bazaar’s four shards
Elastic’s guidance is 10–50 GB a shard. One 100 GB primary takes about 43 minutes to rebuild; 40 tiny ones make every search run 40 shard tasks.
Number it, copy it, then say “saved”
Mid-sale, with two replicas: the primary stamps sequence number 1,042 in term 3, sends it to both in-sync replicas at once, and acks after both apply it.
Quorum stores need three copies to survive a failure. Elasticsearch needs two.
The trick
The masters hold the quorum. A copy that misses a write leaves the in-sync set before the ack, so it’s never promoted.
Each shard sends its top 3 brands. The “winner” is really third.
Rangrez misses shard 0’s cut: reported 275, truly 370. Keep shard_size generous and read doc_count_error_upper_bound.
Limits that bite a healthy-looking cluster
- Free-form seller fields hit the 1,000-field limit
- At 95% disk, a node’s indices go read-only
- Heap ≤ half of RAM, under ≈ 26 GB
- Replicas copy a bad delete; keep snapshots
Remember
Three dedicated masters elect one by majority. Documents hash to a fixed count of 10–50 GB shards. Each write is numbered and waits for every in-sync copy. Facet counts are top-N per shard.