25M
rows counted to rank a median player
A B-tree can’t count the players above you. At 10M entries a second, that’s 2.5 s per view, 30,000 cores at peak.
How does a skip list know your rank without counting?
Each forward pointer stores a span, the positions it jumps. The search for 77 sums 4 + 2 + 1: rank 7. About 52 steps for anyone.
Two players hit 48,250. Who ranks first?
Bad: Points × 10¹³ + ms
≈ 4.8 × 10¹⁷, past 2⁵³. Silently rounded to a multiple of 64.
Good: Points × 2²⁵ + inverted time
Earlier wins ties, and the maximum, 2⁵³ − 1, is exact.
50k/s
sorted-set writes one Redis primary takes
Redis Cluster never splits a key, so a board is one thread. The 200k-a-second burst is 4× over. Shard by player into 8 keys.
Sharding makes every exact rank 8 calls. Must the 12.4-millionth player see an exact rank?
The trick
Nobody can tell 12,400,000th from 12,400,350th. Keep the top 10,000 exact; below, a histogram in memory says “top 24.8%”.
2M fans watch one top 10. Redis is read once a second.
A publisher diffs the top 10 each second and sends one versioned delta. 40 gateways subscribe once, each pushing to about 50k viewers.
A failover can erase first place. Log first:
- Append to Kafka (acks=all), reply 202
- Apply to the database only if new
- ZADD absolute scores, never increments
- After a failover, rewind and replay
Remember
Log every score first, then rank in sorted sets, not SQL. Break ties inside 53 bits, shard by player past one thread, approximate the deep tail, and push one delta a second.