400k/s
reads on one key, a live IPL score
12 million apps poll every 30 s. One node manages about 100,000 ops a second, and more shards don’t help. They spread keys, not requests.
Give the hot key fewer reads and more owners.
A 1-second in-process cache in each of 200 pods cuts Redis to about 200 reads a second. Split the key into 8 copies on different shards.
A key read 20,000 times a second just expired.
Recomputing takes 200 ms, so 4,000 identical queries hit the database. They slow it down, which lets even more requests miss.
The key expired. Who recomputes?
Bad: Every miss
4,000 identical queries, and a slower database lets in more.
Good: One caller, the rest get stale
A lock or single-flight, plus soft and hard TTLs, so nobody waits.
XFetch lets each read refresh a little early at random, likelier as expiry nears.
The trick
The first refresh starts about 1.7 s early, about 2.7 recomputes instead of 4,000, and no miss.
Stampede, avalanche, penetration. Name them precisely.
One hot key missing; many keys expiring at once, so jitter TTLs; or keys that exist nowhere, so cache the absence behind a Bloom filter.
Invalidation at scale: reach every copy
- One change event; every layer subscribes
- Near caches: broadcast, or a 1–5 s TTL
- Tags: one purge clears every dependent page
- Namespace bump: one INCR invalidates millions
Remember
Spread a hot key with an L1 and copies. Let one caller recompute, serve stale and refresh early. Invalidate with one event per change, fanned out to every copy, and jitter every TTL.