A dependency hangs on 2% of calls
Bad: A 30-second timeout
Stuck calls pin 600 threads; the pool has 200. Every request fails.
Good: A 300 ms timeout
They pin just 6 threads. 98% of users notice nothing.
Pick the timeout from measured latency
- Choose a false-timeout rate, say 0.1%
- Use that percentile of its latency: the p99.9
- Pad for distant callers and tight distributions
- One per operation: lookups aren’t bulk exports
One deadline for the whole request
Set it at the edge; each hop gets what’s left. A hop that can’t finish in time fails fast, not after its caller has gone.
Your payment call timed out. Safe to retry?
The catch
A timeout means the answer didn’t arrive, not that the money didn’t move. Retry with the same idempotency key; the server replays the first result.
243
database attempts from one tap
Five layers that each try three times multiply, peaking just when the database is weakest. Retry at one layer only: the one above the failure.
1,000 clients fail together. When do they come back?
Plain backoff returns them in waves that swamp the server again. Full jitter, a random sleep from zero to the backoff, spreads them out.
Full outage: every first attempt fails
Bad: Three tries per request
Nearly 3× the load on a dependency that’s already down.
Good: A shared retry budget
Retry only while retries are under 10% of requests: about 1.1×.
The interview line
Time out every call near its p99.9, pass one deadline down, and retry transient errors at one layer, with jitter, inside a budget.