4
seconds until all 200 threads hang
Say 1,000 requests a second, no timeouts, and a downstream hangs on 5%. That strands 50 threads a second. The service is down, CPU idle.
A read timeout won’t stop a server that trickles
It times the gap between bytes, so a byte every 20 s never trips a 30 s one. Only a total deadline bounds the call.
Who should retry a failed call?
Bad: Every layer, 3 tries each
One tap becomes 3 × 3 × 3 = 27 database calls.
Good: One layer, safe calls only
Idempotent or keyed only, jittered backoff, retries capped at 10%.
Random 502s, worse at quiet times. Nothing in your logs.
The race
Node closes idle connections after 5 s. The ALB keeps them 60 s and reuses a closed one. Set keep-alive to 65 s.
A 302 quietly turns your POST into a GET
A trailing-slash redirect lets the client resend as GET without the body: 200, no order. Never redirect a write; 307 and 308 keep the body.
A 200 with an error inside fools every layer
Caches store the failure, retries skip it, health checks pass and the 5xx alert never fires. Send the real code, with problem details.
Walk one request, hop by hop
- Client: connect timeout, total deadline, one retry layer
- Balancer: keep-alive outlives its idle timeout
- Response: honest status code, explicit caching
- Writes: idempotency keys, If-Match, no redirects
Say it like this
Most HTTP API failures live between hops. Walk the request: deadlines and one retry layer, keep-alive above the balancer’s idle timeout, honest status codes and caching, keys and If-Match on writes.