6 pods at 90% CPU, target 60%. How many pods?
The HPA computes ceil(6 × 90 ÷ 60) = 9: the same work over 9 pods is 60% each. Measure, compare with a target, change the count.
Steer on what?
Bad: p99 latency
Flat, then explosive near saturation. A slow database raises it on every pod.
Good: Load per pod
Requests, in-flight or CPU per pod: it falls as you add pods. Alert on p99.
Traffic jumps now. When do new pods start serving?
About 90 s when a node has room. About 5 minutes when they wait for a new VM to boot, join and pull images.
71%
of safe capacity, not 90%
Say load grows 2,000 requests/s a minute from 20,000, and new capacity takes 4 minutes. Today’s fleet must carry all 28,000. Headroom = growth × lag.
You schedule extra pods an hour before the 10:00 tatkal rush. At 10:00 they’re gone. Why?
The catch
At 09:45 the metric looks quiet, so target tracking scales back in. Raise the minimum instead.
Out at 60%, in at 45%. Why won’t 3 instances settle?
At 45% one goes; the other two run at 67.5%, so it comes back. Scale-in must sit below 60 × 2 ÷ 3 = 40%.
What autoscaling can’t fix
- A spike that peaks faster than its own lag
- A database at its connection limit
- A retry storm or a CPU-burning bug
- Your max, your quota, a region out of machines
The interview line
Steer on load per instance, keep headroom of growth × lag, pre-scale known peaks, and cap the max at what your database survives.