6.7/s
30-second exports fill 200 threads
Little’s law: slots in use = rate × time held. Each export holds its thread for 30 s, so 200 ÷ 30 ≈ 6.7 a second leaves login nowhere to run.
A background thread is enough?
Bad: Start a thread, return 202
Fast response, but the CPU still burns on the same pod, unlimited.
Good: Another tier
Computation needs other machines; async doesn’t help.
In Node.js, one big parse blocks everyone
All your JavaScript runs on one thread. A 50 MB catalogue takes about 1.3 s to parse, and meanwhile nobody is served, not even the health check.
Validate, enqueue, answer 202
The handler spends about a millisecond and returns a status URL. Workers resize the photo. A burst of 2,000 uploads becomes a backlog, not 2,000 requests fighting login.
Login is slow, so you double the web pods. Fixed?
The trap
Each new pod gets its share of uploads and fills up too. Scaling the front end spreads heavy work; it doesn’t isolate it.
Keep the request path short
- Validate, enqueue, return 202 + status URL
- Presigned uploads straight to object storage
- Scale workers on backlog; watch message age
- Bulkhead heavy routes that must stay synchronous
12
worker vCPUs for 20 uploads a second
20 × 0.4 s keeps 8 vCPUs busy, so about 12 at a 70% target. Cap their concurrency, or the overload just moves to the database.
Remember
The request path does only what the user waits for. Validate, enqueue, return 202, and let workers scaled on backlog do the heavy work: a burst becomes a queue, not a slow login.