Concurrency is structure. Parallelism is execution.
Concurrency deals with lots of things at once, tasks in progress and taking turns. Parallelism does lots at once, on several cores at the same instant.
20×
more requests per core, by overlapping waits
Each request needs 5 ms of CPU and 95 ms of waiting. One at a time, a core serves 10 a second; overlapped, 200.
More threads make it faster?
Bad: Add threads
Past the core count, CPU work only slows, from switching and contention.
Good: Find the bound
Waiting needs concurrency; computing needs cores. Extra concurrency on CPU work only queues.
CPU-bound work doesn’t care how many are in flight.
Thumbnails need 200 ms of pure CPU, so 8 cores top out at 40 a second. Put 1,000 in flight and each waits 25 seconds.
50,000 mostly idle WebSockets, one OS thread each: about 50 GB of reserved stack.
The trick
Eight event loops can hold the same sockets, and 50,000 goroutines start at about 100 MB.
One slow callback stalls every client on its loop.
Parsing a 50 MB JSON document took 1.3 s in Node.js. On the event loop, every other client waits 1.3 s too. Move heavy work to workers.
Concurrency’s price: races on shared state
- Lost update: the second write erases the first
- Fix: an atomic or conditional update
- Or give each wallet a single writer
- Deadlock? Always lock in a fixed order
Remember
Waiting needs concurrency; computing needs parallelism. Size in-flight work with Little’s law, cap it at the narrowest stage, and give hot shared state one writer or an atomic update.