The kill test: kill one instance at noon.
Send its next request to a sibling. If no user can tell, it’s stateless. If someone is logged out, disconnected or loses progress, it’s stateful.
Does stateless mean no state?
Bad: Stateless means no database
Throw the state away and scale freely.
Good: The state moved
Into a database, a cache or the client. The system is never stateless.
2,000×
slower than memory, one network round trip
Memory is about 100 ns; a round trip on a 1 Gbit/s network about 200 µs. Against a 200 ms page budget, a session lookup is still noise.
2M sessions: 4 GB of memory, 200,000 reads a second.
Each 2 KB session is read every 10 seconds. The memory fits one node; the reads want three or four shards. The store becomes real infrastructure.
Put the session in a signed JWT: every instance checks it locally, with no lookup.
The trick
Until logout. A token stays valid until exp, unless every instance checks a denylist, a shared lookup again.
Stateless where you can, stateful where you must.
Gateways own sockets and little else. A registry and pub/sub route messages, logic stays stateless, and data lives in stores built to hold it.
Hidden state breaks “stateless” apps quietly
- A rate limiter counting in local memory
- An upload saved to one instance’s disk
- A cron job that runs once per instance
Remember
Keep the request tier stateless, so any instance serves any request for a sub-millisecond lookup. Make a few tiers deliberately stateful, owned by key, replicated or checkpointed, and drained on deploys.