Numbers to know

Numbers to know are the orders of magnitude that let you size a design without running a benchmark: how long a memory read, a disk read or a round trip takes, how much one disk, database, cache or managed partition can sustain, and how big everyday objects are. A number is only useful if you know where it came from and which decision it settles, so every figure here carries a label: measured, documented, derived or assumption.

Picture it first: stretch a nanosecond into a second

Slow a computer down a billion times, so that 1 ns lasts 1 s. Reading the CPU’s L1 cache is picking up the pen on your desk: a second. Main memory is a walk to the next room: under two minutes. A random read from a fast SSD is a trip across town: about four hours. A round trip to another server in the same data centre takes nearly a week, a disk seek a season, and one round trip from Pune to Frankfurt four and a half years.

The exact durations don’t matter. What matters is that each rung is a different kind of errand. Nobody crosses town to fetch a pen, and no design should fetch over the network, inside a loop, what it could read from memory.

Map it back: the desk is the CPU cache, the next room is RAM, the trip across town is local storage, the neighbouring building is another machine, and a letter abroad is a cross-region call. Good designs keep the hot path on the low rungs.

Four kinds of number

Hardware changes every year, so a number you can’t trace is a number you can’t defend. Every figure on this page wears one of four labels:

• Measured: someone ran it and published the setup, such as Kafka’s 2014 benchmark or Azure’s monthly round-trip medians.
• Documented: a vendor publishes it as a specification or quota, such as S3’s per-prefix rates or DynamoDB’s per-partition capacity. The most dependable numbers you can quote, but check them, because vendors raise them.
• Derived: arithmetic from physics or a specification, shown in full, such as the speed of light in fibre or a memory channel’s bandwidth.
• Assumption: a rule of thumb that depends on your code and hardware, such as requests per app server. Say these out loud as assumptions.

Interviewers don’t expect you to know a drive’s exact IOPS. They expect the right power of ten and an honest answer to “where does that come from?”

The latency ladder

Two classic lists anchor this ladder. Peter Norvig’s “Teach Yourself Programming in Ten Years” gives approximate timings for a typical PC from 2002, and Colin Scott’s interactive version projects the classic list forward year by year from hardware trends. Its flat lines, such as memory latency and data-centre round trips, deserve more trust than its rising curves, as the going-deeper section shows.

Round trips and the speed of light

Light in glass fibre travels at about two-thirds of its vacuum speed, roughly 200,000 km a second, so every 100 km of distance costs 0.5 ms each way: 1 ms of round trip per 100 km. That is a floor. Real paths add longer cable routes, routers and queues.

How much do they add? Microsoft publishes median round trips between Azure regions, measured continuously by probes on its backbone; the current table covers the 30 days to 30 July 2026. Set beside the straight-line floor:

The working rule: measured round trip ≈ 1.5–2× the straight-line floor, more on short hops where fixed overheads dominate. Inside one region, AWS’s fault-isolation whitepaper places availability zones up to about 100 km apart, “close enough to use synchronous replication with single-digit millisecond latency”.

What these numbers decide:

• Synchronous replication. Across zones (1–2 ms) it is cheap. Across India (20 ms) it suits modest write rates. To Europe (140 ms) a single sequential writer manages about 1,000 ÷ 140 ≈ 7 commits a second.
• Consensus across regions. Google’s SRE workbook budgets about 25 ms per Paxos operation between datacentres a few hundred kilometres apart, so one sequential process manages 40 operations a second, and 10,000 a second needs at least 250 in parallel.
• User-facing calls. A fresh HTTPS connection costs three round trips (TCP, TLS 1.3, the request) before the first byte. A user in Chennai served from Frankfurt (about 150 ms each; Azure measures 151 between those regions) waits about 450 ms before any server work. That is why multi-region designs home users near their data.

Throughput per unit

Throughput numbers describe one unit: one disk, one node, one partition. Your design’s capacity is that number times the number of units, minus headroom. The storage gap matters most, because it decides where data must live.

The numbers come from AWS’s EBS and EC2 storage-optimised instance documentation (400,000 random-read IOPS at 4 KiB for one i4i drive) and the SRE workbook’s estimate that “a common 4 TB HDD might be able to sustain 200 IOPS”. That is a 2,000× gap per device. gp3’s baseline is 3,000 IOPS and 125 MiB/s, provisionable to 80,000 IOPS and 2,000 MiB/s; older guides quote 16,000 and 1,000, which now apply only on Outposts. For what this means inside a database engine, see reads and writes.

Nodes you run

Redis’s docs add a warning that applies to every cache: 4 KB values at 100,000 operations a second need 3.2 Gbit/s, so the network often binds before the CPU (see distributed caching).

Managed-service limits

Sizes of things

Scale anchors

A few published totals keep estimates honest. Fewer, verified anchors beat many remembered ones.

• UPI. RBI’s Payment System Indicators count 2,416,168.86 lakh UPI transactions in FY 2025-26: 2.4 × 10¹¹ ÷ 3.15 × 10⁷ s ≈ 7,700 a second on average. September 2026 alone was 240,683.80 lakh, about 9,300 a second and 23% more than September 2025. A design sized for today’s average is short within a year. Back-of-envelope estimation turns this into a peak.
• Aadhaar. UIDAI’s home page counts 145.32 crore enrolments, about 1.45 × 10⁹. At 1 KB of text per person that is 1.45 TB: one drive. A population-sized table is not automatically big data; request rates and per-event records are what grow.
• Twitter, 2013. About 5,700 tweets a second on average against a one-second record of 143,199: a 25× spike.
• Kafka, 2014. Three commodity machines absorbed 2 million writes a second.

Capacity rules of thumb

Five formulas cover most sizing questions:

• Servers = peak requests/s ÷ per-server requests/s ÷ target utilisation (0.6–0.7), then add spares to survive losing a zone.
• Cache memory = items × (key + value) × overhead (assume 1.5–3× for small values).
• Disk per year = writes/s × bytes × 3.15 × 10⁷ s × copies (usually 3), plus indexes.
• Bandwidth = requests/s × response bytes × 8.
• Partitions = the largest of total rate ÷ per-partition limit, bytes ÷ per-partition size, and the hottest key’s rate ÷ per-partition limit.

Worked example: rider locations for a delivery app

Assume 300,000 riders on shift at the dinner peak, each phone sending a GPS fix every 5 seconds, and 1 million customers watching an order map that refreshes every 10 seconds.

• Rates: 300,000 ÷ 5 = 60,000 writes/s; 10⁶ ÷ 10 = 100,000 reads/s.
• Memory for the latest fix per rider: 300,000 × 32 B × 3 (assumed key and bookkeeping overhead) ≈ 29 MB. Memory decides nothing.
• Cache nodes: 160,000 ops/s ÷ (100,000 × 0.6) ≈ 2.7, so 3 Redis shards plus replicas. Bandwidth is 160,000 × ~100 B ≈ 16 MB/s, about 128 Mbps: trivial. Operations per second size this cluster.
• History stream: 60,000 records/s needs 60 Kinesis shards by record count but only 2 by bytes (60,000 × 32 B ≈ 1.9 MB/s). Pack 20 fixes per record and 3 shards do. On Kafka it is a rounding error.
• A year of history: assume the day averages a third of the peak: 20,000 × 32 B × 86,400 s ≈ 55 GB a day, × 365 ≈ 20 TB, × 3 copies ≈ 60 TB a year. Object storage, not Redis.

The geospatial indexes article covers how to query those positions.

Worked example: live-score polling during a final

Assume 25 million phones on a cricket score screen, polling every 10 seconds for a 2 KB response.

• Rate: 25 × 10⁶ ÷ 10 = 2.5 million requests/s.
• Servers at the origin: 2.5 × 10⁶ ÷ 2,000 ÷ 0.6 ≈ 2,100.
• Egress: 2.5 × 10⁶ × 2 KB = 5 GB/s = 40 Gbps.

Every phone gets the same answer, and it changes a few times a minute. Cache it at the CDN with a 1-second TTL and the origin sees about one request a second per edge cache. With an assumed 300 edge caches, that is 300 requests/s: one server and a spare. The deciding number was not the request rate but the fact that every response was identical.

Worked example: a checkout’s latency budget

App servers run in a Mumbai region with a server-side p99 target of 200 ms. The handler loads 40 cart items, calls a fraud service in another zone, and calls a payment provider whose API is served from Singapore (about 53 ms per round trip, Azure’s Pune–Singapore median).

Counting round trips found 150 ms before anyone profiled a line of code. The fixes are the N+1 query problem, chatty I/O and connection pooling.

Availability and conversions, in four lines

• 99.9% allows about 43 minutes of downtime a month, 99.99% about 4.3 minutes, 99.999% about 26 seconds (availability and the nines).
• A day is 86,400 s ≈ 10⁵; a year ≈ 3 × 10⁷ s; 1 million a day ≈ 12 a second; 1 billion a day ≈ 12,000 a second.
• 2¹⁰ ≈ 10³; 1 Gbps = 125 MB/s; multiply bytes by 8 for links.
• Rounding rules, peak factors and the full conversion table are in back-of-envelope estimation.

Going deeper: when numbers mislead

What changed, and what didn’t

The SRE workbook’s standard machine has 16 cores, 64 GB of RAM and 1 Gbps, with 4 TB disks at 200 IOPS. One storage-optimised EC2 instance (i4i.32xlarge) now lists 128 vCPUs, 1,024 GiB, 75 Gbps and eight NVMe drives with 3.2 million random-read IOPS between them: 16,000× the random reads of one of those disks, 75× the network, 16× the memory.

What didn’t move: memory latency is still about 100 ns, a data-centre round trip still a few hundred microseconds, and Norvig’s 2002 “US to Europe and back” of 150 ms sits beside Azure’s 2026 median of 145 ms from California to the Netherlands. Bandwidth and IOPS got cheap; round trips didn’t. Designs that batch age well; chatty ones don’t.

Models drift; check them against physics

Scott’s page extrapolates trend lines, and some run away. Its disk-seek curve drops below 3 ms by 2020, yet half a rotation of a 7,200 rpm platter alone takes 4.2 ms. Its network curve doubles every two years, which by 2026 implies a commodity network card moving 125 MB/s × 2^11.5 ≈ 360 GB/s, nearly 3 Tbps. When a number comes from a model, test it against a specification or a physical limit before you quote it.

Totals hide the binding constraint

Averages hide tails: AWS designs gp3 volumes to deliver their provisioned performance 99% of the time, so your p99 may land in the other 1%. Totals hide hot keys, as the warning above shows. And local and network-attached flash differ by up to about 100× in latency (tens of microseconds against single-digit milliseconds), which is why “the database is on SSD” is not one number. Database scaling shows how these ceilings set a database’s write limit.

Using numbers in the interview

• Say the label. “I’ll assume about 10,000 small durable writes a second per Postgres primary; I’d confirm with a load test.” A documented limit can be stated flatly; an assumption should sound like one.
• Apply the factor-of-10 test. A number is about right if it is within 2–3× and on the correct side of every threshold it touches. If it sits within 10× of a threshold, say so and name the escape hatch, such as sharding later by user ID.
• Round in the safe direction: capacities down, demands up.
• Prefer documented limits whenever the design touches a managed service; they are the only numbers you can quote without a caveat.
• Cross-check by a second route: physics for latency, a published anchor for scale, dividing back to one user for totals.

Cheat sheet

How to say it in the interview

“Let me put numbers on this. Inside the region each call costs about half a millisecond of round trip, so four calls are 2 ms; the real cost is a database read that misses memory, which is single-digit milliseconds on a network volume. For capacity I’ll assume 2,000 requests a second per app server and 10,000 small writes a second per Postgres primary; I’d load-test both. Peak writes are 30,000, so one primary won’t do; I’ll shard by user. If we used DynamoDB instead, its documented limit is 1,000 writes a second per partition, so the hottest key matters more than the total. Users are in India, so the region is Mumbai; Europe would add about 140 ms per round trip.”

Where does 10,000 writes a second come from? It’s an assumption built from the commit path. If every commit waits for its own 1 ms log flush, the log caps out near 1,000 a second; group commit lets one flush cover about ten commits, giving roughly 10,000. Wide rows, many indexes or a working set bigger than RAM lower it, so I’d load-test before committing to a shard count.

How many requests can one server handle? Anywhere from hundreds to tens of thousands. Twitter’s Rails hosts managed 200–300 a second in 2013, and its JVM services 10–20K on similar hardware. I plan on 2,000 and run servers at 60–70% busy.

Why not serve Indian users from a European region? Physics. Pune to Frankfurt is about 6,700 km, so at least 67 ms per round trip, and Azure measures 140. A cold HTTPS request needs three round trips, nearly half a second before the server starts work.

Hasn’t new hardware made these numbers obsolete? Bandwidth and IOPS have grown by orders of magnitude: one NVMe drive does 2,000× a hard disk’s random reads. Memory latency, data-centre round trips and the speed of light haven’t moved, so designs bound by round trips gain nothing from new hardware.

What if your number is off by 10×? I check whether it crosses a threshold. If 2,000 peak writes became 20,000, one primary would no longer do and I’d shard; if a 30 MB cache became 300 MB, nothing changes.

Where to go next

• Back-of-envelope estimation: the method that turns these numbers into rates, storage and server counts.
• Latency vs throughput vs bandwidth: percentiles, Little’s law and why you plan for 60–70% busy.
• Availability and the nines: the downtime table and how dependencies multiply.
• Caching 101 and distributed caching: what a 100 ns hit and a 0.5 ms hop buy you.
• Databases for system design interviews: which store to pick once you know your rates.
• Database sharding: turning “10,000 writes a second per primary” into a partitioning scheme.
• Multi-region and active-active: designing around 50–200 ms between regions.
• The delivery framework: when in the round to bring numbers out.

References and further reading

1. Teach Yourself Programming in Ten Years (Peter Norvig)norvig.comThe 2002 table of approximate timings: 100 ns memory fetch, 25 ns mutex, 8 ms disk seek, 20 ms to read 1 MB from disk, 150 ms US–Europe round trip.
2. Latency Numbers Every Programmer Should Know, by year (Colin Scott)colin-scott.github.ioA trend model of the classic list: cache hits in cycles at 3 GHz, 16 µs SSD reads, 500 µs data-centre round trips, and curves that overshoot when extrapolated.
3. Non-Abstract Large System Design (Google SRE Workbook, chapter 12)sre.google200 IOPS per 4 TB hard disk, a 16-core, 64 GB, 1 Gbps standard machine, and about 25 ms per cross-datacentre Paxos operation.
4. Azure network round-trip latency statistics (Microsoft Learn)learn.microsoft.comMeasured P50 round trips between Azure regions for the 30 days to 30 July 2026, used for the India, Singapore, Europe and US routes.
5. AWS Fault Isolation Boundaries: Availability Zones (AWS whitepaper)docs.aws.amazon.comZones up to about 100 km apart, close enough for synchronous replication with single-digit millisecond latency.
6. Amazon EBS General Purpose SSD volumesdocs.aws.amazon.comgp3: 3,000 IOPS and 125 MiB/s baseline, up to 80,000 IOPS and 2,000 MiB/s, single-digit ms latency; io2 Block Express under 500 µs on average.
7. Amazon EC2 storage optimized instance specificationsdocs.aws.amazon.comRandom-read IOPS per local NVMe drive (400,000 for one 3,750 GB i4i drive) and a 128-vCPU, 1,024 GiB, 75 Gbps instance.
8. Best practices design patterns: optimizing Amazon S3 performancedocs.aws.amazon.com3,500 write and 5,500 read requests per second per prefix, 100–200 ms small-object latency and up to 100 Gb/s per instance.
9. Redis benchmark (redis-benchmark) documentationredis.io180,000 SETs a second with 50 clients, 1.5–1.8 million with 16-command pipelines, and why the network often limits Redis before the CPU.
10. Benchmarking Apache Kafka: 2 Million Writes Per Second (On Three Cheap Machines) (Jay Kreps, LinkedIn, 2014)engineering.linkedin.com2,024,032 100-byte records a second on three brokers with 3× replication, the hardware used, and 2 ms median end-to-end latency.