Content delivery network (CDN)

A content delivery network (CDN) is a fleet of caching reverse proxies spread across hundreds of sites close to users. Each request lands on a nearby edge server, which answers from its cache when it can and fetches from your servers (the origin) when it can’t. You get shorter round trips, a fraction of the origin traffic, and a wide front door that soaks up flash crowds and attacks.

Picture it first: the dairy and the milk booths

A dairy on the edge of the city bottles all the milk. Rather than have every household queue at its gate, it stocks a booth every few streets: you walk two minutes, and the dairy sees one van per booth instead of a thousand customers.

Booths keep what the street buys daily and order rarer items from a zonal depot. If fifty booths ask for paneer at once, the depot places one order with the dairy.

Every packet has a use-by date; after it, the booth checks with the depot before selling. And when the dairy changes a recipe, it doesn’t chase old packets around the city. It prints a new label.

Map it back: the dairy is your origin, booths are edge servers in points of presence (PoPs), the depot is a regional tier or origin shield, the use-by date is max-age, checking with the depot is revalidation, and the new label is a fingerprinted URL.

What a CDN buys you

Three things; name all three, with numbers.

Shorter round trips

Light in fibre covers about 200,000 km a second, so each 100 km adds roughly 1 ms of round trip (latency vs throughput vs bandwidth). Chennai to Ashburn, Virginia, home of AWS’s us-east-1, is about 13,800 km in a straight line: a floor of 138 ms. Cables don’t run straight, so take 220 ms as an illustrative real figure.

A fresh HTTPS connection pays one round trip for the TCP handshake, one for TLS 1.3 and one for the request: 3 × 220 = 660 ms before the first byte. With an edge in Chennai about 10 ms away, a hit costs 30 ms. Even a miss is faster: the handshakes stay local and the edge forwards the request over a connection to the origin it already holds open, so the long haul is paid once: 30 + 220 = 250 ms.

Origin offload

The origin only sees misses: origin requests/s = (1 − hit ratio) × requests/s. At 100,000 requests a second, a 95% hit ratio sends 5,000 a second to the origin and 99% sends 1,000. Four points of hit ratio, a 5× difference in origin capacity. Bytes, which are the bill, follow the same rule: serve 400 TB a month at a 95% byte hit ratio and 20 TB leaves your origin; at 99%, 4 TB. Track both ratios, since a few big videos can score a high byte hit ratio while thousands of small API responses miss (Caching 101 has the general maths).

A wider front door

A CDN has far more bandwidth than your origin, spread over hundreds of sites. A flash crowd (board-exam results, the tatkal window, a viral reel) lands on the edges, and if the content is cacheable the origin barely notices. The spread absorbs DDoS floods too: 1 Tbps split evenly over 300 PoPs is about 3.3 Gbps each, which a PoP can filter, while a 10 Gbps origin link simply fills. Real floods split unevenly, but the principle holds.

What you pay: another company sits on every request’s path, there are two layers of caches to reason about, and wrong headers serve stale data or, worse, someone else’s.

How a request finds the nearest edge

DNS-based steering. You point img.shop.in at the CDN with a CNAME. The CDN’s authoritative DNS answers each lookup with the unicast IP of a PoP chosen for that client: near, healthy and not overloaded. CloudFront steers this way, typically to the PoP nearest in latency. Akamai groups clients into “map units” (an IP prefix plus a traffic class) and matches them to server clusters with a variant of the stable-marriage algorithm, balancing performance, load and cost.

The weakness: DNS sees the resolver, not the user, so a user in Pune whose resolver sits in Singapore may be sent to Singapore. EDNS Client Subnet (ECS, RFC 7871) lets the resolver pass on a truncated client prefix; the RFC encourages 24 bits for IPv4 and 56 for IPv6, for privacy. The other weakness is the TTL: an answer is cached for its lifetime, so a failed PoP keeps receiving traffic until resolvers ask again (DNS).

Anycast. Every PoP announces the same IP prefix over BGP, and routers deliver each packet to the closest announcement in routing terms. A failed PoP withdraws its route and traffic flows to the next nearest with no DNS change, and attack traffic spreads across every PoP (Cloudflare on anycast). The cost is control: you can’t easily tell the internet “Mumbai is full, send some users to Chennai”, routing distance isn’t latency, and a route change mid-connection can move a TCP flow to a PoP that has never seen it, so operators tune announcements to stop routes flapping.

Video services add a third method: the API hands the player URLs naming specific servers, as Netflix’s Open Connect does using each appliance’s reported files, health and routes.

Inside the CDN: PoPs, tiers and the origin shield

A PoP is a site with racks of edge servers, well connected to local ISPs. Inside it, each URL is hashed to one server, so the PoP stores one copy per object rather than one per machine; Akamai uses consistent hashing for this. On a miss, an Akamai edge server asks its cluster peers, then a “parent” cluster, and only then the origin. That parent idea is now standard, as tiered caching:

• Edges (lower tier): many small caches near users that keep what is locally popular and evict the rest, by an LRU you usually can’t tune (cache eviction policies).
• Regional tier: fewer, bigger caches. CloudFront’s regional edge caches keep objects that have fallen out of individual PoPs: product photos, older videos, a news story that suddenly trends again.
• Origin shield: one designated cache in front of the origin. With CloudFront Origin Shield, requests from every regional cache pass through one Region you choose, the one closest to your origin, and concurrent misses are merged, “resulting in as few as one request going to your origin”.

Cloudflare’s Tiered Cache makes the same split, and only upper tiers may contact the origin; its generic global topology picks the upper tier by a consistent hash of the cache key, so every edge in a region asks the same site for a given object. Within one site, request collapsing merges simultaneous misses into one upstream fetch (hot keys, cache stampede and invalidation covers its traps).

You gain a higher hit ratio on the long tail and an origin that serves a handful of connections instead of hundreds. You pay an extra hop on every miss, a separate charge for the shield, and a concentration point: if the shield’s Region struggles, every miss feels it. Tiers only help cacheable content; CloudFront sends dynamic requests and POST, PUT and DELETE straight from the PoP to the origin.

Cache keys: what counts as the same object

The cache key decides whether two requests share one stored copy. RFC 9111 starts from the URL and method, a Vary response header adds named request headers, and CDNs let you trim or extend the result. CloudFront’s default key is just the distribution’s domain plus the URL path; query strings, headers and cookies stay out unless your cache policy adds them (understand the cache key).

Put too much in the key and the hit ratio collapses. CloudFront’s docs show Accept-Language arriving as en-US,en, en,en-US, en-US, en and en-US, four copies of one English page; User-Agent has thousands of variants; tracking parameters and per-user cookies can make almost every request unique. Allowlist the query parameters that change the bytes, sort them, normalise Accept-Encoding to the two or three encodings you serve, and put language in the path (/en-IN/) rather than a header.

Too little in the key is worse: if the response depends on something you dropped, the first visitor’s version goes to everyone.

Two attacks live here. In cache poisoning, the origin reflects an unkeyed input, say an X-Forwarded-Host header copied into a script URL, so one crafted request stores a malicious page under the normal key for every visitor (Kettle, 2018). In cache deception, a victim is lured to /account/orders/x.css, a lenient origin returns the orders page, and an extension-based cache stores it. Defences: key on every input that changes the response, strip unused headers at the edge, and return 404 for unknown paths.

The HTTP headers the edge obeys

RFC 9111 calls a CDN a shared cache and the browser a private cache. A shared cache takes a response’s freshness lifetime from the first of these present: s-maxage, max-age, Expires minus Date, or a heuristic, for which the RFC suggests no more than a fraction, typically 10%, of the time since Last-Modified. A file last changed 30 days ago may be served as fresh for 3 days. Heuristics apply to responses marked public and to status codes RFC 9110 calls heuristically cacheable, including 200, 301 and 404, so send explicit headers on everything.

no-cache means “check before using” and still saves bandwidth through 304s; only no-store means “don’t keep it”. A common split is max-age=60, s-maxage=86400: the browser keeps the page for a minute and the CDN for a day, because you can purge a CDN but not a phone.

Age. A cache serving a stored copy adds an Age header: seconds since the origin generated or last validated it. Downstream caches subtract it, so an object with max-age=600 and Age: 500 is fresh in the browser for only 100 more seconds.

Validators and 304. A stale copy needn’t be downloaded again. The cache sends a conditional request with the copy’s ETag in If-None-Match (or its Last-Modified date in If-Modified-Since; If-None-Match wins if both are sent). If nothing changed, the origin answers 304 Not Modified: no body, just headers such as Cache-Control, ETag and Date, and the copy is fresh again. A 304 saves bytes and origin work, not the round trip.

The stale windows stop one expiry turning into an origin stampede; hot keys, cache stampede and invalidation explains RFC 5861’s rules.

Changing cached content: versions and purges

Copies everywhere are the point of a CDN, and the problem when content changes. Four tools, best first.

Fingerprinted URLs. Put a hash of the content in the file name, such as app.3f9a1c.js, and serve it with Cache-Control: max-age=31536000, immutable, the example RFC 8246 itself gives (31,536,000 seconds is 365 days). A new build gets a new name and the old file never changes, so no cache anywhere can serve a wrong version. Only the HTML that names the files needs a short TTL or no-cache. This is the only approach that also works for browser caches, which no purge can reach.

Purge by URL. Ask the CDN to drop /products/42. Purges take seconds to minutes to reach every PoP and cost money at scale: on CloudFront the first 1,000 invalidation paths a month are free and the rest are billed, though a wildcard like /images/* counts as one path (invalidation pricing).

Purge by tag. The origin labels each response with keys such as product-42 (a Surrogate-Key header on Fastly; CloudFront now supports tag invalidation too), and one call clears every page carrying that tag: the product page, three category pages and a search result.

Soft purge. Mark objects stale instead of deleting them, so edges serve the old copy while one request revalidates; hot keys, cache stampede and invalidation covers tags and soft purges in depth.

Pull, push and dynamic content

Pull is the default: the edge fetches on the first miss and keeps the object as long as headers and eviction allow. Push puts content on the CDN before anyone asks, by uploading it or pre-warming URLs, so the first viewer gets a hit.

Dynamic acceleration. A payment-status call or a personalised feed can’t be cached, yet a CDN still speeds it up. TLS ends at the edge, so handshakes cross 10 ms instead of 220; the edge keeps warm connections to the origin, skipping handshakes and TCP slow start; and traffic can ride the CDN’s own network. Akamai measured that its routing overlay speeds up downloads in every region, more so for clients far from the origin, and acts as “catastrophe insurance” when a cable cut breaks the direct path. For shared responses such as a live score, micro-caching for 1–5 seconds adds offload.

Edge compute runs your code in the PoP. CloudFront Functions run JavaScript on viewer requests and responses in under a millisecond, with 10 KB of code and no network access: enough to normalise cache keys, redirect or check a JWT. Lambda@Edge allows up to 30 seconds, network calls and origin-side hooks; Cloudflare Workers fill a similar role. Anything that needs your database belongs at the origin.

Video at the edge

Video suits a CDN perfectly: huge, read-mostly and watched by many people. HLS and DASH cut each rendition into segments a few seconds long, each its own URL, listed in a manifest, so a two-hour film becomes thousands of small files that never change. Players also request byte ranges, and RFC 9111 lets a cache store partial (206) responses and join ranges of the same version.

Segments, live or on demand, never change once published: cache them for a long time, along with on-demand manifests. A live manifest changes with every new segment, so give it a TTL shorter than one segment, a second or two, and let request collapsing absorb the rush.

Live sport is the hard case. In a cricket final, viewers pour in at the toss and again in the last overs, all wanting the same segment within the same few seconds. Per-PoP collapsing, a regional tier and an origin shield are mandatory; AWS names just-in-time packaging for live streams as an Origin Shield use case. Spreading an event over several CDNs multiplies origin fetches unless they share a shield.

Netflix goes further with Open Connect, its own CDN, begun in 2011: appliances in internet exchanges and, free of charge, inside qualifying ISPs, filled during off-peak windows from predictions of what members will watch. Push, not pull. Netflix says this directed caching reduces demand on upstream capacity “by several orders of magnitude” compared with a demand-driven CDN (Open Connect overview). See Design Netflix for that design and Design YouTube for user uploads, where a pull CDN fits better.

Security at the edge

• TLS termination. The edge holds a certificate for your domain, decrypts traffic and opens a separate TLS connection to the origin. The CDN can read your traffic; that is part of the trust you buy (HTTP and HTTPS).
• WAF, bot rules and rate limits stop injection attempts, credential stuffing and scalping bots before the origin (rate limiting); spare capacity and anycast absorb DDoS floods.
• Signed URLs and cookies keep paid content cacheable: your app signs a URL with an expiry, and the edge checks it without calling you. CloudFront suggests signed URLs for single files and signed cookies when a user needs many, such as every segment of an HLS video.
• Origin protection. All this is wasted if attackers can skip the CDN. For an S3 origin, CloudFront’s origin access control lets the bucket policy admit only your distribution; for servers, accept only the CDN’s address ranges or a secret header the edge adds. Then even a cache-busting flood, a random query string per request, must pass edge rate limits first.

Going deeper: failure modes and costs

The origin stampede after a purge or a cold start

Purge everything and every edge misses at once; a new PoP or a second CDN starts empty. Without tiers, each popular object becomes 300 origin fetches. Purge by tag rather than /*, prefer soft purges, keep tiered caching and the shield on, and pre-warm before a big launch.

Cached errors

Errors are cacheable too (negative caching). RFC 9110 lists 404 as heuristically cacheable; without headers, Cloudflare caches a 404 for 3 minutes, and CloudFront caches errors for an “error caching minimum TTL” of 10 seconds by default (CloudFront error handling). A classic outage: HTML naming app.8b2e47.js goes live seconds before the file, the first visitor’s 404 is cached, and that PoP serves a broken page until it expires. Upload assets before HTML; keep error TTLs short.

Serving stale while the origin is down

The useful flip side: CloudFront keeps serving an expired object for the error-caching TTL when the origin returns a 5xx or times out, and stale-if-error makes that promise explicit. must-revalidate forbids it, so keep it for data where stale is dangerous.

Hit-ratio erosion

The hit ratio drifts down quietly as someone adds a cookie or query parameter to the key, Vary: User-Agent, or Set-Cookie on static files. Edges also evict rarely requested files whatever their max-age: a one-year TTL promises freshness, not retention. Chart the hit ratio per path and alert on drops.

The CDN itself fails

A CDN is a shared dependency for thousands of sites, a textbook single point of failure. On 8 June 2021, one Fastly customer’s valid configuration change triggered a bug introduced by a 12 May software deployment, and 85% of Fastly’s network returned errors. Fastly detected it within a minute, and 95% of the network was operating normally within 49 minutes.

Multi-CDN buys resilience and pricing leverage: DNS or a steering service splits traffic between two providers and fails over when one breaks. You pay with two sets of configuration, certificates, purges and logs to keep in step, a lower hit ratio at each, and more origin fetches unless a shared shield sits in front. A cheaper middle ground is one CDN plus a tested runbook to point DNS at the origin, which works only if the origin can take the load.

Cheat sheet

How to say it in the interview

“Static files and media go through a CDN. Users reach a nearby PoP by DNS or anycast, so handshakes cost about 10 ms instead of a couple of hundred. Bundles and images get content-hashed names and a one-year immutable TTL; HTML and shared API responses get a short s-maxage with stale-while-revalidate; anything per-user is private. Edges sit behind a regional tier and an origin shield, so a new object costs the origin one fetch, and at 99% hits it sees 1% of requests. Content changes ship as new names or tag purges. Uncacheable APIs still go through the CDN for TLS termination, warm origin connections and the WAF, and the origin only accepts CDN traffic.”

How do you ship a new JavaScript bundle without users getting a mix of old and new? Give it a new content-hashed name, upload it first, then publish HTML that references it. The HTML is no-cache or short-lived; open pages keep using the old file, which still exists. Nothing needs purging.

Your origin falls over every time you purge. Why, and what do you change? A hard purge empties every PoP at once and each misses independently. Purge by tag instead of everything, use soft purges so edges serve stale while one request refetches, and turn on tiered caching and an origin shield so misses collapse to one fetch per object.

Can a CDN help an API whose responses can’t be cached? Yes. TLS terminates near the user, saving two long round trips per new connection; the edge reuses warm connections and good routes to the origin; and the WAF, rate limits and DDoS absorption still apply. Shared responses that tolerate a second of staleness can also be micro-cached.

How would you stream a cricket final to millions of phones? Segments of a few seconds cached long, a live manifest cached for about a second, request collapsing at each PoP, a regional tier and an origin shield in front of the packager, CDN capacity booked well before the match, signed cookies for paying viewers, and adaptive bitrate so congested viewers drop quality instead of stalling.

Where to go next

• Caching 101: hit-ratio maths and the cache hierarchy from browser to disk.
• Hot keys, cache stampede and invalidation: request collapsing, stale-while-revalidate, surrogate keys and soft purges.
• Cache eviction policies: why an edge evicts a file long before its TTL ends.
• Domain Name System (DNS): resolvers, TTLs and CNAMEs behind DNS steering.
• HTTP and HTTPS: the handshakes a nearby edge shortens.
• Latency vs throughput vs bandwidth: round trips and propagation delay.
• Single point of failure: anycast front doors and the CDN as a shared dependency.
• Design Netflix and Design YouTube: push versus pull delivery at video scale.
• Design an object store like S3: a typical origin for static files and media.

References and further reading

1. RFC 9111 — HTTP Cachingrfc-editor.orgShared versus private caches, the freshness-lifetime order, the 10% heuristic, Age, Vary, validation, every Cache-Control directive, and the cache-poisoning and Set-Cookie warnings.
2. RFC 9110 — HTTP Semanticsrfc-editor.orgConditional requests with If-None-Match, the 304 Not Modified response and the list of heuristically cacheable status codes, including 404.
3. RFC 8246 — HTTP Immutable Responsesrfc-editor.orgThe immutable directive for versioned URLs and its max-age=31536000 example.
4. RFC 7871 — Client Subnet in DNS Queriesrfc-editor.orgWhy DNS-based steering sees the resolver rather than the user, and the /24 and /56 truncation it recommends.
5. Amazon CloudFront Developer Guide: How CloudFront delivers contentdocs.aws.amazon.comDNS routing to the best POP, regional edge caches, persistent origin connections, and which requests skip the regional tier.
6. Amazon CloudFront Developer Guide: Use Amazon CloudFront Origin Shielddocs.aws.amazon.comOne extra caching layer in front of the origin that consolidates misses from every regional cache, its use cases (live streaming, multi-CDN) and when it doesn’t help.
7. Amazon CloudFront Developer Guide: Understand the cache keydocs.aws.amazon.comThe default key of domain plus path, and why Accept-Language, User-Agent and session cookies fragment the cache.
8. Cloudflare Cache documentation: Tiered Cachedevelopers.cloudflare.comLower and upper tiers, only upper tiers contacting the origin, and consistent hashing of the cache key to pick an upper tier.
9. Algorithmic Nuggets in Content Delivery (Maggs and Sitaraman, ACM SIGCOMM CCR, 2015)people.cs.umass.eduInside Akamai’s CDN: DNS mapping with client subnets and stable allocation, consistent hashing within clusters, parent clusters, and the speedups of a routing overlay.
10. Summary of June 8 outage (Fastly, 2021)fastly.comA latent bug triggered by a valid customer configuration change: 85% of the network returning errors, detection in one minute, 95% recovered within 49 minutes.