The Domain Name System (DNS) turns a name like api.example.com into the IP addresses a client should connect to, by following a chain of servers from the root of the name tree down to the servers that own the name, with caches at every step. In system design it matters twice over: every new connection to a new name waits for a lookup, and the answer you hand out is your cheapest tool for steering traffic between regions and your slowest for moving it away from a failure.
Picture it first: finding a shop in a new city
You have just moved to Delhi and want a sweet shop in Lajpat Nagar. You know its name, not its address, so you ask your cousin, who has lived there for years.
He doesn’t know either, but he knows how to find out. A national business directory can’t tell him about shops, but it knows which traders’ association covers Delhi. The association doesn’t keep addresses, but it has every member shop’s phone number. The shop gives its address, your cousin passes it on, and he writes it in his diary: “good for a week.”
All week, anyone in the family who asks gets the answer from the diary in seconds: the speed win. If the shop moves on day two, your cousin hands out the old address until the week is up: the staleness cost. Had the shop said “good for an hour”, he would phone more often but never be wrong for long.
Map it back:
- You are the application, and your phone’s address book is the stub resolver in your OS.
- Your cousin is the recursive resolver (your ISP’s, 1.1.1.1, or your VPC’s), and his diary is its cache.
- The directory, the association and the shop are the root, TLD and authoritative servers.
- “Good for a week” is the record’s TTL, and the shop chooses it, not your cousin. Short TTLs buy freshness and cost lookups; long ones buy speed and resilience and cost agility.
The resolution path, step by step
Names form a tree read from right to left: www.example.com. sits under example.com, which sits under com, which sits under the unnamed root. Each level delegates the next by publishing NS records that name the servers responsible for the child zone, so no server holds the whole tree.
- Stub resolver: the library in your OS that
getaddrinfocalls. It checks local caches and the hosts file, then asks a recursive resolver. It never walks the tree. - Recursive resolver: walks the tree and caches what it learns. It serves many clients, so one lookup warms the cache for all of them.
- Root and TLD servers: know who runs each top-level domain, and each domain beneath it.
- Authoritative servers: hold the zone’s actual records and give the final answer.
RFC 1034 defines two ways to ask. In a recursive query the server chases the answer for you and returns it or an error, never a referral. In an iterative query the server answers from what it knows or returns a referral to servers closer to the answer. Every name server must support iterative queries; recursion is optional. The stub sends one recursive query (with the RD, “recursion desired”, flag set) and the resolver makes the iterative ones.
Root servers: 13 names, about 2,000 sites
The root zone lists 13 named servers, a.root-servers.net to m.root-servers.net, run by 12 independent organisations. Each name is an anycast address announced from many places: on 27 October 2026 root-servers.org counted 2,049 operational instances. A resolver bootstraps from a “root hints” file of these addresses, then rarely needs them because the .com referral stays cached.
What a lookup costs
- Browser or OS cache hit: an in-memory lookup, far below a millisecond.
- Resolver cache hit: one round trip. At the 2018 launch of 1.1.1.1, Cloudflare cited DNSPerf measurements averaging about 14 ms globally; a distant resolver on a mobile network can be several times slower.
- Full miss: add a round trip to each server whose answer isn’t cached. Illustratively, with root and
.comcached, a resolver 15 ms away and authoritative servers 150 ms away on another continent: 15 + 150 ≈ 165 ms before the TCP handshake can start. A CNAME into another provider’s domain can add a second walk.
Every new hostname on your critical path pays this on first use, so keep their number small.
Record types you will meet in a design
| Type | Holds | Where it shows up |
|---|---|---|
| A | An IPv4 address | Almost every hostname: load balancers, APIs |
| AAAA | An IPv6 address | Dual-stack clients ask for both A and AAAA |
| CNAME | An alias for another name | Pointing www or static at a CDN or SaaS hostname |
| NS | The zone’s authoritative servers | Delegation; two DNS providers for resilience |
| SOA | Zone metadata | Sets how long “no such name” is cached |
| MX | Mail servers, each with a preference | Lowest number is tried first |
| TXT | Free text | Ownership proofs; SPF (v=spf1 …) lists allowed mail senders |
| SRV | Priority, weight, port and target | Discovery that needs a port, such as Kubernetes named ports |
| CAA | Which certificate authorities may issue | CAs must check it first (RFC 8659) |
| PTR | Address → name, under in-addr.arpa |
Reverse lookups for mail checks and logs |
Two details matter. DNS hands out addresses, not ports: without SRV, the port comes from the URL scheme. And a CNAME costs a second lookup of its target; Route 53 even bills a CNAME into its own zones as two queries. The addresses themselves are covered in IP addresses.
Why a CNAME can’t sit at the zone apex
RFC 1034 says that a name holding a CNAME must hold no other data, so a resolver can cache the alias and follow it for every record type. The zone apex, example.com itself, must hold SOA and NS records, so it can never be a CNAME. www.example.com can point at shop.cdn-x.net; example.com can’t.
Providers work around this with ALIAS, ANAME, CNAME flattening or Route 53 alias records: the authoritative server resolves the target itself and answers with plain A or AAAA records. You gain a working apex (on Route 53, with free queries to AWS targets). You pay with lock-in, since it isn’t a standard, and Route 53 alias records to AWS resources use the resource’s TTL, not yours. The other fix is an HTTP redirect from the apex to www.
CDNs plug in the same way: a CNAME into the CDN’s zone, whose servers pick an edge address per resolver. The CDN article covers the rest.
Caching and TTLs
Every record set carries a TTL in seconds, chosen by the zone owner. Caches count it down and pass on what remains: a 300-second record cached 200 seconds ago reaches your OS with 100 seconds left. On top of the general rules in caching 101, DNS has four cache layers: the browser; the OS (systemd-resolved or mDNSResponder, though many Linux servers run none, so every lookup goes to the resolver); the application runtime; and the shared recursive resolver, where RFC 8767 recommends capping any TTL at 7 days.
Negative answers are cached too
“No such name” (NXDOMAIN) and “no record of this type” (NODATA) are cached as well. Per RFC 2308, the server attaches the zone’s SOA record to a negative answer, whose TTL is the smaller of the SOA record’s own TTL and its MINIMUM field; the RFC suggests one to three hours. The trap: if a health checker or deploy script looks up new-feature.example.com before you create it, resolvers may keep saying “doesn’t exist” for that long afterwards. Create records before anything queries them.
Clients that ignore your TTL
The TTL is a request, not a contract
Honest resolvers respect it. Many clients don’t, and they decide how long a failover really takes.
- The JVM caches lookups for its own period, not the record’s TTL. Per the Java docs, with a security manager installed, successful lookups are cached forever; without one, for an implementation-specific time, which is 30 seconds in OpenJDK. Failed lookups are cached for 10 seconds. Set the
networkaddress.cache.ttlsecurity property explicitly; AWS recommends 5 seconds. - Long-lived connections (connection pools, HTTP/2 and gRPC channels, database drivers) resolve once and keep using that IP for hours. Changing the record does nothing until the connection breaks, so cap connection lifetimes.
- Kubernetes Pods get three search domains and
ndots:5, soapi.example.comis first tried with each cluster suffix appended: up to three wasted NXDOMAIN round trips. Use a trailing dot for external names, or lowerndots.
Choosing a TTL
Each resolver re-fetches a record at most once per TTL, so a back-of-envelope load on your authoritative servers is active resolvers ÷ TTL. With 20,000 resolvers serving your users:
- TTL 3,600 s: 20,000 ÷ 3,600 ≈ 6 queries/s
- TTL 300 s: 20,000 ÷ 300 ≈ 67 queries/s
- TTL 60 s: 20,000 ÷ 60 ≈ 333 queries/s
- TTL 5 s: 20,000 ÷ 5 = 4,000 queries/s
Users feel the other side: shorter TTLs mean more full lookups. Long TTLs also keep cached answers working while your DNS provider is down.
| 30–60 s | 300 s | 3,600–86,400 s | |
|---|---|---|---|
| Typical use | Health-checked failover records; records about to change | Ordinary service endpoints | NS, MX, rarely changing records |
| Worst-case staleness | About a minute | About 5 minutes | An hour to a day |
| Authoritative load | 333–667 queries/s | 67 queries/s | 0.2–6 queries/s |
| If your DNS provider is down | Cache lasts seconds | Minutes | Hours |
Route 53’s DNS best practices agree: 60 to 172,800 seconds overall, 60 or 120 seconds for health-checked failover records, and an hour to a day for NS and MX.
Lower the TTL before a planned move, not during it
Cut the TTL to 60 s at least one old TTL before the move (a day ahead if it was 86,400 s), so every cache already holds the short value. Switch, watch, then raise it again. Lowering it at the moment of the switch is too late.
DNS as a traffic-steering tool
An authoritative server doesn’t have to give everyone the same answer. That turns DNS into the first, coarsest layer of load balancing.
Round-robin DNS
Publish several A records for one name. Clients get the whole set, often shuffled, and most connect to the first. You gain free spreading with no extra hop. You pay because DNS doesn’t know which addresses are alive (a dead one stays until you remove it and the TTL expires), a big resolver hands one order to many users so load is lumpy, and you can’t set percentages.
Answers chosen by policy
Managed DNS computes the answer per query. Route 53’s routing policies are a good catalogue:
- Weighted: 90% of answers to A and 10% to B, for canaries (see deployment strategies).
- Latency-based: the region with the lowest measured latency to the requester.
- Geolocation and geoproximity: by the requester’s location, for data residency or language. Always define a default, or users from unmapped places get no answer.
- Failover: the primary while its health check passes, otherwise the secondary.
- Multivalue answer: up to eight healthy records at random, which is round-robin with health checks.
Health checks make these work. Route 53 probes from checkers around the world every 30 seconds by default, or every 10. An HTTP check must connect within 4 s and get a 2xx or 3xx within 2 s, status flips after 3 consecutive results by default, and an endpoint stays healthy while more than 18% of checkers agree. See health checks done right for what to probe.
The resolver-location problem
The authoritative server sees the resolver’s address, not the user’s, so a user in Pune whose public resolver answers from Singapore gets the Singapore answer. EDNS Client Subnet passes on a truncated piece of the client’s address, which Route 53’s latency routing uses when present, but not every resolver sends it. GeoDNS is a good guess, not a guarantee.
Anycast: one address, many places
Anycast gives everyone the same address, announced with BGP from many sites, and lets internet routing carry each packet to the nearest site in routing terms (usually, not always, the nearest on a map). Root servers, public resolvers such as 1.1.1.1, CDNs and DNS providers all use it.
You gain failover with no cache in the way, since a failing site withdraws its route, and floods are spread across every site. You pay with infrastructure (your own address block and BGP sessions, or a provider that has them), coarse control, and the risk that a route change mid-connection sends a long TCP flow to a site that has never seen it.
| Technique | How it picks | Reacts to failure | Good for | Watch out for |
|---|---|---|---|---|
| Round-robin | Same set for everyone | When you edit it, plus TTL | A few always-on endpoints | Dead IPs stay in rotation |
| Weighted | Your percentages | Health check + TTL | Canaries, migrations | Big resolvers make splits lumpy |
| Latency or geo | Resolver’s location | Health check + TTL | Multi-region, data residency | Resolver far from user; no default |
| Failover | Primary while healthy | Health check + TTL | Active-passive regions | Standby must be warm |
| Anycast | Internet routing | Route withdrawal, no TTL | DNS, CDNs, global front doors | Needs BGP; coarse control |
Regional failover with DNS, and why it’s slow
Say api.example.com has failover records: the primary points at the Mumbai load balancer with a health check, the secondary at Hyderabad, and the TTL is 60 seconds. At t = 0 Mumbai goes dark.
- Detection. With 30-second checks and a threshold of 3, Route 53 needs three failures in a row. If the last passing check came just before the failure, that is up to 3 × 30 = 90 s (or 3 × 10 = 30 s with fast checks).
- Switch. Route 53 starts answering with Hyderabad. No record is edited, so nothing has to propagate.
- Cache drain. A resolver that refreshed just before the switch keeps the dead address for one more TTL: 90 + 60 = 150 s after the failure.
- Stragglers. OpenJDK adds up to 30 seconds; a cache-forever JVM, a pinned connection pool or an app with its own cache keeps the dead address until it restarts or reconnects.
So “TTL 60” means about two and a half minutes for well-behaved clients, and longer for the rest. Detection, not the TTL, is usually the bigger number.
Ways to make it faster, and what each costs:
- Detect sooner. 10-second checks cut detection to about 30 s, at a higher price and more risk of flapping.
- Short TTLs only where they matter: 60 seconds on health-checked records, long everywhere else.
- Let clients fail over themselves. Return two healthy addresses or a secondary hostname, with sane timeouts and retries, so a client reacts in one timeout, not one TTL.
- Bound connection lifetimes so pooled clients re-resolve.
- Put a fixed front door in front. An anycast address or global load balancer keeps one IP and health-checks behind it, so no DNS cache is involved.
- Keep the standby warm, or failover becomes a second outage; see failover and multi-region and active-active.
In the interview
Put a number on it: “DNS failover is detection plus TTL plus clients that ignore the TTL. With 30-second checks and a 60-second TTL that’s about two and a half minutes for well-behaved clients. If I need seconds, I fail over behind a load balancer inside the region, or put an anycast front door in front so the address never changes.”
DNS or a load balancer?
They are not competitors: DNS picks the region, the load balancer picks the server.
| DNS steering | Load balancer | |
|---|---|---|
| Decides | Once per lookup per resolver, cached for the TTL | Per connection (L4) or per request (L7) |
| Granularity | A region or endpoint | One server, even one request |
| Reacts to failure | Minutes: detection + TTL + stragglers | Seconds: its own health checks and retries |
| Knows about load | No | Yes: least connections, queue depth |
| Extra hop | None | One proxy hop that must itself scale and stay up |
| Reach | Global, even across providers | Usually one region |
L4 vs L7 load balancing covers the second column, and proxy vs reverse proxy explains that extra hop.
Transport: UDP, TCP and encryption
UDP first, TCP when the answer won’t fit
Queries normally use UDP port 53: one packet each way, no handshake. RFC 1035 capped UDP messages at 512 bytes; a longer answer is truncated with the TC bit set, and the client retries over TCP port 53. EDNS(0) lets a client advertise a bigger UDP buffer, and because fragmented UDP is unreliable and easier to spoof, DNS Flag Day 2020 settled on 1,232 bytes as the safe default. RFC 7766 made TCP support mandatory. DNSSEC signatures, long TXT records and many A records make answers big, so a firewall that allows only UDP 53 breaks exactly those: allow TCP 53 too. TCP vs UDP explains why UDP comes first.
Encrypted DNS: DoT and DoH
Classic DNS is plaintext, so café Wi-Fi or your ISP can see and tamper with every lookup. DNS over TLS (RFC 7858) runs on TCP port 853; DNS over HTTPS (RFC 8484) sends queries as HTTPS requests that blend in with web traffic. Both protect only the stub-to-resolver hop; the resolver still sees every query. In a company, DoH in the browser can bypass the corporate resolver.
DNSSEC: signed answers, not secret ones
DNSSEC signs record sets and builds a chain of trust from the root key down through each delegation, so a validating resolver can prove an answer came unmodified from the zone owner. RFC 4033 is explicit about the limits: no confidentiality and no protection against denial of service. Signatures also make answers bigger, which brings back the TCP fallback.
Cache poisoning in one paragraph
A resolver accepts the first response that matches its outstanding query, so an attacker whose forged reply wins the race plants a false record for every user behind that resolver until the TTL expires. For years the main secret was a 16-bit query ID, only 65,536 values, often from a predictable port. In 2008 Dan Kaminsky made the race repeatable: query random names under the target domain, so the answer is never already cached, and stuff the forged replies with a fake delegation for the whole domain. The fixes are random source ports (up to 16 more bits to guess), accepting only records within the zone being asked about (RFC 5452), and DNSSEC. As Cloudflare notes, that makes the attack harder, not impossible.
Going deeper: when DNS is the outage
Facebook, October 2021: running servers nobody could reach
During maintenance, a command meant to assess backbone capacity took down every link in Facebook’s backbone; a bug in an audit tool failed to stop it. Meta’s post-mortem explains the twist: its authoritative DNS servers withdraw their own BGP route announcements when they can’t reach the data centres, a sensible reaction to a partial failure. With the whole backbone gone, every location withdrew at once, so the DNS servers kept running but nobody could reach them.
From outside, Cloudflare saw facebook.com fail to resolve on 1.1.1.1 from about 15:50 to 21:20 UTC, while resolvers worldwide handled 30× their usual queries as apps and people retried. Inside, losing DNS broke many of the tools engineers needed, so recovery took people on site. The lessons: a health check that removes capacity must never remove the last of it, emergency access must not depend on your own DNS, and stale-serving resolvers (RFC 8767) only help names already in cache.
Dyn, October 2016: one provider, many victims
Dyn, a large managed DNS provider, took two waves of DDoS traffic on 21 October 2016, from about 11:10 to 13:20 UTC and from 15:50 to 17:00 UTC. Dyn’s analysis named Mirai botnets as the main source, with up to 100,000 attacking endpoints flooding port 53. Legitimate resolvers made it worse: retrying to refresh their caches, they generated 10–20× normal traffic. Customers’ sites became unreachable for many users, however healthy their own servers were. The lessons: name servers from two independent providers, long TTLs on stable records, and clients that back off instead of joining a retry storm.
Your own resolver is on every request path
Inside your platform, the VPC resolver or cluster DNS sits in front of every outbound call. The glibc stub accepts up to three name servers, tries them in order and by default waits 5 seconds before moving on, so a dead first resolver adds about 5 seconds to every lookup that misses the cache: a mysterious latency spike everywhere at once. Run a node-local cache, keep resolvers redundant and monitored, and treat DNS like any other single point of failure.
Cheat sheet
| Topic | Remember |
|---|---|
| Path | Browser → OS stub → recursive resolver → root → TLD → authoritative |
| Query kinds | Stub → resolver recursive; resolver → servers iterative (referrals) |
| Root | 13 names, 12 operators, 2,049 anycast instances (27 Oct 2026) |
| Apex | No CNAME beside SOA and NS; use ALIAS, flattening or a Route 53 alias |
| TTL | Set by the zone owner; caches count it down and pass on what is left |
| Negative TTL | min(SOA TTL, SOA MINIMUM); one to three hours suggested |
| Load from a TTL | Active resolvers ÷ TTL: 20,000 ÷ 60 ≈ 333 queries/s |
| Lookup cost | Local hit under 1 ms; resolver hit ≈ one round trip; cold miss up to hundreds of ms |
| Transport | UDP 53; 512 bytes classic, 1,232 with EDNS(0); TC bit → retry on TCP 53 |
| Security | DoT and DoH encrypt the stub hop; DNSSEC signs but doesn’t encrypt |
| Failover time | Detection (3 × 30 s) + TTL (60 s) ≈ 150 s, plus clients that ignore TTLs |
| DNS vs LB | DNS picks the region in minutes; the LB picks the server in seconds |
How to say it in the interview
“The client checks its local caches, then sends one recursive query to a resolver. On a miss, the resolver goes iteratively from a root server to the
.comservers to our authoritative servers, gets the A records with a TTL, and every layer caches them. For this design I’d host DNS with a managed anycast provider, maybe two, and use latency-based records with health checks to send each user to the nearest healthy region: 60-second TTLs on those records, long ones elsewhere. Inside each region a load balancer picks the server. DNS failover takes a couple of minutes, so anything that must fail over faster happens behind the load balancer.”
Walk me through the DNS part of “what happens when I type a URL”. Browser cache, OS cache and hosts file, then a recursive query to the resolver. On a miss, the root refers it to the TLD servers, they refer it to the authoritative servers, and those return the A or AAAA record and TTL, cached on the way back. Then come the TCP and TLS handshakes (HTTP and HTTPS).
Why is DNS failover slow, and how would you speed it up? Detection (3 × 30 s by default), the TTL still running in caches, and clients that ignore the TTL or never re-resolve. Use 10-second checks and a 60-second TTL, cap connection lifetimes, let clients retry a second address, or put an anycast or global load-balancer address in front so the IP never changes.
Why not set the TTL to 1 second? Authoritative load rises to about one query per resolver per second, most users pay for a full lookup on each new connection, and detection still dominates failover time.
Why can’t I put a CNAME at the apex?
A name with a CNAME can’t hold other records, and the apex must hold SOA and NS. Use an ALIAS, flattening or alias record that answers with plain A records, or redirect the apex to www.
What protects you if your DNS provider is attacked, as Dyn was? Name servers from two independent providers, long TTLs on stable records so cached answers survive, an anycast provider that can absorb floods, and clients that back off instead of retrying in a storm.
Where to go next
- IP addresses: what the A and AAAA answers contain.
- TCP vs UDP: why DNS starts on UDP and when it needs TCP.
- HTTP and HTTPS: the handshakes right after the lookup.
- Load balancing: L4 vs L7: the per-request layer behind DNS.
- Content delivery network: what happens after the CNAME to a CDN.
- Failover and multi-region and active-active: DNS steering in a full regional strategy.
- Service discovery: DNS for finding services inside your platform, and its limits.
References and further reading
- RFC 1034 — Domain Names: Concepts and Facilitiesrfc-editor.orgDelegation, recursive versus iterative queries, referrals, and the rule that a name holding a CNAME holds no other data.
- RFC 1035 — Domain Names: Implementation and Specificationrfc-editor.orgMessage format, the 512-byte UDP limit, the TC bit and TCP retry on port 53, TTLs, and reverse lookups under IN-ADDR.ARPA.
- RFC 2308 — Negative Caching of DNS Queries (DNS NCACHE)rfc-editor.orgHow NXDOMAIN and NODATA answers are cached, with a TTL equal to the smaller of the SOA’s TTL and its MINIMUM field.
- Root Server Technical Operations Association (root-servers.org)root-servers.org13 root server identities run by 12 operators, and the live count of anycast instances (2,049 on 27 October 2026).
- Amazon Route 53 Developer Guide: Best practices for Route 53 DNSdocs.aws.amazon.comTTL guidance (60 to 172,800 s overall, 60–120 s for health-checked records, an hour to a day for NS and MX), the cost of CNAMEs and change propagation.
- Amazon Route 53 Developer Guide: How Route 53 determines whether a health check is healthydocs.aws.amazon.com10- or 30-second check intervals, the connect and response deadlines, and the more-than-18%-of-checkers rule.
- Java SE 21 API: java.net.InetAddress, “InetAddress Caching”docs.oracle.comnetworkaddress.cache.ttl, the 10-second negative cache, stale caching, and cache-forever under a security manager.
- More details about the October 4 outage (Meta Engineering, 2021)engineering.fb.comHow a backbone command plus DNS servers that withdraw their own BGP routes made Facebook’s authoritative DNS unreachable.
- Understanding how Facebook disappeared from the Internet (Cloudflare, 2021)blog.cloudflare.comThe outside view: route withdrawals, SERVFAIL from public resolvers from about 15:50 to 21:20 UTC, and 30× normal resolver load.
- Dyn Analysis Summary of Friday October 21 Attack (Dyn, 2016; archived copy)web.archive.orgTimeline of the Mirai-driven DDoS on Dyn’s managed DNS, up to 100,000 attacking endpoints, and 10–20× legitimate retry traffic.