HTTP and HTTPS

HTTP is the request–response protocol behind almost every web page, app and API call: a client sends a method, a target and headers, and the server answers with a status code, headers and a body. HTTPS is the same protocol inside TLS, which encrypts the conversation, detects tampering and proves the server owns the name you asked for. A designer needs to know what methods and status codes promise, how HTTP/1.1, HTTP/2 and HTTP/3 move bytes, and how many round trips a new connection costs.

Picture it first: a parcel counter with a sealed window

Picture the parcel counter at a busy railway station. You fill in a standard form: what you want done (book, track, cancel), which parcel, a few labelled details (weight, fragile), then hand over the parcel. The clerk serves hundreds of people an hour and remembers none of them, so to track yesterday’s parcel you show your receipt number.

Now make it secure. First you check the clerk’s ID card, issued by a head office you trust. Then you talk through a sealed window: nobody in the queue can read your form, and any tampering is noticed.

Checking the ID and sealing the window takes time, so you don’t rejoin the queue for each of your 20 parcels. And the counter might take one parcel per window at a time (so families send six people to six windows), put everyone’s parcels on one shared conveyor (which jams for all when one sticks), or give each parcel its own belt.

Map it back:

• The form is an HTTP request; the forgetful clerk is statelessness, and the receipt number a cookie or token.
• The ID card and head office are the certificate and the certificate authority (CA).
• The sealed window is TLS, sealing it is the handshake, and staying is connection reuse.
• The three ways of accepting parcels are HTTP/1.1, HTTP/2 and HTTP/3.

What an HTTP message looks like

Every HTTP exchange is a request and a response. In HTTP/1.1 both are plain text, which is why you can read them in a terminal.

A request has four parts:

• The request line: method (POST), target (/v1/orders) and protocol version.
• Headers: name: value metadata. Host names the site, so one IP address can serve many domains; HTTP/1.1 requires it. Content-Type says how to parse the body; Authorization carries credentials.
• An empty line that ends the headers.
• An optional body, framed by Content-Length (or sent in chunks) so the receiver knows where the next message begins.

A response mirrors it: a status line (version, three-digit status code, reason phrase), headers, an empty line and a body.

HTTP/2 and HTTP/3 carry exactly the same parts, which RFC 9110 defines once for every version; they just encode them as binary frames, with the request line turned into pseudo-headers such as :method and :path.

Stateless by design

RFC 9110 defines HTTP as stateless: each request can be understood on its own, and a server must not assume two requests on one connection come from the same user. That is what lets a load balancer send your next request to any server.

State still exists; it travels with each request. A cookie holds a session ID the server looks up in a shared store, or a token (often a signed JWT) carries the claims itself, so servers stay interchangeable. See stateful vs stateless design and sessions, tokens and JWT.

Methods: safe, idempotent, cacheable

The method tells every component on the path (browser, CDN, proxy, load balancer) what a request may do. RFC 9110 defines three properties:

• Safe: read-only by intent, so crawlers and prefetchers may call it freely.
• Idempotent: sending it twice has the same intended effect as once, so it can be resent after a dropped connection.
• Cacheable: a response may be stored and reused later.

Three consequences show up in every design.

Retries are only free for idempotent methods. RFC 9110 forbids a proxy from automatically retrying a non-idempotent request. If POST /payments times out, the client cannot tell whether the charge happened. The fix is an idempotency key: the server stores the result under a client-chosen ID, and a retry gets the first answer back instead of a second charge. See idempotency and timeouts, retries and backoff.

Idempotent means the same effect, not the same response. DELETE /orders/42 returns 204 the first time and 404 the second; either way, order 42 is gone.

“Safe” is a promise you must keep. A GET /orders/42/cancel link will be followed by the first crawler or prefetcher, cancelling the order. Put state changes behind POST, PUT, PATCH or DELETE.

Status codes that matter in a design

The first digit carries the meaning: 2xx success, 3xx go elsewhere, 4xx the client must change something, 5xx the server (or something behind it) failed. Proxies and clients act on the class even when they don’t know the exact code.

Two distinctions interviewers like:

• 401 vs 403. 401 means “who are you?” and must carry a WWW-Authenticate challenge. 403 means “I know who you are, and no”; the same credentials will never work.
• 502 vs 503 vs 504 usually come from a proxy and say which hop failed: a bad reply, load shedding, or the proxy’s timer. A wall of 504s at exactly 30 s points at a timeout setting, not a crash.

Don’t retry a 4xx unchanged (except 408 and 429, after waiting); retry 502, 503 and 504 only for idempotent requests, with backoff. Deeper traps are in HTTP API implementation gotchas and rate limiting.

From HTTP/1.0 to HTTP/3

HTTP/1.x: one request at a time per connection

HTTP/1.0 (RFC 1945, 1996) opened a new TCP connection for every request, so each paid a handshake plus TCP’s slow start, the warm-up during which a new connection can’t use the full bandwidth. HTTP/1.1 (1997, now RFC 9112) made persistent connections the default, so the next request reuses the connection.

A persistent connection still carries one exchange at a time. Pipelining (sending several requests without waiting) is allowed, but responses must return in request order, so one slow response blocks everything behind it: head-of-line blocking at the HTTP layer. With buggy proxies on top, browsers ship with it off.

So browsers open up to about 6 connections per origin instead. The arithmetic: a page needs 60 small files from one origin at 50 ms RTT. Across 6 connections that is 10 sequential round trips each, about 10 × 50 ms = 500 ms of pure waiting, plus 6 handshakes.

HTTP/2: many streams on one connection

HTTP/2 (2015, now RFC 9113) keeps the semantics and changes the wire format:

• Binary frames replace text lines.
• Streams: each request–response pair gets a stream ID, and frames from many streams interleave on one TCP connection. The server caps open streams; the spec recommends allowing at least 100.
• HPACK header compression: repeated cookies, user agents and tokens become small references into a table both sides keep.

If the browser knows all 60 files up front, they go out together: about one round trip, roughly 50 ms instead of 500, and one handshake.

The catch is the transport. TCP delivers one ordered byte stream and knows nothing about HTTP/2 streams, so when a packet is lost, every later byte waits for the retransmission, including bytes of other streams. RFC 9113 says plainly that TCP head-of-line blocking is not addressed. On a lossy mobile network, one HTTP/2 connection can do worse than six HTTP/1.1 connections, where a loss stalls only one.

HTTP/3: streams move into the transport

HTTP/3 (RFC 9114, 2022) runs on QUIC, a transport built on UDP that provides the streams itself: reliable in-order delivery per stream, congestion control across the connection, and TLS 1.3 inside its handshake. So:

• A lost packet stalls only the streams whose data it carried.
• Setup is faster: one combined handshake, 1 RTT for a new connection and 0 on resumption.
• Connections migrate: QUIC names a connection by connection IDs, not IP and port, so a phone moving from Wi-Fi to mobile data keeps it.
• QPACK replaces HPACK, which assumed in-order delivery across streams.

Two wrinkles. Clients usually learn about HTTP/3 from an Alt-Svc: h3=":443" header on an earlier TCP response, so the first visit usually goes over TCP. And some networks block UDP, so clients must fall back. In production you serve HTTP/2 and HTTP/3 side by side.

What HTTPS adds

HTTPS is HTTP sent through a TLS connection. RFC 8446 lists what TLS guarantees, even against an attacker who controls the network:

• Confidentiality: only the two endpoints can read the data (lengths, IP addresses and usually the hostname still leak).
• Integrity: any modification in transit is detected.
• Authentication: the server always proves its identity; with mutual TLS, the client does too.

TLS is a hybrid system. The handshake uses public-key cryptography, microseconds to milliseconds per operation, to agree on fresh symmetric keys; the bulk data then uses symmetric ciphers costing nanoseconds. In TLS 1.3 every key exchange is ephemeral Diffie–Hellman, giving forward secrecy: a private key stolen later cannot decrypt recorded past traffic. TLS 1.3 also encrypts everything after the ServerHello, certificate included.

Certificates and the chain of trust

How does your phone know it reached the real api.example.com, not someone on the café Wi-Fi? The server presents a certificate: its name and public key, signed by a certificate authority. The server also sends the intermediate certificate that signed it, and the intermediate is signed by a root CA already in the operating system’s or browser’s trust store.

Any failed check aborts the handshake. Leaf certificates are deliberately short-lived, and getting shorter.

SNI: one IP address, thousands of certificates

A CDN serves thousands of domains from the same IP addresses, so it must pick a certificate before any HTTP is exchanged. The client names the host in its first handshake message, an extension called Server Name Indication (SNI, RFC 6066). SNI is clear text, so an observer learns which site you visit even over HTTPS; Encrypted Client Hello (RFC 9849, 2026) hides it where both sides support it.

The handshake tax, in round trips

Before the first request byte leaves the client, a new connection pays round trips. At RTT = 50 ms:

• TCP + TLS 1.2: 1 RTT for TCP, then 2 for TLS 1.2 (hellos and certificate, then key exchange and Finished). The request leaves at 3 × 50 = 150 ms; the first response byte lands at 200 ms.
• TCP + TLS 1.3: the client guesses the key-exchange group and sends its key share up front, so TLS takes 1 RTT. Request at 2 × 50 = 100 ms; response at 150 ms.
• QUIC (HTTP/3): transport and TLS handshakes are one exchange. Request at 50 ms, response at 100 ms, the same as unencrypted HTTP over TCP.
• Resumption: with a session ticket from a recent visit, TLS 1.2 saves a round trip, and TLS 1.3 and QUIC can put the request in the very first flight (0-RTT).

The numbers scale with distance. Mumbai to northern Virginia is about 12,900 km in a straight line; at roughly 10 ms of round trip per 1,000 km of fibre, that is at least 130 ms RTT, and real routes are longer. Call it 200 ms: TCP + TLS 1.2 then costs 3 × 200 = 600 ms before the request leaves the phone. So where the handshake happens matters as much as the protocol.

Where TLS terminates

To terminate TLS is to decrypt it. Whoever terminates holds a private key and sees plaintext: URLs, headers, cookies, bodies. You choose per hop.

At the edge (CDN or global load balancer). The user handshakes with a point of presence a few milliseconds away, and the edge forwards the request over a connection to the origin it already holds open, so the long hop costs one round trip instead of three. In the figure, 2 × 10 ms of setup + 10 ms to the edge and back + 200 ms over the warm link ≈ 230 ms, against 3 × 200 = 600 ms when the TCP + TLS 1.3 handshake goes all the way to Virginia. Even uncacheable API calls gain, and the edge can also cache, compress and run a WAF. See content delivery network.

At the load balancer or API gateway. An L7 balancer must decrypt to route by path or header, check auth or retry. An L4 balancer can pass TLS through untouched, keeping encryption end to end, but sees only IPs and ports. See L4 vs L7 load balancing, proxy vs reverse proxy and API gateway.

Inside the network. You can forward plain HTTP, which is cheapest but trusts everything on that network, or re-encrypt. Many teams use mutual TLS between services: both sides present certificates from an internal CA, and the certificate becomes the service’s identity, usually handled by a service-mesh sidecar. See service-to-service auth and mTLS.

HSTS: never fall back to plain HTTP

A user who types example.com starts on plain http://, and an attacker on the network can intercept that request and keep the victim on HTTP. HTTP Strict Transport Security (RFC 6797) closes the gap. The HTTPS response carries:

Strict-Transport-Security: max-age=31536000; includeSubDomains

For the next year (31,536,000 seconds) the browser rewrites every http:// URL for that host and its subdomains to https:// before sending, and won’t let the user click through certificate errors. The first visit is still exposed, so browsers ship a preload list of HTTPS-only domains. The trap: includeSubDomains with a long max-age is hard to undo, so make every subdomain serve HTTPS first.

Certificates are automated now

ACME (RFC 8555) lets a server prove control of a domain and get a certificate with no human involved: it serves a token over HTTP (HTTP-01), answers a special TLS handshake (TLS-ALPN-01) or publishes a DNS TXT record (DNS-01). Let’s Encrypt’s free 90-day certificates are built on it.

Lifetimes are shrinking. Under the CA/Browser Forum Baseline Requirements, public certificates issued from 15 March 2026 last at most 200 days, falling to 100 days from March 2027 and 47 days from March 2029. Let’s Encrypt’s default moves from 90 days to 64 in February 2027 and 45 in February 2028, and 6-day certificates are available now. Automate renewal at about two-thirds of the lifetime, alert when one is late, and keep private keys in a managed key store; see encryption and key management.

Keep connections warm

Every handshake number above is paid once per connection; reuse the connection and the next request costs one round trip. That is the case for keep-alive and connection pools:

• Latency. A warm connection skips the handshake and has already left TCP slow start.
• Server CPU. The expensive part of TLS is the handshake’s public-key work. A service calling a downstream 5,000 times a second over fresh connections forces 5,000 handshakes a second; a pool of 50 connections recycled every 10 minutes needs 50 ÷ 600 s ≈ 0.08 a second.
• Memory. Google measured under 10 KB of TLS memory per connection in 2010. One million users with 6 HTTP/1.1 connections each is about 60 GB of TLS state, against 10 GB with one HTTP/2 connection each.

Encryption itself is cheap. When Gmail switched to HTTPS for everyone in 2010, Google added no machines: TLS was under 1% of CPU load and under 2% of network overhead.

Long-lived connections bring two gotchas:

• The idle-timeout race. If a load balancer sends a request just as the backend closes that idle connection, the request fails, and an AWS Application Load Balancer returns a 502. AWS advises making the application’s keep-alive timeout longer than the balancer’s (60 s by default), so the balancer always closes first.
• Sticky imbalance. A long-lived HTTP/2 connection pins a client to one backend, so servers added during a scale-out get nothing from existing clients. Cap connection age at a few minutes.

Going deeper: caching, expiring certificates and Rapid Reset

HTTP caching in one section

HTTP has caching built in (RFC 9111), and browsers, CDNs and reverse proxies all follow it:

• Cache-Control: max-age=600 means fresh for 10 minutes; s-maxage overrides it for shared caches such as a CDN.
• private: only the user’s browser may store it. no-store: store nothing (an account balance). no-cache: store it, but revalidate before every reuse.
• Validators. The server labels each version with an ETag. A client with a stale copy sends If-None-Match: "v7"; if nothing changed, the answer is 304 Not Modified with no body, so a 200 KB response shrinks to a few headers.
• The same validator guards writes: a PUT with If-Match: "v7" gets 412 if someone already wrote v8. That is optimistic concurrency, a lost-update guard without locks; a plain clash with current state, such as a taken username, is a 409.

Keys, invalidation and stampedes belong to caching 101 and hot keys, cache stampede and invalidation.

When certificates expire

Certificates fail on a date known years ahead, and still cause outages. The clearest public example is the end of Let’s Encrypt’s old cross-signed chain: an older R3 intermediate expired on 29 September 2021, and the IdenTrust DST Root CA X3 expired the next day at 14:01 UTC.

Browsers were mostly fine. Scott Helme’s post-mortem shows the breakage hit clients nobody thinks of as clients: Google Cloud Monitoring marked servers with Let’s Encrypt certificates as down, Azure Application Gateway stopped connecting to such backends, and firewalls and hosting platforms had incidents, mostly from outdated OpenSSL and trust stores. Some servers kept sending the expired intermediate until restarted.

The lessons:

• Let automation install the full chain with every renewal, never a hand-copied intermediate.
• Monitor expiry from outside for every endpoint, internal mTLS certificates included. With 47-day certificates, a broken renewal job becomes an outage within weeks.

HTTP/2 Rapid Reset

Multiplexing moves the unit of abuse from connections to streams. In 2023 attackers opened HTTP/2 streams and cancelled each at once with RST_STREAM: a cancelled stream stops counting toward the limit, but the server has already started work, so one connection could fire requests without a cap. Cloudflare measured a peak just above 201 million requests per second from about 20,000 machines (CVE-2023-44487). The defence is to count resets per connection and close abusive ones with GOAWAY: a multiplexed protocol needs per-connection limits on work done, not only on streams open.

Cheat sheet

How to say it in the interview

“HTTP is stateless, so any server can handle any request and session state rides in a token. Reads are GETs with Cache-Control and ETags so the CDN can serve them; writes are POSTs with an idempotency key so clients can retry timeouts. Clients reach a CDN edge that terminates TLS 1.3 and speaks HTTP/2 and HTTP/3, so the handshake happens ten milliseconds from the user and the edge reuses warm connections to our origin. Behind it, an L7 balancer re-terminates TLS and routes by path, and services talk mTLS. Certificates come from ACME, with expiry monitoring.”

Why can HTTP/2 be slower than HTTP/1.1 on a bad mobile network? All streams share one TCP connection, and TCP releases bytes in order, so one lost packet stalls every stream; with six HTTP/1.1 connections it stalls one. HTTP/3 fixes this.

Is plain HTTP inside the VPC acceptable? It is cheaper, but anyone on that network reads tokens and personal data, and many compliance regimes require encryption in transit. I’d re-encrypt by default; mTLS also gives each service a verified identity.

Can a client safely retry a POST that timed out? Not blindly: the first attempt may have succeeded. Add a client-generated idempotency key the server deduplicates on, then retry with backoff and jitter.

Why not HTTP/3 between services too? Some networks block UDP, so you need TCP anyway, and inside a region round trips are short and loss rare. QUIC pays off on the user’s long, lossy, mobile leg.

Where to go next

• TCP vs UDP: the handshake, slow start and ordering HTTP inherits, and what QUIC builds on.
• Domain Name System: the lookup before any of these round trips.
• REST vs GraphQL vs gRPC: API styles on top of HTTP, and why gRPC needs HTTP/2.
• HTTP API implementation gotchas: status-code misuse, timeouts and retry traps.
• Load balancing: L4 vs L7: what each balancer can see once TLS ends.
• Real-time connections: long polling, SSE and WebSockets, which all start as HTTP.
• Service-to-service auth and mTLS: certificates as service identity.

References and further reading

1. RFC 9110 — HTTP Semanticsrfc-editor.orgStatelessness, the safe, idempotent and cacheable method properties, retry rules, and the definitions of every status code used here.
2. RFC 9112 — HTTP/1.1rfc-editor.orgPersistent connections as the default, pipelining’s in-order response rule, connection limits and the idle-close race.
3. RFC 9113 — HTTP/2rfc-editor.orgBinary frames, streams, header compression, the recommended minimum of 100 concurrent streams, and the admission that TCP head-of-line blocking remains.
4. RFC 9114 — HTTP/3rfc-editor.orgHTTP over QUIC, per-stream loss recovery, QPACK, discovery through Alt-Svc and the fallback to TCP when UDP is blocked.
5. RFC 8446 — The Transport Layer Security (TLS) Protocol Version 1.3rfc-editor.orgWhat TLS guarantees, the changes from TLS 1.2 (forward secrecy, encrypted handshake), and the weaker replay and forward-secrecy properties of 0-RTT data.
6. A Detailed Look at RFC 8446 (a.k.a. TLS 1.3) — Cloudflareblog.cloudflare.comWhy TLS 1.2 needs two round trips and TLS 1.3 one, how 0-RTT resumption works, and why only safe requests belong in early data.
7. RFC 6797 — HTTP Strict Transport Security (HSTS)rfc-editor.orgThe Strict-Transport-Security header, no click-through on certificate errors, the bootstrap problem and the preload list.
8. Decreasing Certificate Lifetimes to 45 Days — Let’s Encrypt (2025)letsencrypt.orgThe 90 → 64 → 45-day schedule, ACME profiles, the validation challenges, and the advice to renew at about two-thirds of the lifetime.
9. Let’s Encrypt Root Expiration — Post-Mortem (Scott Helme, 2021)scotthelme.co.ukWhat broke when the R3 intermediate and DST Root CA X3 expired, including Google Cloud Monitoring and Azure Application Gateway failures.
10. Overclocking SSL (Adam Langley, Google, 2010)imperialviolet.orgGmail’s move to HTTPS by default: under 1% of CPU, under 10 KB of memory per connection and no extra machines.