Connections: TCP, HTTP/1.1, HTTP/2 and HTTP/3

"What happens when you type a web address?" is a common question in backend screens (lesson 22). The next questions are about time. "Why is the first request slow?" "What is different in HTTP/2?" "Why does HTTP/3 use UDP?" HTTP/2 and HTTP/3 are newer versions of HTTP.

A weak answer lists the steps: DNS, the TCP connection, the lock of HTTPS, then HTTP. A strong answer counts round trips ("three before the first byte"). It also says how to save them: reuse the connection, or put many requests on one connection.

At SDE-1 level you should list the steps and the cost of each. At SDE-2 level you should count round trips for each HTTP version. At SDE-3 level you should also explain why one lost piece of data can stop a whole page, and what HTTP/3 changes.

Think about ordering food by phone. You dial, the shop says hello, and you say hello back. Now you both know the line works, and only then do you ask for food. If you hang up after each question, you must dial again each time. It is better to stay on the line and ask for everything in one call.

A connection is an agreement between two programs to send data to each other, in order. Lesson 22 showed that TCP (Transmission Control Protocol) sets up a connection. It also showed that a round trip is one message going out and the answer coming back. This lesson counts those round trips.

The TCP handshake

TCP cuts your data into pieces called segments. Each segment travels in one packet, a small unit of data on the network. TCP numbers the segments. The receiver confirms them, and the sender sends again any that are lost.

Before data moves, the two sides run a handshake: a short exchange to agree that both are ready. The client sends a hello, and the server answers. The client can send its request as soon as the answer arrives. So the handshake costs 1 round trip.

The TLS handshake

TLS (Transport Layer Security) is the lock of HTTPS (lesson 22). It gives three things. Nobody on the way can read the data. The client learns who the server is. Any change to the data is detected.

To give these, the two sides run a second handshake and agree on secret keys. TLS 1.3 needs 1 round trip. The older TLS 1.2 needs 2.

What a new secure connection costs

Add the steps. Ignore DNS and the server's work.

With a round trip of 100 ms, the first request costs 300 ms. A second request on the same connection costs only 100 ms, because the handshakes are done. This is the reason to reuse connections.

Slow start

TCP does not know how fast the network is, and sending too much could overload it. So a new connection starts slowly. The window is how much data the sender may send before it hears back. Many servers start a new connection with a window of 10 segments, which is at most 14,600 bytes (10 segments of 1,460 bytes). Each confirmation lets the window grow. In this phase it about doubles every round trip. This phase is slow start.

When a packet is lost, TCP takes it as a sign that the network is full. After a quick sign of loss, it cuts the window to about half. If no confirmation comes for a while (a timeout), it goes back to 1 segment. A connection that is idle (nothing is sent on it) also shrinks its window again.

So a new connection is slow twice. The handshakes cost round trips, and the window is small. A connection that stays open and busy has a large window and is fast.

Keep-alive and connection reuse

A persistent connection, also called keep-alive, stays open after an answer so the next request can use it. HTTP/1.1 does this by default. A server closes a connection that stays idle too long. The nginx server does so after 75 seconds by default.

A client keeps a pool of open connections, one list for each host (the host name of lesson 22). To send a request, it takes an idle connection from the pool. It opens a new one only when none is free.

HTTP/1.1: one request at a time

On one HTTP/1.1 connection, the client sends a request and waits for the full answer. Only then does it send the next one. A slow answer blocks every request behind it on that connection. This is head-of-line blocking: the first in line holds up all the others.

HTTP/1.1 also allows a client to send several requests without waiting. The answers must still come back in the same order, and browsers do not use this in practice.

So a browser opens several connections to one host. Chrome opens at most 6. Assumption: we use 6 in our examples.

HTTP/2: many requests on one connection

HTTP/2 puts many requests on one connection. Each request and its answer form a stream with a number. HTTP/2 cuts data into small frames and puts the stream number on each frame. Frames of different streams are mixed on the same connection. This is multiplexing. A slow answer no longer blocks the others, because their frames can go in between.

HTTP/2 also compresses the headers (the small notes of lesson 22), because many requests repeat them.

Its new weakness is in TCP. TCP delivers bytes in order, and it does not know about streams. If one packet is lost, TCP holds back all later bytes until the lost packet is sent again. So one lost packet stops every stream, often for about one round trip. This is head-of-line blocking again, now at the TCP level.

TCP and UDP side by side

UDP (User Datagram Protocol) has no setup, no confirmation and no resend. It does not promise that data arrives, or arrives in order. That is bad for a web page. It is good when late data is worse than missing data.

A live cricket score is one case. If the update for over 12 is lost, the update for over 13 replaces it. With TCP, the newer update would wait behind the lost one. A voice call is the same, because a word that arrives too late is useless.

HTTP/3: QUIC on UDP

HTTP/3 uses QUIC, a way to send data that runs on top of UDP. QUIC builds its own confirmation and resend, so no data is lost. It uses UDP because UDP packets already pass through existing networks and devices. This makes QUIC easier to start using. HTTP/3 changes three things:

1. Independent streams. QUIC does not order bytes between streams. A lost packet stops only the streams that had data in it.
2. A shorter handshake. QUIC joins the TCP-style handshake and the TLS 1.3 handshake into one. A new connection needs 1 round trip, not 2.
3. No handshake for a returning client. A client that talked to the server before can put its request in its very first message. This is 0-RTT, which means zero round trips of waiting. The server may accept it only for requests that are safe to send twice. An attacker could copy the message and send it again.

QUIC also gives each connection an ID that is not tied to the network address. When a phone moves from Wi-Fi to mobile data, the same connection can continue.

Some networks block UDP. Then the client falls back to a version that uses TCP. A server tells a browser that HTTP/3 exists with a header called Alt-Svc. So a first visit may use HTTP/2.

NAT: many devices, one address

An IP address (Internet Protocol address) names one computer on the network (lesson 22). NAT (network address translation) lets many devices in one home, office or mobile network share one public IP address. The NAT device remembers which inside device each connection belongs to.

Two results matter. First, a server sees the same address for a whole office or mobile network, so one address is not one person. Second, the NAT device forgets a connection that is idle too long. Standards ask for at least 2 minutes for UDP and 2 hours 4 minutes for TCP. Real devices can be stricter.

So a connection that must stay open sends a small message now and then. Assumption: every 30 seconds. For UDP, the standard says not more often than every 15 seconds.

Anycast: one address, many places

With anycast, the same IP address is offered from many places. The network sends each packet to a place near the sender. Users in Chennai and Delhi use the same address and reach different servers. There is one risk for TCP. If the network changes its choice during a connection, the packets reach another place, and the connection breaks.

A long-lived connection

Some connections stay open for minutes or hours, so the server can send data whenever it wants. The cost is memory, because every open connection uses some. Assumption: 10 KB for each connection. For 100,000 open connections: 100,000 x 10 KB = 1,000,000 KB = 1 GB. This lesson names no product that uses such a connection.

How to measure

The curl tool can print the time of each step. Each number is the time since the start, so you subtract the one before.

# Not run here. -o /dev/null hides the page; -s hides the progress bar.
curl -o /dev/null -s -w "dns %{time_namelookup}\nconnect %{time_connect}\ntls %{time_appconnect}\nfirst byte %{time_starttransfer}\ntotal %{time_total}\n" https://example.com

• connect minus dns is the TCP handshake, about 1 round trip.
• tls minus connect is the TLS handshake, about 1 round trip with TLS 1.3.
• first byte minus tls is 1 round trip plus the server's work.
• total is the time for the whole answer.

Three answers to know

• "What happens when I type a URL?" First the DNS lookup. Then 1 round trip for TCP, 1 for TLS 1.3 and 1 for the request, plus the server's work. With a round trip of 100 ms that is 300 ms plus DNS and work.
• "HTTP/1.1 or HTTP/2?" HTTP/1.1 sends one request at a time on each connection, so a page with many files needs many connections. HTTP/2 sends all of them as streams on one connection.
• "Why is the first request slow?" A new connection pays three round trips and starts with a small window. Later requests reuse the open connection and pay only one.

Assumptions (made up to teach with): round trip 100 ms; TLS 1.3; 30 small files; the browser already knows all 30; 6 connections for HTTP/1.1; no DNS, no speed limit on the link, no server work, no slow start. On a real page the browser learns the file names from the first page, which costs one more round trip. So real times are longer.

HTTP/1.1. The 6 connections open at the same time. Each needs TCP (1) and TLS (1) = 2 round trips. Each carries 30 / 6 = 5 files, one after the other, 1 round trip each. Total = 2 + 5 = 7 round trips = 700 ms. With TLS 1.2 it is 8 round trips = 800 ms.

HTTP/2. One connection: 2 round trips of handshake. All 30 requests go out together and the answers return together: 1 round trip. Total = 2 + 1 = 3 round trips = 300 ms. This answers the predict box.

HTTP/3. The handshake is 1 round trip, then 1 for the requests. Total = 1 + 1 = 2 round trips = 200 ms. A returning client with 0-RTT needs 0 + 1 = 1 round trip = 100 ms.

Four bars drawn to the same scale. Each box is one round trip of 100 ms, with TLS 1.3 and no DNS. HTTP/1.1 with 6 connections and 5 files each: a TCP box, a TLS box, then 5 file boxes, which is 7 round trips or 700 ms. HTTP/2 with one connection: a TCP box, a TLS box, then one box for all 30 files, which is 300 ms. HTTP/3: one QUIC box, then one box for the 30 files, which is 200 ms. HTTP/3 for a returning client: only the box for the 30 files, which is 100 ms.

One lost packet. Assumption: the resend costs 1 extra round trip (100 ms). Assumption: the lost packet carried data of one stream only. On HTTP/2 all 30 files wait: 300 + 100 = 400 ms. On HTTP/3 only that stream waits. The other 29 files are done at 200 ms, and the last one at 300 ms.

Slow start. Now one file of 1 MB on a fresh connection. Assumption: 1 MB = 1,000 KB. The window starts at 10 segments x 1,460 bytes = 14,600 bytes = 14.6 KB and doubles each round trip. After n round trips the data sent is 14.6 KB x (2 to the power of n, minus 1):

After 6 round trips only 919.8 KB has arrived. So the file needs 7 round trips = 700 ms after the handshakes. This model ignores the speed limit of the link and any loss.

• Reusing a connection solves the cost of handshakes: 100 ms for a request instead of 300 ms. It costs memory on both sides, and idle connections get closed. I would change this when connections sit idle for long. Then I send a small message now and then, or open a new connection when needed.
• HTTP/2 solves the queue of HTTP/1.1: 300 ms instead of 700 ms in the example. It costs a new weakness: one lost packet stops all streams. I would change to HTTP/3 when users are on weak mobile networks.
• HTTP/3 solves the stop-and-wait after a loss, and it saves one round trip: 200 ms. It costs newer software to run, and some networks block UDP, so it needs a fallback to HTTP/2. I would keep both versions on.

"HTTP/3 uses UDP, so it can lose data." QUIC adds its own confirmation and resend on top of UDP, so data arrives complete. What HTTP/3 removes is the stop-and-wait from one lost packet.

"HTTP/2 removed head-of-line blocking." It removed it at the HTTP level. TCP still delivers bytes in order, so one lost packet stops every stream on the connection.

"Keep-alive means the connection stays open forever." The server closes idle connections (75 seconds by default in nginx). A NAT device can forget them sooner. A connection that must stay open needs a small message now and then.

1. Name the three round trips of a first HTTPS request with TLS 1.3. What does HTTP/2 multiplexing mean?
2. The round trip is 80 ms and TLS 1.3 is used. A page has 20 small files, and the browser uses 6 connections for HTTP/1.1. How long do all files take on HTTP/1.1, and on HTTP/2? Use the assumptions of the worked example.
3. A screen shows a live match score over one long connection. Users on mobile networks say it stops updating after about a minute of quiet and never recovers. What do you check, and what do you change?