Skip to content

Trade-offs · Foundations

L4 vs L7 load balancing

Which load balancer layer to use, L4 or L7, with the uneven-load numbers for long connections and the cost of reading requests.

Asked at: SDE-1SDE-2Senior

The decision

A load balancer sits in front of your servers and passes each client to one of them. Network messages have layers. Layer 4 (L4) is the connection, the TCP link between two computers. An L4 load balancer picks a server for each connection and passes the bytes through. It sees only addresses and port numbers. Layer 7 (L7) is the application, the HTTP request. An L7 load balancer reads each request, with its path, headers and cookies, and picks a server for each request.

Choose L4 when

  • The traffic is not HTTP, such as a database protocol or a game.
  • You need very high volume at low cost per request, or a fixed IP address for the front door.
  • Each connection is short, or carries about the same load as the others.

Choose L7 when

  • You route by path, host name, header or cookie. For example /api goes to app servers and /images goes to file servers.
  • Clients keep one connection open and send many requests on it, as HTTP/2 and gRPC clients do.
  • You want sticky sessions by cookie, redirects, or limits for each route.
  • You want to decrypt HTTPS at the balancer and read the request.

The numbers that change the answer

Worked example (Assumption: internal gRPC clients). A request costs 8 ms of CPU. A 4-core server does 4 x 1,000 / 8 = 500 requests per second at full load, and 350 at 70% busy. You run 10 servers. Eight internal services call you. Each opens one long HTTP/2 connection and sends 300 requests per second on it. Total = 8 x 300 = 2,400 requests per second.

  • L4: the balancer picks a server once for each connection. A managed L4 balancer chooses by a hash of the connection details, and each connection stays on one target for its life (see Sources). So 8 connections land on 10 servers more or less at random.
    • Ways to place 8 connections on different servers: 10 x 9 x 8 x 7 x 6 x 5 x 4 x 3 = 1,814,400. All ways: 10 to the power of 8 = 100,000,000. Chance = 1.8%.
    • So with chance 98.2%, at least two connections share a server. It gets 2 x 300 = 600 requests per second, but can do 500. That is 120% of its full capacity, and 171% of the 350 target. Other servers sit idle.
  • L7: the balancer picks a server for each request. Each server gets 2,400 / 10 = 240 per second, which is 240 / 500 = 48% busy.

Cost of reading requests. Assumption: an L7 balancer adds 0.5 ms to each request. With a 200 ms response budget, that is 0.5 / 200 = 0.25%. Assumption: one L7 node takes 20,000 requests per second. For a peak of 100,000, that is 100,000 / 20,000 = 5 nodes, plus 1 spare = 6. An AWS Network Load Balancer, which works at layer 4, is documented as able to handle millions of requests per second (see Sources).

QuestionL4L7
What it seesAddresses and portsThe full HTTP request
It balancesConnectionsRequests
Route by path or cookieNoYes
Work per requestLessMore

What breaks with each choice

  • L4: uneven load when connections are few and long, as in the example. It cannot read paths, so it cannot route by path or by cookie. Fix: use L7, or open more connections from each client and close old ones after a set age so that clients spread out again.
  • L7: more CPU and a little more delay for each request. It must also see the plain request, so it holds your HTTPS certificate. A fault in its rules can send all traffic to the wrong place. Fix: scale the L7 nodes, keep rules in version control, and test rule changes on a small share of traffic first.
  • Both: the balancer is a single point of failure. Fix: run more than one, in different data centres.

What it costs

  • Both are often managed services that you pay for by use. Check the price page, because prices change.
  • L7 needs certificates, routes and rules. Assumption: 3 to 5 engineer-days for a first setup.
  • Changing later is easy: you can put an L7 balancer behind an L4 one without changing the servers.

How each level answers

SDE-1: L4 works on connections and sees only addresses. L7 works on HTTP requests and sees paths, headers and cookies.

SDE-2: I use L7 for a web API, because I route /api and /images to different servers and may need cookies for stickiness. I use L4 for non-HTTP traffic, such as a database protocol.

Senior: With 8 long gRPC connections on 10 servers, two share a server with chance 98.2%. That server gets 600 requests per second against a limit of 500. L7 gives each server 240. The failure of L7 is cost and a single point of failure, so I run 6 nodes. The fallback is L4 in front for volume.

Staff: I choose the layer by what the product needs to route and who runs it. Many systems use both: L4 at the front for a fixed IP and volume, then L7 for routing. The platform team owns the edge. I start with a managed L7 balancer. I add L4 when I need a fixed IP, or when L7 cost grows too high. I measure the busiest server against the average, added p99 (the time within which 99 of 100 requests finish) delay, and error rate by route.

Say it in the interview

"For HTTP APIs I use L7 to route by path and balance each request, and I keep L4 for non-HTTP traffic and as a fast front door. L4 picks a server once for each connection, so 8 long gRPC connections on 10 servers put two on one server with chance 98%. That server then gets 600 requests per second against a limit of 500."

Common mistakes

  • Using L4 for gRPC clients that keep one connection open. It is wrong because all the requests of a connection go to one server, so the load is uneven.
  • Saying "L7 is always better". It is wrong when traffic is not HTTP, or when volume is so high that reading every request costs too much.
  • Forgetting that the balancer can fail. A single balancer is a single point of failure.

Sources

Understand the trade-off. Practise explaining your choice.

In an AI mock interview, the interviewer can push on a choice like this one: it changes a number and asks if your answer still holds.

See the mock interviewsAll trade-offs