What does a load balancer do?
A load balancer distributes incoming traffic across a group of backend targets, such as virtual machines, containers, or servers. It checks which targets are healthy and stops sending traffic to any that fail. This keeps any one server from being overloaded, and it keeps the application available when a server goes down. A load balancer can be a hardware appliance, a software program, or a managed cloud service.
How a request flows through a load balancer
- A client connects to the load balancer’s address, which is usually a single virtual IP address or DNS name.
- The load balancer selects a healthy backend, using an algorithm and recent health information.
- It forwards the request to that backend.
- The response returns to the client, usually through the load balancer.
Layer 4 and Layer 7 load balancing
- Layer 4 balances TCP and UDP traffic based on IP address, port, and protocol. It does not look inside the request, so it cannot route by URL path. It is fast and suits non-HTTP traffic.
- Layer 7 understands application protocols, usually HTTP and HTTPS. It can route by host name, path, or header, which is useful for microservices and for several websites behind one address. It can also terminate TLS encryption.
Cloud providers offer both kinds. AWS has a Network Load Balancer (layer 4) and an Application Load Balancer (layer 7), and a Gateway Load Balancer that works at layer 3 to send traffic through virtual security appliances. Azure has Azure Load Balancer (layer 4) and Application Gateway (layer 7). Google Cloud Load Balancing offers both application and network options.
Balancing algorithms
- Round robin sends requests to each server in turn. It works well when servers are similar and requests take similar amounts of time.
- Weighted round robin sends more requests to servers with more capacity.
- Least connections sends traffic to the server with the fewest active connections. It suits workloads where requests take varying amounts of time.
- Least response time favors the server that is responding fastest.
- IP hash uses the client’s IP address to choose a server, so the same client usually reaches the same server.
AWS’s Application Load Balancer defaults to round robin and also offers least outstanding requests. The Network Load Balancer uses flow hashing.
Health checks
Load balancers check whether backends are healthy before sending them traffic. A check can be a TCP connection attempt, an HTTP or HTTPS status-code check, a call to an application-specific health endpoint, or a response-time measurement. A target that fails its checks is removed from rotation until it passes again.
Checks have settings for how often they run, how long they wait, and how many passes or failures count as healthy or unhealthy. As an example, the AWS Application Load Balancer’s defaults are a 30-second interval, a 5-second timeout, and 5 healthy and 2 unhealthy thresholds. Defaults change, so check the current documentation. The ALB’s default idle timeout is 60 seconds.
Other features
- TLS termination decrypts traffic at the load balancer, so backend servers do not have to handle encryption.
- Sticky sessions send a client’s repeat requests to the same backend, usually using a cookie. They help applications that keep session state on one server, but they can concentrate traffic on a few servers.
- Connection draining lets existing connections finish before a server is removed from the group.
- Cross-zone balancing spreads traffic across availability zones.
- Internal or internet-facing placement decides whether the load balancer serves public traffic or only traffic inside a private network.
Managed options and self-managed software
Managed cloud load balancers scale automatically and avoid hardware upkeep. They are billed per hour, plus usage charges. AWS, for example, adds load balancer capacity unit (LCU) charges to the hourly rate. Self-managed options include the open-source NGINX, HAProxy, and Envoy. Hardware appliances, such as F5 BIG-IP, are used in some data centers.
Load balancers and autoscaling
Load balancers and autoscaling work together, but they do different jobs. An autoscaling group adds and removes instances. It can register and deregister them with the load balancer automatically. The load balancer only distributes traffic, so it does not add capacity by itself. The provisioning and scheduling article covers how autoscaling responds to demand, including its delays.
Common misconceptions
- A load balancer is not autoscaling. It spreads load across existing targets. Autoscaling adds targets.
- A Layer 4 load balancer cannot route by URL path. Path-based routing requires Layer 7.
- Managed load balancers are distributed, but a single-zone setup is still a weak point. Spread backends across at least two availability zones.
- Sticky sessions can unbalance traffic. Use them only when the application needs them.
When is a load balancer needed?
A single server handling low traffic may not need one. Load balancing becomes useful when you run several copies of an application, need to maintain servers without downtime, or expect uneven or growing traffic. Load balancers usually sit in the public subnet of a virtual private cloud, in front of private application servers.