A load balancer is a network device or software service that receives client traffic at a single address and distributes connections or requests across a pool of backend servers according to an algorithm and their health. In networking, it is the core component of load balancing: it lets many servers act as one scalable, highly available service while hiding individual failures from clients.
Load balancers range from dedicated hardware appliances to software such as HAProxy, NGINX and Envoy, and to managed services offered by every major cloud provider. They differ mainly in which network layer they work at and therefore in what information they can use to make decisions.
How a load balancer distributes traffic
A load balancer publishes a virtual IP address (VIP) and port that clients connect to. For each new connection or request, it selects a backend from the server pool using a balancing algorithm, forwards the traffic and returns the responses. Clients see only the VIP, so servers can be added, removed or replaced without changing anything on the client side.
To avoid sending traffic to a failed server, the load balancer runs health checks. These range from simple TCP connection attempts to HTTP requests that expect a specific status code or body. A server that fails a configured number of consecutive checks, commonly two or three at intervals of 5 to 10 seconds, is removed from rotation and returned automatically once it recovers.
The forwarding method determines the return path. In full proxy mode, the load balancer terminates the client connection and opens a separate one to the backend, so responses flow back through it. In direct server return, the backend replies straight to the client, which saves load balancer capacity for traffic with small requests and large responses, such as video streaming.
Layer 4 and layer 7 load balancing
The most important distinction is the OSI layer at which decisions are made. A layer 4 load balancer sees only IP addresses, ports and the transport protocol, so it forwards whole TCP or UDP connections without reading their content. A layer 7 load balancer parses the application protocol, typically HTTP, and can route each request by host name, URL path, header or cookie.
| Property | Layer 4 | Layer 7 |
|---|---|---|
| Decision basis | IP addresses, ports, protocol | Host, path, headers, cookies, method |
| Unit of balancing | Connection or flow | Individual request |
| TLS handling | Usually passes encrypted traffic through | Terminates TLS to inspect requests |
| Performance cost | Very low, high throughput | Higher CPU use per request |
| Protocol support | Any TCP or UDP protocol | HTTP, HTTP/2, gRPC and similar |
| Typical features | Fast failover, connection distribution | Content routing, header rewriting, web application firewall |
Many architectures use both in sequence: a layer 4 tier spreads raw connections across several layer 7 proxies, which then route individual requests to microservices. Layer 7 balancing also matters for long-lived HTTP/2 or gRPC connections, where a layer 4 device would pin all requests of one connection to a single backend.
Load balancing algorithms
The balancing algorithm decides which backend receives the next connection or request. Round robin is the simplest and works well when servers and requests are uniform; other methods take server capacity, current load or client identity into account.
| Algorithm | How it selects a server | Best suited for |
|---|---|---|
| Round robin | Next server in a fixed rotation | Identical servers, short uniform requests |
| Weighted round robin | Rotation in proportion to assigned weights | Servers with different capacities |
| Least connections | Server with the fewest active connections | Long or variable session durations |
| Least response time | Fastest server combined with low load | Latency-sensitive services |
| Source IP hash | Hash of the client address maps to a server | Simple affinity without cookies |
| Consistent hashing | Hash ring that remaps few keys when servers change | Caches and sharded data |
When an application stores session state in server memory, the load balancer must apply session persistence, also called a sticky session, so that a client keeps reaching the same backend. Persistence improves compatibility but weakens even distribution and complicates failover, so storing session state in a shared cache or database is usually the better design.
Deployment options and high availability
A single load balancer would itself become a single point of failure, so production deployments run at least two instances in a redundant pair. In active-passive mode, the standby takes over the VIP within seconds through protocols such as VRRP; in active-active mode, both handle traffic and each must keep enough headroom to carry the full load alone.
| Form | Characteristics | Typical setting |
|---|---|---|
| Hardware appliance | Dedicated chips, very high throughput, high cost | Large data centers, service providers |
| Software load balancer | Runs on standard servers or containers, flexible | Private clouds, Kubernetes ingress |
| Cloud managed service | Elastic capacity, billed by usage | Applications hosted in public clouds |
| Global server load balancing | DNS or anycast steering between regions | Multi-region availability and disaster recovery |
Global server load balancing works at a different scale: it chooses a data center or region for each client, typically by returning different DNS answers, while local load balancers in each site choose individual servers.
Common mistakes and limitations
- Shallow health checks that only test whether a port is open keep sending traffic to servers whose application or database connection has failed.
- Lost client IP addresses in proxy mode break logging and access control unless the load balancer adds an X-Forwarded-For header or uses the PROXY protocol.
- Undersized redundant pairs in active-active mode collapse when one unit fails, because the survivor cannot carry the combined load.
- Overly aggressive timeouts cut long uploads, WebSocket sessions or slow reports in the middle.
- Load balancing does not fix a bottleneck behind the servers: if every backend waits on the same database, adding servers adds little capacity.
Understanding it simply
A load balancer works like the host at the entrance of a busy restaurant. Guests do not choose a waiter themselves; the host sends each new party to a table whose waiter has capacity and skips any section that is closed.
For a website, the guests are visitors and the waiters are servers. Visitors always come to the same front door, while the load balancer quietly spreads them across many machines. The main benefit is that one server can fail without anyone noticing, and more servers can be added when traffic grows.
Applied in practice
An online ticket seller expects demand to jump roughly tenfold when concert tickets go on sale at 10:00. Its web application runs on six identical application servers behind a redundant pair of layer 7 load balancers that terminate TLS and route requests by URL path: /api/ requests go to the application pool, while images and scripts go to a separate static content pool.
The team selects least connections for the application pool, because checkout requests vary widely in duration, and configures HTTP health checks against a /health endpoint that also verifies the database connection, every 5 seconds with removal after 3 failures. Session data is moved to a shared in-memory cache, so sticky sessions are not required.
Before the sale, the team adds six more servers to the pool without any DNS change. During the peak, one server runs out of memory and fails its health checks; the load balancers remove it within about 15 seconds and customers see no failed checkouts.