A cloud load balancer is a managed service that distributes incoming network traffic across a pool of backend targets, such as virtual machines, containers or serverless functions, and sends requests only to targets that pass health checks. In cloud networking, it is the standard entry point that turns many interchangeable instances into one reliable, scalable endpoint.
Unlike a hardware appliance, a cloud load balancer is provisioned through an API, scales its own capacity automatically and is billed by usage. The provider operates the underlying fleet, so the customer configures listeners, rules and target groups rather than servers.
How a cloud load balancer routes traffic
A cloud load balancer receives traffic on a listener, a combination of protocol and port such as HTTPS on 443. Routing rules then select a target group, and a balancing algorithm picks one healthy target in that group. Within cloud networking, the targets usually sit in private subnets of a VPC, while only the load balancer is exposed.
Health checks probe each target at a fixed interval, commonly every 5 to 30 seconds, and mark it unhealthy after a configured number of consecutive failures, typically 2 to 5. Unhealthy targets stop receiving new requests until they recover. When a target is removed during deployment, connection draining, also called deregistration delay, lets in-flight requests finish, usually within 30 to 300 seconds.
Capacity is spread across multiple availability zones. If one zone fails, the load balancer continues serving from the remaining zones, which is why the service is a core building block of highly available architectures.
Layer 4 and layer 7 load balancers
The most important design choice is the OSI layer at which the load balancer operates. A layer 4 load balancer forwards TCP or UDP connections based on addresses and ports, while a layer 7 load balancer terminates HTTP and understands paths, headers, cookies and host names.
| Type | Operates on | Typical strengths | Typical limits |
|---|---|---|---|
| Network (layer 4) | TCP, UDP, TLS passthrough | Very low latency, static addresses, huge connection counts | No content-based routing |
| Application (layer 7) | HTTP, HTTPS, gRPC, WebSocket | Path and host routing, TLS termination, header rewriting | Higher per-request processing cost |
| Global (anycast) | HTTP or TCP at edge locations | One address worldwide, nearest-region routing | More complex failover testing |
| Internal | Private addresses only | Service-to-service traffic inside a VPC | Not reachable from the internet |
Regional load balancers serve one region and suit most applications. Global load balancers advertise a single anycast address from many edge locations and direct users to the closest healthy region, which supports multi-region failover without changes to DNS records.
Algorithms and key parameters
The balancing algorithm determines how evenly load spreads. Round robin suits uniform requests; least outstanding requests suits workloads with variable request durations; consistent hashing keeps a client on the same target, which helps caching but can create hot spots.
Several parameters have a direct effect on availability and user experience. The table lists the settings that are most often tuned and their usual ranges.
| Parameter | Common range | What it affects |
|---|---|---|
| Health check interval | 5-30 s | Speed of detecting a failed target |
| Unhealthy threshold | 2-5 failed checks | Balance between fast failover and false alarms |
| Idle timeout | 60-4,000 s, provider-dependent | Long-lived connections such as WebSocket |
| Deregistration delay | 30-300 s | Clean deployments without dropped requests |
| Session affinity | Off, cookie or source address | Statefulness versus even distribution |
| Cross-zone balancing | On or off | Even load versus inter-zone data charges |
TLS termination at a layer 7 load balancer centralizes certificate management and lets the load balancer inspect requests. When end-to-end encryption is required, traffic is re-encrypted to the targets or passed through untouched at layer 4.
Cloud load balancer compared with related services
A cloud load balancer handles inbound traffic to a pool of targets, which separates it from neighboring cloud networking services that also sit in the traffic path. Confusing them leads to architectures that work in testing but fail under load or during an outage.
| Service | Primary job | Relation to a load balancer |
|---|---|---|
| Cloud load balancer | Distribute inbound requests across healthy targets | Reference point |
| DNS-based balancing | Return different addresses per query | Slower failover because of record caching |
| API gateway | Authentication, quotas, request transformation | Often placed in front of or behind it |
| Cloud NAT | Outbound internet access for private resources | Handles the opposite traffic direction |
| Private endpoint | Private access to a service across networks | Often backed by an internal load balancer |
Security groups complement the load balancer: a common pattern allows backend instances to accept traffic only from the load balancer’s security group, so targets cannot be reached directly from the internet.
Common mistakes and limitations
Most load balancer incidents come from misconfigured health checks rather than from the service itself. A check that tests only whether a port is open marks a broken application as healthy; a check that tests every downstream dependency can mark the whole fleet unhealthy when one database slows down.
- Shallow health checks route users to instances that return errors.
- Overly deep health checks cause cascading failure during a partial outage.
- Sticky sessions without need concentrate load and slow scale-in.
- Short idle timeouts break long-running uploads and streaming connections.
- Ignoring cross-zone data charges produces unexpected networking costs.
- Missing client address headers hide the real client address from logs at layer 7.
Scaling is automatic but not instantaneous. A sudden spike of many times normal traffic can arrive faster than the provider adds capacity, so some platforms offer pre-warming or capacity reservations for planned events, and load testing before launches remains essential.
The idea in brief
Think of a busy supermarket with many checkout lanes and one staff member at the entrance who points each customer to a free lane. If a lane closes, nobody is sent there anymore.
A cloud load balancer does this job for a website or app. It sends each visitor to a working server, skips servers that have problems and lets the team add more servers when the crowd grows. Visitors simply see one address that keeps working.
A practical scenario
An online ticketing service runs its web application on 12 container instances spread over three availability zones. An application load balancer terminates HTTPS, routes requests under the checkout path to a dedicated target group and sends all other traffic to the general web group.
Health checks call a lightweight status endpoint every 10 seconds and remove an instance after 3 failures. During a release, each old instance drains for 60 seconds while new ones join. When one zone loses power on a busy evening, the load balancer shifts traffic to the two remaining zones within seconds, and autoscaling adds instances there. Customers see slower pages for a short time, but no checkout requests are lost.