Connection draining

On this page 7

Connection draining is the load balancing mechanism that stops sending new connections or requests to a backend while allowing the connections it already holds to complete gracefully within a defined timeout. In networking and application delivery, it is the standard way to remove a server from a pool for maintenance, scaling in or deployment without dropping in-flight requests.

Vendors use different names for the same idea: deregistration delay, graceful shutdown, drain mode or connection quiescing. The underlying contract is identical: the backend is marked as leaving, the balancer stops choosing it, and the remaining sessions are given a bounded amount of time to finish before they are forcibly closed.

How connection draining works

Connection draining works by changing a backend from an active state to a draining state in which the load balancer excludes it from new selection decisions but keeps forwarding packets for existing flows. The balancer continues to track the connection table or session state for that backend until each flow ends or the drain timer expires.

The sequence is predictable. An operator, an autoscaling group or an orchestrator triggers deregistration; the balancer stops including the target in its algorithm, whether that is round robin, least connections or consistent hashing; existing TCP connections and HTTP requests continue; and when the drain timeout is reached, any remaining connections are reset or closed. Only then is the backend reported as fully removed.

The following table summarizes the backend states that most implementations expose during this process.

StateNew connectionsExisting connectionsTypical trigger
ActiveAcceptedServed normallyHealthy and registered
DrainingRefused or reroutedAllowed to finishDeregistration or scale-in
DrainedRefusedNone remainingAll flows ended
Forced removalRefusedReset at timeoutDrain timer expired
Backend states during a drain

Draining at layer 4 and layer 7

The behavior of connection draining depends on which layer the balancer operates at, because a layer 4 device sees TCP or UDP flows while a layer 7 proxy sees individual HTTP requests. This difference determines what counts as finished work.

In layer 4 load balancing, draining keeps existing five-tuple flows pinned to the leaving backend until the client or server closes them. A long-lived TCP session, such as a database connection pool, may therefore hold the backend for the entire timeout. In layer 7 load balancing, the proxy can be smarter: it can finish the current request, then send Connection: close in HTTP/1.1 or a GOAWAY frame in HTTP/2, prompting clients to reconnect elsewhere at a clean boundary.

Traffic typeUnit of workGraceful signalMain difficulty
Short HTTP/1.1 requestsSingle requestConnection: close headerKeep-alive reuse
HTTP/2 and gRPCStream on shared connectionGOAWAY frameMany streams per connection
WebSocketLong-lived sessionApplication close messageSessions lasting hours
Raw TCPEntire connectionFIN from either sideNo protocol-level hint
UDPPseudo-flow by tupleNoneIdle timeout only
Draining behavior by protocol and layer

UDP has no connection semantics, so draining usually means keeping tuple affinity until an idle timer expires. Sticky sessions add another dimension: a draining backend may still receive requests from clients whose session cookie points to it, depending on whether the balancer honors affinity during the drain.

Choosing the drain timeout

The drain timeout should be slightly longer than the longest legitimate request the service handles, and no longer. A timeout that is too short cuts off uploads, reports or streaming responses; a timeout that is too long slows deployments and scale-in and keeps paying for capacity that serves almost nothing.

Typical defaults range from 30 to 300 seconds, and some cloud balancers allow up to one hour. Engineers should measure the 99th or 99.9th percentile request duration from access logs before choosing a value, then add a safety margin of about 20 to 50 percent.

WorkloadTypical durationSuggested timeoutReason
REST APIsUnder 2 s10 to 30 sRequests finish quickly
Web pages and assetsUnder 5 s30 to 60 sKeep-alive tails are short
File uploads and exportsSeconds to minutes120 to 600 sLarge bodies take time
WebSocket or streamingMinutes to hours300 s plus app reconnectTimeout alone cannot cover it
Batch or database TCPVariableMatch pool recycle timePools hold idle connections
Typical drain timeout ranges by workload

For long-lived sessions, a timeout is not enough on its own. The application must cooperate by closing sessions voluntarily, ideally with randomized delays so that thousands of clients do not reconnect at the same instant and create a reconnection storm on the remaining backends.

How draining fits deployments and autoscaling

Connection draining is the foundation of zero-downtime operations, because rolling updates, blue-green deployments and scale-in events all remove backends while traffic flows. Without it, every removal produces a burst of HTTP 502 or connection reset errors.

In container orchestrators, draining must be coordinated with process shutdown. A common pattern is a preStop delay: the pod is first removed from service endpoints, the process keeps serving for several seconds while the balancer converges, and only then does it receive SIGTERM and stop accepting work. The application should also fail its readiness check as soon as shutdown begins, so health-check-driven balancers stop routing to it.

  • Rolling deployment: each old instance drains before termination, so capacity never drops to zero.
  • Blue-green switch: the old environment drains after traffic moves, then is retired.
  • Autoscaling scale-in: the scaling group waits for the drain to finish before terminating the instance.
  • Maintenance window: an operator sets drain mode manually, patches the server and returns it to active.

Limitations and common mistakes

Connection draining guarantees only that the balancer stops choosing the backend; it does not guarantee that clients stop using it. Several failure modes appear repeatedly in production, and most of them result from mismatched timers between layers.

SymptomLikely causeFix
Errors at the end of every deployProcess exits before drain endsAlign shutdown grace period with drain timeout
Errors at the start of every deployProcess stops before balancer convergesAdd a preStop delay of a few seconds
Deployments take far too longTimeout much longer than requestsBase timeout on measured percentiles
Backend never emptiesPersistent keep-alive or WebSocket clientsSend close signals from the application
Load spike on remaining serversAll clients reconnect at onceStagger closes and drain fewer nodes per step
Draining problems, causes and fixes

A frequent misconception is that health checks alone provide draining. A failing health check removes a backend after several intervals, and many balancers then reset existing connections immediately. Proper draining is an explicit, separate state with its own timer, and it should be used whenever a removal is planned rather than caused by failure.

Put simply

Connection draining is like closing a checkout lane at a supermarket. The cashier puts up a lane closed sign so no new shoppers join the line, but everyone already waiting is still served. Only when the last customer has paid does the cashier leave.

A load balancer does the same with servers. It stops sending new visitors to a server that is about to be switched off, lets the current visitors finish what they are doing, and then removes the server. The result is that people using the website never notice the change.

In practice

An online retailer runs its checkout API on twelve containers behind a layer 7 load balancer and deploys new versions several times a day. Before enabling draining, each rolling update produced a few hundred failed payment requests, because containers were terminated while still processing orders.

The platform team measured request durations and found that the 99.9th percentile was 8 seconds, driven by calls to an external payment gateway. They set the deregistration delay to 15 seconds, added a 5-second preStop pause, raised the container shutdown grace period to 30 seconds and made the service fail its readiness probe on SIGTERM.

After the change, deployments replaced two containers at a time, each one draining before shutdown. Failed checkout requests during releases fell to effectively zero, and a full rollout still completed in under three minutes.