Service discovery

On this page 7

Service discovery is the mechanism by which a client or another service finds the current network location of a service instance, typically an IP address and port, at the moment it needs to connect. Within Services & processes on servers, it replaces hard-coded hostnames in configuration files with a dynamic lookup that follows instances as they start, stop, move or fail.

The idea matters most where instances are short-lived: containers rescheduled across hosts, autoscaling groups that add and remove virtual machines, and blue-green deployments that swap whole fleets. In those environments a static address list becomes stale within minutes, and service discovery keeps routing accurate without manual edits.

How service discovery locates instances

Service discovery works through three cooperating parts: registration, a service registry and lookup. When an instance starts and binds its port, it is recorded in the registry under a logical name such as orders or payments-api, together with its address, port and metadata like version or availability zone.

Clients then ask the registry for healthy instances of that logical name instead of connecting to a fixed host. The registry answers with one or more endpoints, and the client or an intermediate proxy chooses one. Health checking keeps the answer honest: an instance that stops responding is marked unhealthy or removed, so traffic stops reaching it.

The table below shows the components and the role each one plays in the discovery loop.

ComponentRoleTypical implementation
Service registryStores live instances per service nameConsul, etcd, ZooKeeper, Kubernetes API
Registration agentAdds and removes instance entriesSidecar agent, orchestrator, client library
Health checkerMarks failing instances as unavailableHTTP, TCP or script checks every 5-30 s
ResolverTranslates a service name into endpointsDNS server, client library, proxy
Load balancerPicks one endpoint per requestClient-side logic or reverse proxy
Core components of a service discovery system

Registration and lookup patterns

Implementations differ in who registers an instance and who performs the lookup. In self-registration, the service process calls the registry at startup and sends periodic heartbeats; in third-party registration, an orchestrator or agent observes processes and registers them on their behalf, which keeps application code free of registry logic.

Lookup is either client-side, where a library queries the registry and balances load itself, or server-side, where the client calls a stable address on a load balancer or proxy that consults the registry. The following table compares these patterns.

PatternHow it worksMain advantageMain drawback
Client-side discoveryClient queries registry and selects instanceNo extra network hopLogic duplicated in every language
Server-side discoveryClient calls a proxy that resolves instancesClients stay simpleProxy becomes a critical component
DNS-based discoveryName resolves to A or SRV recordsWorks with almost any clientCaching delays updates
Service mesh sidecarLocal proxy handles lookup and routingUniform retries, TLS and metricsAdded latency and operational load
Discovery patterns and their trade-offs

DNS-based discovery deserves special attention because it is the most universal. SRV records carry both host and port, and a short time to live of 5-30 seconds keeps answers fresh. However, many runtimes and resolvers cache DNS results longer than the TTL, so a removed instance may still receive connections for a while.

Consistency, health and failure behavior

A registry is itself a distributed system, so its consistency model shapes how discovery behaves during network partitions. Registries built on consensus protocols such as Raft or Zab, including etcd, Consul servers and ZooKeeper, favor strong consistency and may refuse writes when a quorum is lost. Registries that favor availability keep serving possibly outdated data instead.

Neither choice is universally better. For discovery, a slightly stale answer is often preferable to no answer, which is why many clients cache the last known instance list and keep using it if the registry becomes unreachable. Typical settings that control this behavior are summarized below.

ParameterTypical rangeEffect of a lower valueEffect of a higher value
Health check interval5-30 sFaster failure detection, more loadSlower detection, less overhead
Deregistration timeout30 s-5 minQuicker cleanup, more flappingGhost entries linger longer
DNS record TTL0-60 sFresher answers, more queriesStale endpoints after changes
Client cache refresh10-60 sCloser to registry stateTolerates registry outages better
Key timing parameters in service discovery

Discovery also interacts with graceful shutdown. An instance should deregister before it stops accepting connections, then drain in-flight requests; otherwise clients keep sending traffic to a process that is already exiting, which produces bursts of connection errors during every deployment.

Service discovery overlaps with several neighboring concepts but solves a distinct problem: mapping a logical name to live endpoints. A load balancer distributes requests, a configuration store holds settings, and a process supervisor keeps processes running; discovery connects them by telling callers where the running processes are.

ConceptQuestion it answersRelationship to discovery
Static configurationWhere was the service when the file was writtenReplaced by dynamic lookup
Load balancingWhich instance should handle this requestConsumes the discovered endpoint list
Process supervisionIs the process running and restarted on crashProduces instances that discovery tracks
Service dependency mappingWhich services rely on which othersUses discovery names as identifiers
Configuration storeWhat settings should a service useOften shares the same backend, such as etcd
How service discovery differs from neighboring concepts

Common mistakes and operational risks

Most discovery incidents come from mismatched timing and from treating the registry as infallible. The most frequent problems are listed here.

  • Health checks that test too little: a TCP check passes while the application returns errors, so broken instances stay in rotation. A check should exercise a lightweight application path.
  • Health checks that test too much: a check that calls downstream databases can mark every instance unhealthy at once during a dependency outage, causing a cascading removal.
  • Ignoring client DNS caching: runtimes that cache resolved addresses indefinitely defeat short TTLs and keep connecting to retired hosts.
  • Running a single registry node: the registry becomes a single point of failure; consensus-based registries need three or five nodes to tolerate one or two failures.
  • Missing deregistration on shutdown: crashed or killed processes leave ghost entries until the timeout expires.

Security is the other frequent gap. An open registry lets any process register a rogue endpoint and intercept traffic, so production deployments should require authenticated registration and access control lists, and ideally mutual TLS between services.

A non-technical view

Imagine a large office where people change desks every day. Instead of memorizing where a colleague sits, you check a live directory at reception that is updated whenever someone moves, arrives or goes home.

Service discovery is that directory for server programs. When one program needs another, it asks the directory for the current address instead of relying on an old note. If a program stops working, it is taken off the list, so nobody keeps calling an empty desk. The key point is simple: programs look each other up by name, and the system keeps the addresses current.

A concrete example

An online retailer runs its checkout on a container platform with twelve instances of an inventory service spread across four hosts. During a sales event, autoscaling raises the count to thirty within ten minutes, and several instances are rescheduled when one host runs out of memory.

The checkout service never stores host addresses. It resolves inventory through the platform registry, which returns only instances whose HTTP health check at /healthz passed within the last ten seconds. When the overloaded host fails, its instances are removed in about fifteen seconds, and new ones appear in the list as soon as they report healthy.

During the next deployment, each old instance deregisters first, drains requests for twenty seconds and then exits. The result is a release with no visible checkout errors, even though every address behind the inventory name changed during the day.