Worker process

On this page 7

A worker process is an operating system process that a master or parent process starts to perform the actual work of a server application, such as handling network requests, executing scripts or processing queued jobs. On servers, the master usually reads configuration, binds privileged ports and manages the lifecycle, while a pool of workers does the request handling in parallel.

The model appears in web servers such as NGINX and Apache httpd, in application servers such as PHP-FPM, Gunicorn and Puma, and in job runners. It turns one long-running service into a supervised group of replaceable units, which is the basis for graceful restarts, crash isolation and controlled resource use.

How the master and worker model operates

The master process starts first, often as root, opens the listening sockets and then forks the workers, which typically drop privileges to an unprivileged account. Because a forked child inherits open file descriptors, every worker can accept connections on the same socket without the master forwarding traffic.

After startup the master rarely touches requests. It monitors its children, replaces any worker that exits unexpectedly, reacts to signals such as SIGHUP or SIGUSR2 and reopens log files. Each worker has its own process ID and its own memory space, so a crash or memory leak in one worker does not corrupt the others.

The following table shows how responsibilities are usually divided between the two roles.

ResponsibilityMaster processWorker process
ConfigurationReads and validates itReceives a copy at fork time
Listening socketsBinds ports, including 80 and 443Accepts connections on inherited sockets
PrivilegesOften runs as rootRuns as an unprivileged user
Request handlingNone or minimalParses, executes and responds
LifecycleSpawns, monitors and replaces workersExits on signal, limit or error
Division of duties in a master and worker server

Worker concurrency models

A worker process can serve one request at a time or many concurrent connections, depending on its internal design. This choice determines how many workers a server needs and how memory scales with traffic.

ModelConcurrency per workerTypical examplesMain trade-off
Prefork, one request per processOnePHP-FPM, Apache prefork MPMSimple and isolated, but memory-heavy
Threaded workersSeveral to dozens of threadsApache worker MPM, PumaLess memory, but threads share failures
Event-driven workersThousands of connectionsNGINX, Apache event MPMVery efficient, but blocking code stalls the loop
Job workersOne job or a small batchQueue consumers, cron runnersDecoupled from requests, needs a queue
Common designs for worker concurrency

For event-driven servers the usual rule is one worker per CPU core, because each worker already multiplexes many connections. For prefork and blocking application servers, the worker count is limited mainly by available memory divided by per-worker memory, since most workers spend time waiting on databases or external services.

Key parameters for sizing worker processes

Correct sizing keeps the server busy without pushing it into swap or out-of-memory kills. The table lists parameters that most worker-based servers expose under different names.

ParameterTypical rangeWhat it affects
Worker count1 per core (event) to 5-200 (prefork)Parallelism and total memory use
Memory per workerAbout 10 MB to over 300 MBMaximum safe worker count
Maximum requests per worker500 to 10,000, or unlimitedRecycling that contains slow memory leaks
Request timeout30 to 300 secondsHow long a stuck worker blocks a slot
Spare or idle workersA few to about 20 percent of the poolReadiness for traffic spikes
Typical worker settings and what they control

A practical starting formula is usable RAM divided by the measured average worker size, reduced by a safety margin of 10 to 20 percent for the operating system, caches and the database. The measured resident memory of real workers under load matters more than any default value.

A worker process is often confused with neighboring concepts in services and processes. The distinctions below clarify who starts what and who restarts whom.

ConceptRoleRelation to a worker process
DaemonLong-running background serviceThe master of a daemon commonly spawns workers
Process supervisorExternal manager such as systemdRestarts the whole service, not individual workers
ThreadExecution unit inside one processShares memory; a worker may contain many threads
Graceful restartReload without dropping connectionsAchieved by replacing workers one generation at a time
Worker process and neighboring terms

Common mistakes with worker processes

Most production incidents involving workers come from sizing errors and blocking behavior, not from the model itself.

  • Too many workers for the RAM causes swapping and kernel out-of-memory kills, which is slower than queuing requests.
  • Too few workers lets slow upstream calls occupy every slot, so new requests wait even though the CPU is idle.
  • Blocking calls in an event loop freeze every connection that the affected worker holds.
  • No request recycling lets small memory leaks grow until workers reach several times their normal size.
  • Per-worker state such as local caches or sessions breaks when the next request reaches a different worker.

Explained without jargon

Think of a busy restaurant. The manager opens the doors, assigns tables and replaces anyone who goes home sick, but the waiters do the actual serving. On a server, the manager is the master process and the waiters are the worker processes.

If one waiter drops a tray, the others keep working and the manager hires a replacement. Hiring more waiters helps only while there is room in the kitchen, which is why the number of workers must match the server memory.

A typical use case

An online shop runs PHP-FPM on a virtual server with 8 GB of RAM. Measurements show each PHP worker uses about 80 MB under normal load. After reserving roughly 3 GB for the database, the operating system and caches, the administrator sets the pool to 60 workers and recycles each one after 1,000 requests to contain a known plugin memory leak.

During a seasonal sale, request volume doubles. Because the pool is capped, extra requests wait briefly in the listen queue instead of starting new processes, the server never swaps, and response times rise moderately rather than collapsing. The result is a stable shop under peak load with predictable memory use.