Process limit

On this page 7

A process limit is a cap on the number of processes or threads that a user, service, container or hosting account may run at the same time. In servers and server resources, it protects the whole machine from runaway workloads, fork bombs and misbehaving applications that would otherwise exhaust process table slots, memory and CPU time.

Process limits exist at several layers: the kernel-wide maximum, per-user limits set with ulimit or PAM, cgroup limits for services and containers, and per-account limits on shared hosting. When a limit is reached, new processes fail to start while existing ones keep running.

How process limits are enforced

Every new process or thread on Linux is created through the fork() or clone() system calls. Before creating it, the kernel checks each applicable limit. If any limit is exceeded, the call fails with the error EAGAIN, which programs usually report as Resource temporarily unavailable or cannot fork.

On Linux, threads count as tasks just like processes. A Java application with 2,000 threads therefore consumes 2,000 slots against a per-user or cgroup limit, even though it appears as a single process in many monitoring tools. This is the most common source of confusion about process limits.

The table summarizes where process limits are configured and what each one controls.

LayerSettingScope
Kernel PID spacekernel.pid_maxHighest process ID, default 32768 or 4194304 on large systems
Kernel task countkernel.threads-maxTotal tasks the kernel will create
Per-user limitRLIMIT_NPROC via ulimit -u or limits.confAll tasks owned by one user
systemd serviceTasksMaxAll tasks in one service cgroup
cgroup v2pids.maxTasks in a container or slice
Hosting accountPanel or CloudLinux NPROC and EP limitsOne customer on a shared server
Layers of process limits on Linux

Process limits on shared hosting

Shared hosting providers use process limits to keep one account from starving the others. With CloudLinux, two related values are common, and they are often confused.

LimitWhat it countsTypical valueSymptom when reached
Entry processes (EP)Concurrent web requests entering the accountAbout 10 to 30HTTP 508 resource limit reached
Number of processes (NPROC)All processes and threads of the accountAbout 50 to 100500 errors, failed cron jobs, SSH refusals
Entry processes and number of processes compared

A site that hits its entry process limit usually has slow pages, not too many visitors. If each PHP request takes 2 seconds, 20 entry processes support only about 10 requests per second; making pages faster raises capacity more than raising the limit.

Process limits compared with other resource controls

A process limit controls how many tasks exist, not how much work they do. It is one of several controls that together keep a server stable.

ControlWhat it limitsFailure mode when exceeded
Process limitNumber of processes and threadsNew tasks fail to start
CPU quotaCPU time per periodTasks are throttled and run slower
Memory limitResident memory of a groupReclaim, swap, then the OOM killer
Open files limitFile descriptors per processToo many open files errors
I/O limitDisk bandwidth or IOPSTasks wait on storage
Process limits and related resource controls

These controls interact. CPU utilization and load average often rise sharply when many processes compete, and a high process count with large memory per process can push the system into swap or trigger the OOM killer. A sensible process limit is therefore part of memory planning as well.

Setting sensible values

A good process limit is high enough for normal peaks and low enough to stop a runaway. The right value comes from measuring real usage, then adding headroom.

SituationRecommended approachReason
Interactive users on a shared serverLimit of a few hundred tasksStops fork bombs without hindering normal work
Thread-heavy services such as JVMsRaise TasksMax and NPROC well above peak thread countThreads count as tasks
ContainersSet pids.max per containerPrevents one container from exhausting the host
Worker pools such as PHP-FPMSize the pool by memory, then set limits above itPool size is the real concurrency control
Build servers and CI runnersGenerous limits with monitoringParallel compilers spawn many short tasks
Guidance for choosing process limits

Current usage can be measured with ps -eLf | wc -l for all tasks, ps -u USER -L for one user, or the pids.current file in a cgroup. Comparing peak usage over several weeks with the limit shows how much headroom really exists.

Common mistakes and symptoms

Process limit problems are often misread because the error messages are vague. The following patterns appear repeatedly in server operations.

  • Unable to create native thread in Java logs usually means the per-user or cgroup task limit is reached, not that memory is exhausted.
  • Limits set in limits.conf do not apply to systemd services, which use TasksMax and LimitNPROC in the unit file instead.
  • Raising limits without fixing a leak only delays failure; a steadily growing thread count points to an application bug.
  • Zombie processes still occupy PID slots until their parent reaps them, so a buggy parent can exhaust the limit.
  • Setting a limit of unlimited everywhere removes the protection that limits exist to provide.

The idea in brief

A server can run many programs at the same time, and each running program is called a process. A process limit is a rule about how many of them one person or service may run at once.

It works like a restaurant that seats each party at a fixed number of tables. One very large group cannot take every table and leave other guests waiting outside. Everyone keeps getting served, even when one guest misbehaves.

A practical scenario

A company runs a Java order service under systemd on a Linux server with 32 GB of memory. After a busy weekend, the service crashes with the error unable to create native thread, although monitoring shows 12 GB of free memory.

The administrator checks the service cgroup and finds pids.current at the service ceiling of TasksMax 4,915, the systemd default of 15 percent of the kernel PID maximum on that machine. A slow downstream API had caused the thread pool to grow without bound. The team caps the application thread pool at 400, adds a timeout for downstream calls and sets TasksMax to 2,000 as a safety net. The service has since run through several peak periods with thread counts below 600, and a runaway would now be contained instead of crashing the service.