Throughput is the rate at which a server completes useful work, measured as units of work per unit of time, for example requests per second, transactions per second or megabytes per second. In server performance, it describes capacity under load: how much traffic a web server, database, disk or network link can actually process, as opposed to how quickly a single request finishes.
Throughput is always read together with latency and error rate. A server that handles 5,000 requests per second but answers half of them after ten seconds, or with errors, does not deliver useful throughput, so practitioners usually speak of goodput, the rate of successful work within an acceptable response time.
How throughput relates to latency and concurrency
The link between throughput, latency and concurrency is captured by Little’s law: the average number of requests in the system equals throughput multiplied by average response time. Rearranged, throughput equals concurrency divided by latency. A service holding 200 requests in progress with an average response time of 0.25 seconds therefore completes 200 ÷ 0.25 = 800 requests per second.
This relationship explains why lower latency raises throughput for a fixed number of workers, and why adding concurrency helps only while some resource still has spare capacity. Once the bottleneck resource is fully used, extra concurrency only lengthens queues and response times.
| Metric | What it measures | Typical unit | Question it answers |
|---|---|---|---|
| Throughput | Completed work per unit of time | Requests/s, MB/s, TPS | How much can the system process? |
| Latency | Time to complete one operation | Milliseconds | How fast does one request finish? |
| Bandwidth | Theoretical maximum transfer rate of a link | Gb/s | What is the upper limit of the channel? |
| Concurrency | Operations in progress at the same moment | Count | How much work is in flight? |
| Utilization | Share of time a resource is busy | Percent | How close is the resource to its limit? |
Units of throughput across server components
Each layer of a server has its own throughput unit, and the end-to-end rate is limited by the slowest one. Comparing them requires care with bits versus bytes and with decimal versus binary prefixes.
| Component | Throughput unit | Example conversion or note |
|---|---|---|
| Network link | Bits per second | 1 Gb/s = 125 MB/s before protocol overhead |
| Disk or SSD | MB/s and IOPS | 10,000 IOPS × 4 KiB blocks ≈ 41 MB/s |
| Web or API server | Requests per second | Depends heavily on payload and backend calls |
| Database | Transactions or queries per second | Writes are usually far costlier than reads |
| Message queue | Messages per second | Batching raises throughput at some latency cost |
Real network throughput stays below nominal bandwidth because of Ethernet framing, IP and TCP headers, encryption and retransmissions. A healthy 1 Gb/s link typically delivers roughly 110 to 118 MB/s of file transfer payload, not the full 125 MB/s.
What limits server throughput
The ceiling of a server is set by its most saturated resource. For a PHP application served by a worker pool, the maximum rate is roughly the number of workers divided by the average service time: 32 PHP workers that each spend 80 ms per request can complete at most 32 ÷ 0.08 = 400 requests per second, no matter how many visitors arrive.
- CPU: computation-heavy code, template rendering, compression and TLS handshakes.
- Worker and connection pools: a fixed number of PHP workers or database connections caps parallel work.
- Disk I/O: random writes, fsync calls and logging on slow storage.
- Network: link speed, packet loss and high round-trip times for chatty protocols.
- Locks and contention: hot database rows, global mutexes and serialized sections of code.
Queueing theory adds an important warning: as utilization approaches 100%, response time rises steeply while throughput barely grows. In a simple single-server queue model, mean response time equals service time divided by (1 − utilization), so at 90% utilization a request takes about ten times its bare service time. This is why capacity plans usually keep critical resources well below saturation.
Measuring and improving throughput
Throughput is measured with load tests that raise the offered load step by step while recording completed requests, latency percentiles and errors. The useful result is the knee of the curve, the load at which throughput stops rising and p95 or p99 latency starts to climb.
| Situation | Recommended measure | Reason |
|---|---|---|
| CPU saturated by PHP execution | Enable OPcache, profile hot code paths | Less work per request |
| Database connection waits | Tune the connection pool, add read replicas | Removes waiting for a scarce resource |
| Repeated identical responses | Add page, object or CDN caching | Requests skip the backend entirely |
| Many small writes | Batch writes and use asynchronous queues | Fewer round trips and fsync calls |
| Single node at its limit | Scale horizontally behind a load balancer | Adds parallel capacity |
Reported throughput figures are meaningful only with their test conditions: request mix, payload size, concurrency, cache state and latency target. Benchmarks that hit a single cached URL can overstate real capacity by an order of magnitude or more.
Common misconceptions about throughput
- Throughput is not speed for one user. A system can process many requests per second while each request is slow.
- Bandwidth is not throughput. A 10 Gb/s link carrying data from a slow disk transfers only what the disk can supply.
- Average throughput hides bursts. Traffic peaks within a minute can exceed the hourly average many times over.
- More workers do not always help. Beyond the bottleneck’s capacity, extra workers add contention and memory pressure.
- Peak benchmark numbers are not sustainable capacity. Thermal throttling, cache eviction and log growth reduce long-run rates.
Stripped to the essentials
Throughput is how much work a server gets done in a given time, such as how many web pages it can deliver each second.
Picture a supermarket. How quickly one customer gets through the checkout is latency, while how many customers all the checkouts serve per hour is throughput. Opening more checkouts raises throughput, but only until the store runs out of staff or space, and then the lines simply get longer.
Where it is applied
A media publisher expects 1,200 requests per second during a live news event. Its WordPress servers each run 32 PHP workers with an average service time of 80 ms, so one server can complete at most about 400 dynamic requests per second. A load test confirms the knee at about 350 requests per second per server, where p95 latency starts to rise sharply.
The team adds full-page caching, which serves about 85% of requests without PHP, leaving roughly 180 dynamic requests per second. Two servers, each running at about a quarter of its measured knee, handle the event comfortably with one server’s worth of spare capacity, and p95 latency stays under 300 ms.