Log rotation is the automated practice of closing the current log file, archiving it under a new name and starting a fresh file, combined with compression and deletion of old archives according to a retention policy. In server monitoring and logs, rotation keeps disk usage bounded, keeps individual files small enough to search, and enforces how long access logs, error logs and application logs are kept.
Without rotation, a busy web server can fill a disk with a single ever-growing log file, and a full disk then stops databases, mail queues and the application itself. Rotation is therefore a reliability measure as much as a housekeeping task.
How log rotation works step by step
A rotation cycle performs the same ordered sequence each time a trigger fires. The trigger is usually a schedule, such as daily, or a size threshold, such as 100 MB, and the most common tool on Linux servers is logrotate, run daily by cron or a systemd timer.
- Check the trigger: the tool compares file age or size with the configured condition.
- Rename the active file:
access.logbecomesaccess.log.1, and older archives shift up by one number or receive a date suffix. - Signal the writer: the service is told to reopen its log file, or the tool copies and truncates the file instead.
- Compress archives: older files are compressed, often with a one-cycle delay.
- Delete expired files: archives beyond the retention count or age are removed.
The third step is the delicate one. A process writes through an open file descriptor, not through a file name, so after a rename it keeps writing into the renamed file until it reopens its log. Services such as nginx reopen on SIGUSR1, while others need a reload or a dedicated command.
Rotation methods and their trade-offs
There are several ways to make a running service move to a new file, and each has different guarantees about lost or duplicated lines. The table compares the methods found on production servers.
| Method | How it works | Main trade-off |
|---|---|---|
| Rename and signal | Rename file, send a reopen signal or reload | Clean and lossless if the service supports it |
| Copy and truncate | Copy content, then truncate the original to zero | Works with any writer but can lose lines written during the copy |
| Application-managed rotation | Logging library rotates its own files | No external tool, but settings differ per application |
| Piped logging | Service writes to a helper that splits files by time | Flexible, adds a process to manage |
| Journal-based storage | systemd-journald manages size and retention itself | No file rotation, uses its own binary format |
The copytruncate option is popular because it needs no cooperation from the application, but it has a small race window and duplicates disk I/O. Where a service supports a reopen signal, rename and signal is the more reliable choice.
Key log rotation parameters
A rotation policy is defined by a handful of parameters that together determine disk usage, search convenience and how much history survives. The values below are typical starting points, not fixed rules.
| Directive | Typical value | Effect |
|---|---|---|
| daily / weekly | Daily for busy web logs | Time-based rotation trigger |
| size or maxsize | 50-500 MB | Rotates early when a file grows too fast |
| rotate | 7-90 archives | Number of old files kept |
| compress / delaycompress | Enabled | Saves space, keeps the newest archive readable |
| dateext | Enabled | Uses dates instead of numbers in archive names |
| missingok / notifempty | Enabled | Avoids errors and empty archives |
Text logs typically compress to a small fraction of their original size, often around a tenth, because they are highly repetitive. That ratio makes it practical to keep weeks of history locally, while long-term retention usually moves to a central log platform.
Log rotation compared with related logging practices
Rotation is one part of a log lifecycle, and it is often mixed up with retention and shipping. The distinctions matter when designing compliance policies or debugging missing data.
| Concept | Question it answers | Relation to rotation |
|---|---|---|
| Log rotation | When does a new file start | Local file management on the server |
| Retention policy | How long logs are kept | Rotation enforces it locally |
| Log shipping | Where logs are copied | Agents must handle rotated files correctly |
| Log aggregation | Where logs are searched | Central store has its own retention |
| Log sampling | How much is recorded | Reduces volume before rotation |
Log shipping agents follow files by inode and offset, so they detect a rename and continue reading the rotated file until its end. A misconfigured agent that follows only the file name may skip the last lines before each rotation.
Common log rotation mistakes
- Rotating without reopening: the service keeps writing into
access.log.1, which is then compressed or deleted, and new entries vanish. - Deleted but open files: a log removed while still open keeps consuming disk space until the process restarts, so
dfanddudisagree. - Retention set by habit: privacy law and security standards may require minimum or maximum retention periods, so the count should follow policy.
- Forgotten application logs: custom logs outside
/var/logare often missing from the rotation configuration and grow unchecked. - Wrong permissions on new files: the
createdirective must set the owner and mode the service expects, or logging fails silently.
In plain words
A server writes a diary of everything it does, and that diary never stops growing. Log rotation closes the current notebook at a set time, puts it on a shelf and opens a new one.
Older notebooks are shrunk to save space, and the oldest are thrown away after a set period. The server always has recent history at hand without letting the diary fill the whole room.
A real-world example
A hosting provider operates web servers that each write about 2 GB of access logs per day. Twice in one month a server ran out of disk space overnight, which stopped the database and took several customer sites offline.
The operations team configures logrotate for daily rotation, an additional size limit of 500 MB, 14 compressed archives, delaycompress and a postrotate command that sends nginx the reopen signal. Local log storage stabilizes at a predictable level, logs older than two weeks are available in the central log platform, and no further disk-full incidents occur.