rsync

On this page 7

rsync is an open-source file synchronization and transfer utility that copies files between directories or hosts while sending only the differences between source and destination. Backup & recovery workflows on Linux and Unix servers rely on it heavily, because its delta-transfer algorithm keeps network traffic and run times small even when the protected data set is large.

Unlike a dedicated backup application, rsync produces a plain mirror of files rather than a catalog of versions. That simplicity makes restores trivial, but it also means that retention, versioning and protection against deletion must be designed around it rather than assumed.

How the rsync delta-transfer algorithm works

The rsync algorithm avoids resending whole files by comparing block checksums on both sides of the connection. The receiver splits its existing copy of a file into fixed-size blocks and sends a weak rolling checksum and a strong hash for each block to the sender.

The sender then slides a window over its version of the file, byte by byte, and uses the cheap rolling checksum to find candidate matches quickly. Only when the weak checksum matches does it confirm the block with the strong hash. Matching regions are transmitted as short block references, and only unmatched bytes travel as literal data. The receiver rebuilds the new file from its old blocks plus these literals, normally into a temporary file that replaces the original atomically.

Before any of this happens, rsync must decide which files need attention at all. By default it uses a quick check of size and modification time; files whose size and mtime match are skipped entirely. The table below shows the main stages of a typical run.

StageWhat happensMain cost
File list buildingBoth sides walk the directory tree and exchange metadataDisk metadata reads, memory
Quick checkSize and mtime decide which files are candidatesVery low
Checksum exchangeReceiver sends block checksums for changed filesReading old files
Delta generationSender matches blocks and emits literalsCPU and source disk reads
ReconstructionReceiver writes a temporary file and renames itDestination disk writes
Stages of an rsync transfer

Transport modes and key options

rsync can run locally, over a remote shell or against an rsync daemon. In practice most server backups use SSH as the transport, which provides encryption and key-based authentication without any extra service. The daemon mode, listening on TCP port 873, is faster to set up for bulk mirrors but transmits data unencrypted unless it is tunneled.

Behavior is controlled almost entirely by command-line options, and a small set of them determines whether a copy is a faithful backup. The -a archive mode combines recursion with preservation of permissions, timestamps, symbolic links, owners, groups and device files. Several important attributes are not included in archive mode and must be requested separately.

OptionEffectWhy it matters
-aArchive mode: recursive, preserves most metadataBaseline for any backup
-HPreserves hard linksAvoids duplicating linked files
-A and -XPreserve ACLs and extended attributesNeeded for SELinux labels and ACLs
--deleteRemoves destination files missing at the sourceKeeps a true mirror, but propagates deletions
--link-destHard-links unchanged files to a previous copyEnables space-efficient versioned snapshots
-n (--dry-run)Shows what would change without writingEssential before destructive runs
--checksumCompares full-file checksums instead of mtimeCatches silent changes, but is slow
Options that matter most for backups

A frequent source of confusion is the trailing slash on the source path. rsync -a /data/ dest/ copies the contents of /data, while rsync -a /data dest/ creates dest/data. Combined with --delete, a misplaced slash can remove large amounts of data at the destination, which is why experienced administrators always test with --dry-run first.

rsync is a transfer engine, not a complete backup system, and it is best understood next to the concepts it is often combined with. A single rsync mirror behaves like an always-current copy; it becomes an incremental backup scheme only when combined with --link-dest or with filesystem snapshots on the target.

ApproachVersioningDeduplicationEncryption at rest
Plain rsync mirrorNone, latest state onlyNoneNot provided
rsync with –link-destOne tree per runPer file, through hard linksNot provided
ResticBuilt-in snapshotsContent-defined chunksBuilt in
Filesystem snapshotPoint-in-time copyBlock level, copy-on-writeDepends on the filesystem
Full backup archiveOne archive per runNoneDepends on the tool
rsync versus neighboring tools and concepts

The --link-dest pattern deserves special attention. Each run writes a new dated directory, and files that have not changed become hard links to the previous run, so every directory looks like a full backup while consuming space only for changed files. The limit is that a file with a single changed byte is stored again in full, whereas chunk-based tools such as Restic store only the changed chunks.

Designing reliable rsync backups on servers

A dependable rsync backup is a pull-based, versioned and verified process rather than a single cron line. Pulling from a dedicated backup host means that a compromised production server holds no credentials to modify or delete its own backups, which is a meaningful defense against ransomware.

Consistency is the next concern. rsync copies files as they are at the moment of reading, so live databases and open files may be captured in an inconsistent state. The reliable pattern is to dump the database or take a filesystem or LVM snapshot first, and then let rsync copy the frozen view. The decision table below summarizes common situations.

SituationRecommended approachReason
Web root with static filesNightly pull with --link-destCheap daily versions
Running databaseDump or snapshot first, then rsyncAvoids torn data files
Large media archiveMirror plus snapshots on the targetFew changes, huge volume
Offsite copy over WANSSH transport with --bwlimitProtects production bandwidth
Ransomware concernsPull model plus immutable targetSource cannot alter history
Choosing an rsync strategy

Finally, a backup is only proven by a restoration test. Because rsync output is an ordinary directory tree, a restore is simply a reverse rsync, and teams should regularly restore sample paths and compare them with --checksum or independent hash lists.

Limitations and common mistakes

rsync is robust, but several of its properties catch administrators off guard. The most serious risk is that a mirror faithfully copies damage: encrypted, corrupted or deleted files at the source replace good copies at the destination on the next run.

  • Deletion propagation: --delete without versioning turns an accidental deletion into a permanent loss after one run.
  • Silent metadata loss: omitting -H, -A or -X produces restores with broken hard links, missing ACLs or lost SELinux labels.
  • Large file lists: trees with tens of millions of files make the file-list phase slow and memory-hungry.
  • Small-change, large-file cost: very large files such as virtual machine images still require reading the whole file on both sides.
  • No integrity catalog: rsync does not record checksums of what was backed up, so bit rot on the target goes undetected unless it is checked separately.
  • Exit codes ignored: codes such as 23 (partial transfer) and 24 (vanished files) must be monitored, or failed runs look successful.

In simple terms

rsync is a copying tool that is smart about what it copies. Instead of moving every file each night, it looks at what is already on the other side and sends only the parts that changed. That is why it is so fast once the first copy exists.

The catch is that rsync simply makes the copy look like the original. If something is deleted or ruined on the server, the copy can be ruined too, so keeping several dated versions is what turns rsync from a mirror into a real backup.

A practical example

A small hosting team protects 40 web servers, each holding about 150 GB of site files. A dedicated backup host pulls from every server at 02:00 over SSH using rsync -aHAX --delete --link-dest into a new dated directory, and keeps 30 daily and 12 monthly trees. The databases are dumped to a local file ten minutes earlier, so rsync copies a consistent dump instead of live data files.

After the first full copy, a nightly run transfers roughly 2 to 5 GB per server and finishes in a few minutes. When a customer overwrote a theme folder by mistake, an engineer restored it from the tree of three days earlier with a single reverse rsync, and the site was back within 15 minutes. A monthly restoration test confirms that the dated trees remain complete and readable.