OverlayFS

On this page 7

OverlayFS is a union filesystem in the Linux kernel that combines several directories into one merged view: one or more read-only lower layers and an optional writable upper layer. Files from all layers appear together in a single mount point, while every change is written only to the upper layer, so the lower layers remain untouched.

OverlayFS was merged into the mainline kernel in version 3.18 and has become the default storage mechanism for container images in Docker, Podman and containerd through the overlay2 driver. It is also used for live systems, embedded devices and read-only root filesystems, where a pristine base image must be protected from modification.

How OverlayFS merges layers

An overlay mount is defined by four directories. The kernel presents the merged result at the mount point and decides for each file lookup which layer supplies it. When the same path exists in several layers, the upper layer wins, followed by the lower layers in the order given.

DirectoryMount optionRole
Lower directorylowerdirRead-only base layers, several separated by colons
Upper directoryupperdirWritable layer that receives all changes
Work directoryworkdirEmpty scratch space on the same filesystem as upper
Merged directoryMount pointCombined view seen by applications
Directories involved in an overlay mount

A typical mount command looks like mount -t overlay overlay -o lowerdir=/base,upperdir=/upper,workdir=/work /merged. The work directory is used internally to prepare files atomically before they appear in the upper layer, and it must reside on the same filesystem as the upper directory. Without an upper directory, the overlay is entirely read-only, which is useful for stacking several read-only images.

Directories that exist in more than one layer are merged: listing such a directory shows the union of entries from all layers. Regular files are not merged; the topmost copy hides those beneath it.

Copy-up, whiteouts and opaque directories

Because lower layers are read-only, OverlayFS uses a copy-on-write approach. Three mechanisms make modification and deletion possible without altering the base.

MechanismTriggered byWhat happens
Copy-upWriting to or changing metadata of a lower fileWhole file copied to the upper layer first
WhiteoutDeleting a file that exists in a lower layerCharacter device 0/0 created in upper to hide it
Opaque directoryDeleting and recreating a lower directoryExtended attribute hides all lower contents
Redirect directoryRenaming a lower directory (optional feature)Attribute records the original path
Core mechanisms behind writes and deletions

Copy-up works on whole files, not blocks. Appending one line to a 2 GB log file in a lower layer first copies all 2 GB into the upper layer. This is the most important performance characteristic to understand, and it explains why large, frequently modified files belong in volumes or bind mounts rather than in container image layers.

Newer kernels add a metacopy option, which copies only metadata when ownership or permissions change, and defers the data copy until the content itself is written. This makes operations such as chown -R on large trees much cheaper.

OverlayFS in container storage

Container images consist of stacked layers, each representing changes made by one build step. With the overlay2 storage driver, every image layer is a lower directory, and each running container receives its own upper and work directory. Starting a container is therefore nearly instant: nothing is copied until the container writes.

Many containers can share the same lower layers, which saves disk space and page cache. Ten containers from the same image use one copy of the base files on disk and in memory, while each keeps its own small upper layer. Deleting the container simply removes its upper directory.

Rootless container engines can use OverlayFS inside a user namespace on kernels from version 5.11, or fall back to fuse-overlayfs, a FUSE implementation that runs in user space at some cost in speed.

OverlayFS is not a disk filesystem. It stores nothing on its own and relies on an underlying filesystem such as ext4 or XFS for its layers. Its closest relatives are other layering or snapshot mechanisms.

TechnologyLayering methodGranularityTypical use
OverlayFSUnion of directoriesWhole filesContainers, live media
Btrfs snapshotsCopy-on-write subvolumesExtents (blocks)Snapshots, rollbacks
LVM thin snapshotsCopy-on-write block devicesChunksVolume-level snapshots
Bind mountSame directory at a second pathNo layeringSharing directories
tmpfsMemory-backed filesystemNo layeringVolatile upper layers
AUFSOlder union filesystemWhole filesLegacy container hosts
OverlayFS and neighboring storage concepts

Combining OverlayFS with tmpfs as the upper layer is a classic pattern: the system runs from a read-only image, all changes live in memory, and a reboot restores the original state. The underlying lower filesystem for overlay2 should be ext4 or XFS with d_type support, which XFS provides when formatted with ftype=1.

Limitations and practical pitfalls

OverlayFS behaves like a normal filesystem in most cases, but POSIX compliance is not complete. Administrators and developers should know where it differs.

SymptomCauseRemedy
Slow first write to a large fileWhole-file copy-upKeep mutable data in volumes
Different inode numbers over timeFile moves to upper on copy-upEnable xino or avoid inode assumptions
Unexpected lower layer editsModifying lower directories while mountedNever change layers under a live mount
Disk fills in the upper layerLogs and caches written inside containersLog to stdout or mounted volumes
Mount fails on XFSFilesystem formatted without ftypeReformat with ftype=1
Typical problems and remedies

A file opened read-only before copy-up and another opened for writing afterward may refer to different underlying files, which can confuse applications that rely on strict inode identity. Modern kernels reduce such cases, but software that depends on hard links across layers or on stable inode numbers should be tested carefully.

In simple terms

Imagine a printed page covered by a sheet of clear plastic. You can write on the plastic, cross things out and add notes. The page underneath never changes. Looking from above, you see the page and your notes together.

OverlayFS does the same with folders on a Linux system. The original files stay protected, and every change goes onto the top layer. Remove the top layer and the original is back. This is why containers can start in a moment and share the same base files.

A practical example

A company runs 40 web service containers on one host, all built from the same 600 MB base image. With the overlay2 driver, the image layers exist once on the host’s XFS disk, and each container adds only its own upper directory of a few megabytes.

After a week, the disk unexpectedly fills. An investigation with du on the upper directories reveals that one service writes a growing debug log inside the container. The team moves logging to stdout and mounts a volume for the cache directory. Upper layers shrink back to a few megabytes, and new containers continue to start in under a second.