Muxing, short for multiplexing, is the process of combining separately encoded elementary streams, such as video, audio, subtitles and metadata, into one container file or transport stream with shared timing information. In video containers and formats, muxing is the step that turns an H.264 video stream and an AAC audio stream into a playable MP4, MOV, MKV or MPEG-TS file.
Muxing does not change the encoded data itself. It packages the streams, interleaves them, writes timestamps and builds the index that players use to seek. The reverse operation, extracting streams from a container, is demuxing, and re-encoding the streams is transcoding.
How muxing works
A muxer receives encoded packets from each elementary stream, each tagged with timing information. It then writes them into the container’s structure in an interleaved order, so that video and audio for the same moment sit close together in the file. Interleaving lets a player read the file sequentially without jumping back and forth between distant positions.
Two timestamps are central. The presentation timestamp (PTS) says when a frame or audio sample should be shown or heard, while the decoding timestamp (DTS) says when it must be decoded. They differ for codecs with B-frames, where frames are decoded out of display order. Correct timestamps are what keep audio and video in sync.
| Element | Purpose | Example |
|---|---|---|
| Stream packets | Carry the encoded audio, video or subtitle data | H.264 frames, AAC audio frames |
| Timestamps | Define decoding and presentation timing | PTS and DTS values per packet |
| Track headers | Describe codec, resolution, sample rate, language | Video track 1920 x 1080, audio track English |
| Index | Allows fast seeking | MP4 moov atom, MKV cues |
| Metadata | Titles, chapters, color and HDR information | Chapter markers, mastering display data |
Muxing in different container formats
Each container defines its own structure, codec support and streaming behavior, so muxing rules differ. The same video and audio streams can be muxed into several containers without re-encoding, provided the container supports the codecs.
| Container | Structure | Strength | Typical use |
|---|---|---|---|
| MP4 | Boxes, also called atoms | Universal device and web support | Streaming, downloads, social media |
| MOV | QuickTime atoms | Professional codecs and timecode | Editing and mastering, ProRes |
| MKV | EBML elements | Many tracks, flexible codecs and subtitles | Archives, multi-language releases |
| MPEG-TS | Fixed 188-byte packets | Resilient to errors and cut-in | Broadcast, IPTV, HLS segments |
| Fragmented MP4 | Repeated moof and mdat fragments | Adaptive streaming segments | DASH and modern HLS |
A practical detail in MP4 is the position of the moov atom, which holds the index. Encoders often write it at the end of the file; for progressive web playback it should be moved to the start, a step called fast start, so playback can begin before the whole file downloads.
Muxing compared with demuxing and transcoding
Muxing, demuxing, remuxing and transcoding are often confused because they appear in the same tools. The key distinction is whether the encoded data is touched. Muxing and demuxing only move packets between containers, while transcoding decodes and re-encodes, with a cost in time and usually in quality.
| Operation | What changes | Quality impact | Typical speed |
|---|---|---|---|
| Muxing | Streams packaged into a container | None | Very fast |
| Demuxing | Streams extracted from a container | None | Very fast |
| Remuxing | Streams moved to a different container | None | Very fast |
| Transcoding | Streams decoded and re-encoded | Generation loss unless lossless | Slow, compute-intensive |
Remuxing is often the right answer to compatibility problems. An MKV file containing H.264 video and AAC audio can be remuxed into MP4 in seconds, while transcoding would take far longer and reduce quality for no benefit.
Practical muxing tasks and settings
Muxing is a routine step in delivery, streaming and archiving. Command-line tools and authoring software expose settings that control how streams are combined and labeled.
| Task | What is done | Why it matters |
|---|---|---|
| Adding audio tracks | Muxing extra language or commentary tracks | One file serves several audiences |
| Adding subtitles | Muxing text or image subtitle streams | Accessibility and localization |
| Setting track flags | Marking default and forced tracks | Players choose the correct track |
| Applying a sync offset | Shifting audio timestamps | Corrects lip sync without re-encoding |
| Segmenting | Splitting streams into short fragments | Required for HLS and DASH delivery |
Segment length for adaptive streaming is typically 2 to 6 seconds, and segments must begin with a keyframe. That requirement is set at encoding time through the GOP structure, so muxing and encoding settings have to be planned together.
Common muxing problems
Most muxing failures show up as sync errors, playback refusals or poor seeking. The underlying causes are usually predictable.
- Unsupported codec for the container, such as certain audio formats in MP4, causes devices to refuse playback.
- Incorrect or missing timestamps after editing raw streams lead to drifting audio sync.
- Variable frame rate sources muxed without proper timestamps play unevenly in editors.
- Missing index or moov at the end delays web playback until the full file has downloaded.
- Wrong language or default flags make players select the wrong audio or subtitle track.
Because muxing preserves the encoded data, these issues can usually be fixed by remuxing with corrected settings, without returning to the original encode.
Understanding it simply
A video file usually contains several separate pieces: the picture, one or more soundtracks and sometimes subtitles. Muxing is the step that packs all of these pieces together into one file and labels them so a player knows when to show and play each part.
It is like packing a lunch box with separate containers for each item. The food itself does not change, only the way it is packed and labeled. Unpacking it again is called demuxing.
Applied in practice
A film distributor prepares an online release of a feature film. The finished H.264 video and three AAC audio tracks, in English, German and Spanish, arrive as separate files, along with subtitles in five languages. Rather than re-encoding anything, the delivery technician muxes all streams into one MKV master for the archive and sets English audio as the default track.
For the streaming platform, the same video and audio streams are remuxed into fragmented MP4 segments of 4 seconds for adaptive playback. A screening reveals a 2-frame audio delay on the German track, which is corrected with a timestamp offset during muxing. The entire packaging takes minutes rather than hours, and no quality is lost at any step.