Containers vs. Codecs: The Packaging and the Contents of a Video File

Why does MKV-to-MP4 take seconds while transcoding takes tens of minutes? And are MP4 and H.264 the same thing or not?

You've probably noticed these phenomena: "converting" a video from MKV to MP4 sometimes finishes in seconds with the file size virtually unchanged; other times it takes tens of minutes and the picture quality may suffer. The same video file opens without complaint in a desktop player, yet your TV reports "unsupported format." Two videos that both end in .mp4—one merges cleanly with another clip, the other throws an error.

Behind all these seemingly unrelated problems is one pair of concepts: the container and the codec. They are the most important—and most frequently confused—fundamentals in digital audio and video. Arguably half of all "why does this happen?" questions in the media world are answered right here.

The Three Layers: Box, Goods, and Manifest

A video file is not "one solid block of video." It is a structured package made of three layers:

Video file (container, e.g. .mp4)
├── Video stream (compressed with a video codec, e.g. H.264)
├── Audio stream (compressed with an audio codec, e.g. AAC)
├── Subtitle stream (optional, e.g. SRT, ASS)
├── Chapter markers (optional)
└── Metadata (title, cover art, recording date, rotation flag...)

A shipping metaphor makes the relationship clear:

• The container is the box: it dictates how goods are arranged inside and where the manifest is attached. A file extension (.mp4, .mkv, .mov) tells you about the box, not the contents.
• The codec is how the goods are packed: video and audio are each compressed with their own algorithm and loaded as separate parcels (streams).
• A stream is each individual parcel in the box: one file can hold multiple video streams, multiple audio streams, and multiple subtitle streams.

With this structure in mind, every question from the opening resolves itself—"format conversion" might mean swapping the box (changing the container) or repacking the goods (changing the codec), and those two tasks differ enormously in cost.

Containers in Detail: Six Mainstream Boxes

One key fact to remember: saying "this video is an MP4" conveys almost no information—it describes the box, not the goods. When two devices differ in whether they can play "MP4," the difference usually comes down to the codec packed inside.

Codecs in Detail: How the Goods Are Packed

Video Codecs

Audio Codecs

Pairing Rules: Which Box Holds Which Goods

Containers and codecs cannot be combined arbitrarily. MP4 and MKV both accept H.264, but the reverse direction has limits—WebM won't take H.264, for example, and AVI cannot properly hold modern AV1 (it can be forced, with terrible compatibility).

In practice, memorizing these "safe pairings" is enough:

• MP4 = H.264/H.265 + AAC: the golden combination for distribution—no platform will refuse it;
• WebM = VP9/AV1 + Opus: the modern choice for web embedding;
• MKV = holds anything: the first choice for collections, multi-language audio, and external subtitles;
• MOV = ProRes/H.264 + AAC/PCM: camera footage and professional intermediates.

What Else Is in the Box: The Overlooked "Attachments"

Beyond audio and video streams, a container can carry quite a few extras—and they explain many everyday phenomena:

• Multiple audio streams: a film can carry the original language, a dub, and a director's commentary, switchable during playback (MKV's party trick).
• Subtitle streams: subtitles exist as independent tracks that can be toggled on and off—this is what "soft subtitles" are.
• Chapter markers: jump to specific sections, DVD-style.
• Cover art and metadata: title, artist, album art, recording date, GPS coordinates.
• Rotation flag: a marker that says "please rotate 90° when playing"—the root cause of sideways smartphone videos (covered in depth in article 05 of this series).

Remux vs. Transcode: Seconds Versus Tens of Minutes

Now we can solve the first puzzle from the opening. "Format conversion" actually covers two fundamentally different operations:

Remuxing: Only Swap the Box

Move the video and audio streams—byte for byte, untouched—from one container into another:

• Extremely fast: it's just data relocation; an entire film "converts" in seconds;
• Zero quality loss: the goods never moved, so the picture is identical to the original;
• Precondition: the target container must accept the existing codecs (H.264+AAC moves from MKV to MP4 perfectly; WebM cannot move into MP4, because MP4 does not universally accept VP9).

Transcoding: Repack the Goods

Decode the video stream back into raw frames, then recompress them with a different codec:

• Slow: every frame must be decoded and re-encoded; a 10-minute 1080p video can take minutes to tens of minutes;
• Quality loss: every recompression with a lossy codec discards some information (generation loss), unless you set a very high bitrate;
• Maximum freedom: you can simultaneously change codec, resolution, bitrate, and frame rate.

Remux:      MKV(H.264) ──swap box──→ MP4(H.264)      seconds, lossless
Transcode:  MKV(VP9) ──recompress──→ MP4(H.264)      minutes, lossy

A practical rule of thumb: if a "format conversion" finishes suspiciously fast, it was almost certainly a remux and no quality was lost; if a progress bar crawled for a long time, it was a transcode, and the output parameters (bitrate, codec) deserve your attention.

Streaming Considerations: The Order of Things Inside the Box

Even the arrangement of data within a container matters. An MP4 file's "table of contents" (the moov atom) indexes all its streams, and a player must read it before playback can begin:

• moov at the end: many encoders place the index at the end of the file by default—fine for local playback, but over a network the whole file must download before playback can start;
• moov at the front (Fast Start): moving the index to the beginning enables "play while downloading" over the network. Applying faststart before uploading a video to a website is a common optimization for online playback.

Going further, the streaming era developed segmented packaging (fragmented MP4, TS chunks) paired with protocols like HLS and DASH for adaptive bitrate playback—when you switch between "1080p" and "720p" on a video site, you're actually switching between sets of small segments behind the scenes.

How to Read a File's True "Packing List"

A file extension can be renamed at will (renaming .mkv to .mp4 does not convert anything), so to see a file's true composition you need to look inside the container:

• MediaInfo: a free, open-source inspection tool with a graphical interface that lists every stream with its codec, bitrate, resolution, and channel count;
• ffprobe: a command-line tool from the FFmpeg family that dumps every technical detail—ideal for batch checks;
• Player properties panels: most players' "media information" windows show the basic composition.

Building the habit of "check the packing list before converting" saves a great deal of wasted effort—if the source already contains H.264+AAC, converting to MP4 only requires a remux, and there's no reason to sit through a lengthy transcode.

Common Misconceptions

1. "Renaming the extension converts the format." Renaming only changes the label; the box and the goods stay exactly the same, and strict players won't be fooled.
2. "MP4 is sharper than MKV." Sharpness is determined by codec and bitrate, not the container. The same H.264 stream looks identical whether packed in MP4 or MKV.
3. "Transcoding and converting are the same thing." A remux is lossless and instant; a transcode is lossy and slow. Confirm which one you actually need first.
4. "MKV is a bad format." MKV is the most capable open container—"my TV won't play MKV" is a device support issue, not a format defect.
5. "A video file contains only picture and sound." Multiple audio tracks, soft subtitles, chapters, and cover art all live in the container—and they can silently vanish during conversion (especially when moving to a less capable container).

Practical Tips

• Distribute in MP4, archive in MKV: share externally as MP4 (H.264+AAC) for guaranteed compatibility; use MKV for personal collections and multi-track content to preserve everything.
• Remux whenever possible: if the source codec is already supported by the target container, prefer remuxing—it saves time and loses nothing.
• Inspect the source before transcoding: use MediaInfo or ffprobe to check the source codec and bitrate. Setting an output bitrate far above the source just inflates the file; quality will not improve.
• Remember faststart for web video: for MP4s uploaded to websites or embedded in pages, make sure the moov atom is at the front so playback can start immediately.
• Watch out for attachments: when converting files with multiple audio tracks or soft subtitles, confirm those streams are carried over.

Further Reading

• Format Conversion: Moving Files Freely Between Platforms — format comparisons and conversion strategies
• Video Compression: Finding the Balance Between Quality and Size — how codecs compress video data
• Video Merging: Making Clips Into a Whole — why clips with mismatched codecs can't be concatenated directly

When processing files, this site's video converter prefers lossless remuxing whenever it's viable—and after reading this article, you can not only read the codec information in your conversion results, but also judge for yourself which route a conversion should take.

Related Tools

• Video Converter
• Audio Converter