Almost every video you have ever watched on a computer passed through code from a project you have probably never heard of. The FFmpeg pipeline — demux, decode, filter, encode, mux — is the assembly line that turns a file on disk into moving pictures on your screen, and it runs inside your browser, your media player, your streaming software, and a great deal of the infrastructure that puts video on the internet in the first place.
Most people meet FFmpeg as an intimidating command line and conclude it is a program with a bad interface. That framing is wrong in a way that makes the whole subject harder than it needs to be. FFmpeg is a set of C libraries. The ffmpeg command is one consumer of those libraries. Once you see the architecture underneath, the command line stops looking arbitrary, the container-versus-codec confusion dissolves, and a dozen everyday mysteries — why one file plays and another doesn’t, why changing a video’s wrapper takes a second while converting it takes an hour — all resolve into the same handful of ideas.
This is the map. No prior media vocabulary assumed; nothing dumbed down.
The One Distinction That Fixes Half of All Video Confusion
Before anything else: a container is not a codec. This single conflation is responsible for more confused hours than every other misunderstanding in digital video combined, and it has a clean fix.
A container is a box. A codec is the compression.
A container — also called a wrapper format — is a file structure that holds one or more already-compressed streams, plus the metadata, timestamps, and (usually) a seek index needed to play them back together. MP4, Matroska (.mkv), WebM, MPEG-TS, and QuickTime (.mov) are containers. A container says nothing about how the video inside it was compressed.
A codec (coder-decoder) is the compression scheme itself: the algorithm that turns 336 gigabytes of raw pixels into a 4 GB file and back again. H.264, HEVC (H.265), AV1, VP9, AAC, Opus, and FLAC are codecs.
The Matroska specification states this about as plainly as a spec ever states anything. Matroska, per its own technical documentation, “is an audiovisual data container format” — “an envelope in which there can be many audio, video, and subtitles streams” — and, explicitly, “it is NOT a video or audio compression format (codec).”
The WebM project makes the split concrete from the other direction. “The WebM file structure is based on the Matroska container,” restricted to video “compressed with the VP8 or VP9 video codecs” and audio “compressed with the Vorbis or Opus audio codecs.” WebM is a container that deliberately pins which codecs may appear inside it — precisely because container and codec normally vary independently.
(A quick disambiguation for developers: this is a completely different sense of the word than the Docker-style software container. Same word, unrelated concept, no shared lineage. The collision is unfortunate and permanent.)
Why “my .mp4 won’t play” is almost never the container’s fault
When a file refuses to play, the wrapper has usually parsed just fine. Your player opened the MP4, read the index, found the streams — and then discovered it had no decoder for what was inside. The failure is a missing or unsupported codec wearing a container’s file extension. An HEVC video in an MP4 fails on a machine with no HEVC decoder; the identical MP4 with H.264 inside plays everywhere.
FFmpeg enforces this distinction structurally rather than politely. Per FFmpeg’s own project description, libavformat handles “demuxers and muxers for multimedia container formats” and libavcodec provides “decoders and encoders for audio/video codecs.” Two libraries, two jobs, two failure modes. You can swap the box without touching the contents, and you cannot touch the contents without unpacking the box.
| Container (the box) | Codec (the compression) | |
|---|---|---|
| What it is | A file structure holding streams, metadata, timestamps, and an index | An algorithm that compresses and decompresses a stream |
| Examples | MP4, Matroska (MKV), WebM, MPEG-TS, MOV, AVI | H.264, HEVC, AV1, VP9, AAC, Opus, FLAC |
| Answers the question | “How are these streams packaged and synchronized?” | “How were these pixels or samples made smaller?” |
| FFmpeg library | libavformat | libavcodec |
| Changing it | Remux — near-instant, bit-exact, no quality loss | Transcode — slow, and lossy unless the codec is lossless |
| Typical failure you’ll see | Rare; corrupt or truncated files | “Unsupported codec,” green screen, audio-only playback |
The same distinction, one level down: standards versus encoders
There is a third layer people collapse, and it’s worth naming now because it recurs constantly.
H.264 is not x264. H.264, HEVC, and AV1 are standards — published specifications describing exactly what a compliant bitstream looks like and how a decoder must interpret it. H.264/AVC is a live, versioned ITU-T Recommendation, maintained and revised like any other formal standard. An encoder is a piece of software that produces a bitstream conforming to that standard, and there can be many competing encoders for one standard, all producing legal output of wildly differing quality and speed.
Attribution here is frequently mangled, so: x264 and dav1d are VideoLAN projects; x265 is MulticoreWare’s, built on x264 code licensed from VideoLAN and first released publicly in 2013. (VideoLAN’s own current project listing includes x265 among its projects — a documented imprecision on their part, not a reason to restate the wrong attribution.)
| Standard (the specification) | Software encoders | Hardware implementations |
|---|---|---|
| H.264 / AVC — ITU-T & ISO/IEC | x264 (VideoLAN), OpenH264 | NVENC, Intel Quick Sync, VAAPI, AMF, VideoToolbox |
| H.265 / HEVC — ITU-T & ISO/IEC | x265 (MulticoreWare) | NVENC, Quick Sync, VAAPI, AMF, VideoToolbox |
| AV1 — Alliance for Open Media | SVT-AV1, libaom, rav1e | Newer GPU and SoC encode blocks |
| AV1 decoding | dav1d (VideoLAN), libaom | Increasingly common in phones and GPUs |
NVENC, Quick Sync, VAAPI, AMF, and VideoToolbox are hardware implementations of standards — not codecs of their own. A file encoded with NVENC and a file encoded with x264 are both just H.264 files; any H.264 decoder plays either. What differs is how many bits each encoder needed to reach a given quality, and how fast it got there.
FFmpeg Is Not a Program. It Is Seven Libraries.
Here is the reframing the whole article rests on. When people say “FFmpeg,” they are usually thinking of a command they pasted from Stack Overflow. What the project actually ships is a stack of C libraries — and FFmpeg’s own About page lists them one by one, each with a single, sharply-scoped job.
| Library | What it does (per FFmpeg’s own description) | Layer it owns |
|---|---|---|
libavutil | “random number generators, data structures, mathematics routines, core multimedia utilities” | Shared foundations |
libavcodec | “decoders and encoders for audio/video codecs” | Compression |
libavformat | “demuxers and muxers for multimedia container formats” | Packaging |
libavfilter | “media filters” | Transformation of raw frames |
libswscale | “image scaling and color space/pixel format conversion” | Pixel-format plumbing |
libswresample | “audio resampling, rematrixing and sample format conversion” | Audio-format plumbing |
libavdevice | “input and output devices for grabbing from and rendering to… common multimedia… frameworks” | Capture and playback devices |
ffmpeg, ffplay, and ffprobe are described by the same page as tools built on those libraries. Three command-line programs, sitting on top of a library stack that vastly exceeds them in importance. The libraries are the product. The CLI is a client.
Who actually uses these libraries — stated precisely
This matters because “X is a GUI for FFmpeg” is one of the most confidently repeated wrong sentences in software. Almost nothing wraps the ffmpeg binary. Serious media applications link the libraries and call them directly, in-process, with no command line involved at all.
Two relationships are documented in primary sources, and they are not the same relationship:
- Chromium ships a fork. Chromium’s own
README.chromiumfor its bundled copy states that “Chromium’s copy of FFmpeg is a fork of the upstream git repository,” carrying Chromium-specific patches, with a stated policy that “every change made must be upstreamed.” So Chrome, Edge, Brave, and every other Chromium-derived browser decodes media through a maintained fork — not unmodified upstream, and certainly not a subprocess. - VLC links one library, not the stack. VLC uses
libavcodecfor many of its codecs while keeping its own demuxers and muxers. So VLC is not “a GUI for FFmpeg” in even the loosest sense: it borrows the compression layer and implements the packaging layer itself.
OBS, HandBrake, and mpv likewise build on the libraries rather than shelling out to the binary. Beyond that documented core, the honest position is this: FFmpeg’s libraries are a near-default dependency wherever software has to handle arbitrary media, which is why the code plausibly runs on your phone, in your browser, in your car’s infotainment stack, and across streaming infrastructure — but every specific “company X uses FFmpeg” claim needs its own evidence, and a surprising number of the ones you’ll read online have none. The reach is real. The specifics deserve citations.
The layering point is the payoff. Nobody sane reimplements five hundred formats. They link libavcodec and inherit decades of accumulated, hard-won, edge-case-riddled correctness for free. That is the entire reason a small volunteer project ended up underneath a large fraction of the world’s video.
The FFmpeg Pipeline: Demux → Decode → Filter → Encode → Mux
Every consumer of these libraries — the CLI, VLC, your browser, a custom C program — runs data through the same five-stage architecture. FFmpeg’s CLI documentation describes it directly in its “Detailed description” section, and the stage names are the vocabulary you need for everything else.
Demuxers “read an input source in order to extract global properties such as metadata or chapters” along with a “list of input elementary streams.” One demuxer is created per input file. This is the stage that opens the box: it parses the container, finds that there is one video stream, two audio streams, and a subtitle track, and starts handing out compressed chunks.
Decoders “receive encoded (compressed) packets… and decode them into raw frames.” This is where H.264 bits become actual pixels — the expensive, mathematical part.
Filtergraphs “process and transform raw audio or video frames.” Scaling, cropping, overlays, volume adjustment, deinterlacing, color conversion. Simple filtergraphs (-vf, -af) have exactly one input and one output of matching type; complex ones (-filter_complex) allow multiple mixed-type inputs and outputs, which is what makes picture-in-picture and multi-source composition possible.
Encoders “receive raw… frames and encode them into encoded packets.” Pixels become compressed bits again, under whatever quality and bitrate constraints you set.
Muxers “receive encoded packets… and write the resulting bytes into the output file.” The new box gets packed, indexed, and closed.
| Stage | Takes in | Puts out | Library | Cost | Skipped by -c copy? |
|---|---|---|---|---|---|
| Demux | A file, URL, or device | Compressed packets + stream info | libavformat | Cheap — parsing only | No |
| Decode | Compressed packets | Raw frames (pixels / samples) | libavcodec | Expensive | Yes |
| Filter | Raw frames | Raw frames | libavfilter, libswscale, libswresample | Varies | Yes |
| Encode | Raw frames | Compressed packets | libavcodec | Very expensive; lossy | Yes |
| Mux | Compressed packets | A file, stream, or device | libavformat | Cheap — packing only | No |
Read that table’s last column carefully, because it is the single most useful thing on this page.
AVPacket vs AVFrame: The Two Types That Define Everything
The pipeline’s stages aren’t just a narrative convenience. They are enforced by two genuinely distinct C data structures, and the boundary between them is where all the interesting behavior lives.
AVPacket “stores compressed data” — it is, in FFmpeg’s own words, “exported by demuxers and then passed as input to decoders, or received as output from encoders and then passed to muxers.” An AVPacket is a sealed envelope of codec bits. Nothing in the container layer knows or cares what’s inside it.
AVFrame “describes decoded (raw) audio or video data” — actual pixel planes or actual audio samples, uncompressed, enormous, and directly manipulable.
The consequence is strict and clarifying: filters only ever see raw frames. Nothing between decode and encode touches compressed bytes. You cannot “just crop the H.264 a little” or “slightly brighten the compressed stream.” Any pixel-level change requires a full round trip through decode → filter → encode, and that round trip re-spends quality every single time.
This is also why a well-typed boundary is worth more than a clever feature. Because compressed and raw are separate types crossing only at two specific points, “skip decode and encode, keep everything else” becomes a clean, expressible operation rather than a special case bolted on later. Which brings us to the most useful flag in the entire tool.
Why -c copy Is Instant and Lossless
-c copy is documented in one short clause: “the stream is not to be re-encoded.” That’s it. Packets travel from the demuxer straight to the muxer, unopened.
The effect is dramatic and, once you understand the pipeline, completely unsurprising. Rewrapping a two-hour MKV into an MP4 with the same codecs inside takes a second or two on a modern machine, produces a bit-identical video stream, and loses exactly nothing. Converting that same file from H.264 to HEVC takes minutes to hours, pins every core you have, and permanently discards image quality — because it must actually decode every frame to AVFrames and re-encode them.
Remux (-c copy) | Transcode (re-encode) | |
|---|---|---|
| What changes | The container only | The compressed streams themselves |
| Pipeline stages used | Demux → mux | Demux → decode → filter → encode → mux |
| Speed on a 2-hour file | Seconds (I/O-bound) | Minutes to hours (CPU/GPU-bound) |
| Quality | Bit-exact; nothing lost | Generation loss, every time |
| What it can’t do | Change codec, resolution, or pixels | Nothing — but you pay for it |
| Typical use | MKV → MP4 for a device; trimming on keyframes; fixing a broken index | Shrinking a file; targeting a device that lacks a decoder |
Being precise about which operation a claim covers is a habit worth building. “Converting a video” is ambiguous and the ambiguity costs people real quality: converting the wrapper is free; converting the compression is not. When someone says a tool “converted my video losslessly,” the only question that matters is whether it re-encoded.
What Actually Happens Inside “Encode”
The encode stage deserves its own section, because it is where the magic and the loss both live — and because the numbers make the case better than any adjective.
Raw video is absurdly, disqualifyingly large
Take one second of uncompressed 1920×1080 video at 8-bit depth with 4:2:0 chroma subsampling — the near-universal consumer format, where full-resolution brightness is paired with quarter-resolution color. That’s 2,073,600 bytes of luma plus two chroma planes of 518,400 bytes each: 3,110,400 bytes per frame. At 30 frames per second, roughly 93.3 MB/s — about 746 Mbit/s. Over an hour, on the order of 336 GB.
Full 4:4:4 video, with no chroma reduction at all, doubles it again — 24 bits per pixel instead of 12.
No consumer network carries 746 Mbit/s. No reasonable disk budget holds 336 GB per hour of footage. If you’ve ever wondered why internet bandwidth is such a persistent constraint on video, that’s the raw number it’s fighting. Compression is not an optimization applied to digital video — it is the precondition for digital video existing as a consumer medium at all.
Two redundancies make the reduction possible
Video compresses well because moving pictures are enormously repetitive in two independent directions:
- Spatial (intra) redundancy — within a single frame, neighboring pixels are usually similar. This is the same insight JPEG exploits on a still photo.
- Temporal (inter) redundancy — consecutive frames usually look a great deal like each other. In a two-person dialogue scene, well over 90% of most frames is unchanged from the previous one.
Codecs since MPEG-1 have built three frame types on that split. An I-frame (intra-coded) is self-contained, coded without reference to any other frame — a complete picture. A P-frame (predicted) has each block predicted from a region of a previously decoded frame, so unchanged areas are never resent. A B-frame (bidirectional) predicts by combining regions from two previously decoded frames — one earlier and one later in display order.
The recurring arrangement of these between keyframes is a GOP (group of pictures). This is not a textbook abstraction: YouTube’s published upload recommendations specify “2 consecutive B frames” and a “closed GOP” for its preferred H.264 encode, alongside CABAC entropy coding — GOP structure as a live encoder setting a platform actually cares about.
Motion compensation is the mechanism behind P- and B-frames. The frame is divided into blocks; each block is predicted from an equally-sized block in a reference frame, shifted by an offset recorded as a motion vector; and only the leftover difference — the residual — is actually compressed. Instead of transmitting a whole new frame, the encoder transmits “this block moved three pixels left” plus a small correction.
Where the loss actually happens
Here is the detail almost every explanation gets wrong. Motion compensation and intra-prediction are exact operations. They lose nothing. The prediction plus the residual reconstructs the original perfectly.
The loss happens one step later. The residual (or the raw block, for an I-frame) is passed through a transform — classically a block-based discrete cosine transform, whose defining property is strong energy compaction: most of the signal’s information concentrates into a few low-frequency coefficients. Those coefficients are then quantized — divided down and rounded off. That rounding is the irreversible step. Everything after it, including the context-adaptive entropy coding that packs the result, is lossless.
This matters practically, not just pedantically. Every quality knob you will ever touch — CRF, a bitrate target, a quality preset — is, mechanically, a knob on quantization step size. And when a badly compressed video breaks into visible squares, you are not looking at “the codec failing.” You are looking at coarse quantization showing its work: blocking artifacts are quantization made visible.
Chroma subsampling: the loss that happened before compression started
One more lossy decision precedes all of that. Because, as Xiph.org’s A Digital Media Primer for Geeks puts it, “the human eye has far less spatial color resolution than spatial luminosity resolution,” color information can be stored at reduced resolution “usually without any significant impact on the apparent raw image quality.” That’s exactly the 4:2:0-versus-4:4:4 trade in the arithmetic above — inherited from analog color television, and it halves the data before compression proper even begins.
It is not entirely free, and the place people meet its cost is compositing: chroma subsampling degrades edge color precision, which is why green-screen and chroma-key work is noticeably harder on 4:2:0 footage than on 4:2:2 or 4:4:4. Fine color detail that was thrown away at capture cannot be recovered by a keyer.
A worked ratio, with its conditions attached
Put the numbers together: YouTube’s published target for 1080p30 SDR uploads is 8 Mbit/s, against roughly 746 Mbit/s uncompressed 4:2:0 — on the order of 90-to-1.
State that carefully or don’t state it. That ratio is specific to this resolution, this frame rate, this chroma format, this codec family, and this platform’s own recommended bitrate. It is a worked illustration of scale, not a constant. Compression figures without their test conditions attached are marketing, not measurement — a habit worth applying to every “50% more efficient” codec claim you will ever read.
Timestamps: PTS, DTS, and Why Sync Is Genuinely Hard
Packets and frames don’t just carry data. They carry time, and the way they carry it explains one of the most common frustrations in media work.
Every packet and frame has a PTS (presentation timestamp): “the time at which the decompressed packet will be presented to the user,” expressed in AVStream->time_base units. Packets additionally carry a DTS (decoding timestamp): “the time at which the packet is decompressed.” PTS must always be greater than or equal to DTS, since you cannot display a frame before you have decoded it.
Why would those two ever differ? B-frames. A B-frame sits, in display order, between the two frames it references — but it cannot be decoded until both of those frames are already available, including the one that comes later on the timeline. So the encoder writes frames to the file in decode order, which genuinely differs from presentation order, and the decoder reorders them internally before display. It’s a direct, unavoidable consequence of letting a frame reference something in its own future.
Then there’s the time_base itself: “the fundamental unit of time (in seconds) in terms of which frame timestamps are represented.” It is a rational number — 1/30, 1/1000, 1/90000 — not a fixed universal clock, and each stream in a container carries its own. A raw PTS integer is meaningless without knowing which time base it’s counted in.
That’s the real answer to “why is A/V sync such a persistent problem?” There is no authoritative “frame N” shared across streams. There is a video stream with its own timestamp sequence in its own rational unit, an audio stream with a different sequence in a different unit, possibly a subtitle track with a third, and a player obliged to reconcile all of them against a wall clock that can drift. Sync is not a solved problem that implementations keep botching; it is a genuinely hard reconciliation problem that mostly works, which is the more impressive outcome.
The Command Line’s Grammar Is the Pipeline Showing Through
Now the payoff for everything above: the ffmpeg CLI’s most notorious quirk is not a quirk.
The documentation states the rule twice in a single section, which tells you how often it trips people: “options are applied to the next specified file. Therefore, order is important,” and “first specify all input files, then all output files… All options apply ONLY to the next input or output file and are reset between files.”
Read that with the pipeline in mind and it stops being arbitrary. One demuxer is instantiated per input. One encoder/muxer pair is instantiated per output. An option like “use H.264, at CRF 20” has to attach to one specific instance — and the parser’s way of knowing which is position. The command line is not a bag of global settings. It is a left-to-right description of a graph being wired up.
Which is why this classic mistake is a category error rather than a typo:
ffmpeg -c:v libx264 -i input.mkv output.mp4 # binds libx264 to the INPUTffmpeg -i input.mkv -c:v libx264 output.mp4 # binds libx264 to the OUTPUTThe first command attaches a video-codec choice to the input file — to the decoder, where specifying an encoder is nonsensical. The grammar did exactly what it documents. The mental model was the thing that was wrong.
Automatic stream selection, and why it surprises everyone
The same “the architecture is showing through” logic explains a second common surprise. Without -map, FFmpeg applies documented automatic stream selection: for video it picks “the stream with the highest resolution”; for audio, “the stream with the most channels”; for subtitles, “the first subtitle stream found.” Data and attachment streams are never auto-selected.
Each of those three rules is individually reasonable. Bundled together and left implicit, they routinely astonish. A file with a high-resolution “behind the scenes” extra alongside the main feature can have the extra selected on resolution alone. A file with 5.1 commentary and stereo dialogue can have the commentary win on channel count. The fix is -map, which is explicit and total: “when -map is used, only user-mapped streams are included.”
The generalizable lesson: a system with several independent default-selection heuristics owes its users an explicit override for each one, not merely better guessing.
Where It Came From, and What It Costs to Keep
FFmpeg’s creator says it in his own words on his own site: “I launched this project in year 2000 and led it for several years.” That’s Fabrice Bellard — the same person behind QEMU, described on that page as “a generic machine emulator and virtualizer,” and TCC, “a tiny but complete ISOC99 C compiler.” An independent history from the codec-development community places the first commit in December 2000, initially under the pseudonym “Gerard Lantau,” hosted on SourceForge.
Leadership passed to Michael Niedermayer, who led the project for roughly eleven years before resigning on July 31, 2015, citing the 2011 project split directly: “The community is split and its very difficult to be the leader when one is on one side of this split.” That split — developers forking FFmpeg as Libav in 2011 — is contested territory with two very different published accounts, and this article deliberately doesn’t adjudicate it. The safe, verifiable fact is that FFmpeg is the project that survived as dominant and actively developed.
Actively is right. FFmpeg 8.1, codenamed “Hoare,” was released on 2026-03-16 — a project a quarter-century old still shipping releases on a steady cadence. Because codec support, defaults, and hardware acceleration all change between releases, it’s worth checking ffmpeg -version rather than trusting anything you read about feature availability, including this article.
One more layer worth naming before we close, because it’s the one that catches companies rather than hobbyists: software licensing is not patent licensing. FFmpeg is LGPL-2.1-or-later by default; building with --enable-gpl (needed to link x264 or x265) makes the resulting binary GPL; and --enable-nonfree produces builds that are not redistributable under any terms at all. None of that grants patent rights, and FFmpeg’s own legal page says so explicitly. The standards implemented by these encoders carry their own separate patent landscape, with its own licensing pools. That is a full article’s worth of material, and a later piece in this series takes it on properly.
The Habit Worth Stealing: Don’t Collapse the Layers
Step back and look at what every confusion in this article had in common.
“My .mp4 won’t play” collapses container into codec. “VLC is a GUI for FFmpeg” collapses a linked library into a wrapped binary. “H.264 versus x264” collapses a standard into one implementation of it. “Convert this video” collapses remuxing into transcoding. “The DCT loses quality” collapses an exact transform into the quantization step that follows it. “The command line is badly designed” collapses a per-instance option grammar into an imagined bag of global flags.
Six confusions, one shape. And the same layering that generates the confusion is exactly what makes the whole edifice possible: because libavformat and libavcodec are separate, -c copy can exist. Because the libraries are separate from the CLI, a browser can link them without a subprocess. Because standards are separate from encoders, five competing encoders can improve independently and still produce files that play everywhere. Because compressed and raw are distinct types, the pipeline can be reasoned about at all.
Nobody reimplements five hundred formats. They link libavcodec. That single act of layering is why a volunteer project started in 2000 ended up quietly decoding a large share of the moving images on earth — and why the most useful question to ask about almost any media problem is not “what’s the command?” but “which layer is this actually happening at?”
❓ Frequently Asked Questions
Is FFmpeg a program or a library? Both, but the libraries are the substance. The project ships seven C libraries — libavcodec, libavformat, libavfilter, libavutil, libswscale, libswresample, and libavdevice — and three command-line tools built on top of them: ffmpeg, ffplay, and ffprobe. Most software that uses FFmpeg links the libraries directly and never touches the command line at all.
What’s the difference between a container and a codec? A container (MP4, MKV, WebM, MPEG-TS) is a file structure that packages already-compressed streams together with metadata, timestamps, and a seek index. A codec (H.264, HEVC, AV1, AAC, Opus) is the compression scheme applied to a stream. Matroska’s own specification states it directly: it “is NOT a video or audio compression format (codec).” You can change the container without touching the compressed data; you cannot change the codec without decoding and re-encoding.
Why won’t my .mp4 file play? Almost certainly because your player lacks a decoder for a codec inside it — commonly HEVC or AV1 video, or an unusual audio codec — not because the MP4 container is broken. The wrapper usually parses fine; the contents are what failed. Symptoms like audio-with-no-video or a green screen point at the codec layer, not the container.
Why is -c copy so much faster than converting a file? Because it skips three of the five FFmpeg pipeline stages. -c copy means “the stream is not to be re-encoded”: compressed packets go straight from the demuxer to the muxer without ever being decoded into raw frames, filtered, or re-encoded. Decode and encode are the expensive stages, so removing them turns hours into seconds — and, because the compressed data is copied bit-for-bit, nothing is lost.
Does converting a video always reduce quality? Re-encoding with a lossy codec does, every time — the quantization step throws away information irreversibly, and doing it repeatedly compounds the damage (generation loss). Remuxing does not: changing MKV to MP4 with -c copy produces a bit-identical video stream. The distinction is whether the pixels were decoded and re-encoded, not whether the file extension changed.
Why does the order of FFmpeg’s command-line options matter so much? Because options bind to files, not to the command as a whole. The documentation is explicit: “All options apply ONLY to the next input or output file and are reset between files.” One demuxer is created per input and one encoder/muxer pair per output, so an option has to attach to a specific instance — and position is how the parser knows which one. Putting -c:v libx264 before -i binds it to the input, where it means nothing useful.
Is x264 the same thing as H.264? No. H.264 is a standard published as an ITU-T Recommendation; x264 is one software encoder (a VideoLAN project) that produces conforming H.264 bitstreams. Other encoders — including hardware ones like NVENC and Quick Sync — produce H.264 too, at different speeds and efficiencies. Any H.264 decoder plays output from all of them. The equivalent distinction holds for HEVC and x265, which is MulticoreWare’s project, not VideoLAN’s.
🔗 Further Reading
- FFmpeg — About — the canonical description of every library and every tool the project ships.
- FFmpeg —
ffmpegdocumentation — the pipeline’s “Detailed description,” the positional-option rule, automatic stream selection, and-c copysemantics. - FFmpeg Doxygen —
AVPacketandAVFrame— the compressed/raw type boundary in the source itself. - Matroska — Basics — a container spec stating plainly that it is not a codec.
- The WebM Project — About — a container that deliberately pins its permitted codecs.
- Xiph.org — A Digital Media Primer for Geeks — the clearest free explanation of sampling, chroma, and why digital video looks the way it does.
- ITU-T H.264 Recommendation — the standard itself, and evidence that standards are documents, not software.
- YouTube — Recommended upload encoding settings — a real platform’s published bitrate, GOP, and entropy-coding targets.
- Fabrice Bellard’s homepage — the founder’s own first-person account, alongside QEMU and TCC.
📚 Recommended Resources
Disclosure: some links below are Amazon affiliate links. If you buy through them, I earn a small commission — at no extra cost to you — which helps keep this site running.
Start with the free material — genuinely. This is one of the few technical subjects where the best resources cost nothing and the printed shelf is thinner and older than most readers expect. There is no mature book on AV1 or VVC at all; for those, the specifications are the literature. Work through these first, and only buy a book if you want the deeper theory in print.
- Digital Video Introduction — Leandro Moreira’s free, illustrated walkthrough of exactly the ground this article covers, taken further: frame types, GOP structure, the transform, and rate control, with runnable examples. It is the single best free primer on video coding, actively maintained, translated into several languages, and the only widely-used introduction that treats AV1 as a first-class citizen.
- FFmpeg’s own documentation —
ffmpeg-all— every option, muxer, demuxer, filter, and encoder in one nightly-regenerated page. Unglamorous and unbeatable: when a blog post and this page disagree, this page is right. - A Digital Media Primer for Geeks — Xiph.org’s free video primer on sampling, quantization, and chroma, and the clearest explanation anywhere of why digital audio and video look and sound the way they do.
- The AV1 Bitstream & Decoding Process Specification and the ITU-T H.264 Recommendation — the actual standards, free to read. If you want to know what an encoder is allowed to emit, this is the only authority.
Introduction to Data Compression
Khalid Sayood · 5th edition, Morgan Kaufmann, 2017 · Paperback, 790 pp
The theory underneath the encode stage, taught properly. Sayood builds up from information theory through entropy coding, quantization, and transform coding — which is to say, he explains why the quantization step in this article is where the loss happens, rather than just asserting it. General compression theory rather than a video-specific text, and the better book for it: the ideas transfer to audio, images, and everything else.
Digital Video and HD: Algorithms and Interfaces
Charles Poynton · 2nd edition, Morgan Kaufmann, 2012 · Hardcover, 752 pp
The reference on the signal-and-color side of the pipeline — gamma, color spaces, chroma subsampling, and the interface details that make 4:2:0 behave the way it does. Worth knowing what it is not: this is a color and signals book, with roughly one part of six on compression and no HEVC or AV1 coverage. Buy it to understand pixels, not codecs.
Coding Video: A Practical Guide to HEVC and Beyond
Iain E. Richardson · Wiley, 2024 · Hardcover, 448 pp
The most current codec text on the shelf, from the author of the standard H.264 books. It walks the modern block-based coding pipeline — prediction, transform, quantization, entropy coding — at the level of detail this article’s “what happens inside encode” section only gestures at. One honest caveat: the publisher’s own description does not promise substantive AV1 or VVC coverage, so treat “and Beyond” as an HEVC-centric book with a forward glance rather than a guide to the newer codecs.