Start with the number, because the number is the entire argument. One second of uncompressed 1920×1080 video — 8 bits per sample, 4:2:0 chroma, 30 frames per second — is 93.3 megabytes. One hour of it is roughly 336 gigabytes. A two-hour film, stored the way a screen actually displays it, would be about 672 GB: most of a laptop’s drive, for one movie. The file you actually stream is a couple of gigabytes at most. The honest answer to why video files are so small is that a codec discards, or simply declines to re-send, something on the order of 99% of that data — and it does so in six distinct stages, only two of which throw information away at all.
That distinction is the spine of this article. Almost everything a video encoder does is lossless: predicting, transforming, and packing are reversible operations that shrink a file without touching a single perceptible detail. Exactly two steps are irreversible — one that exploits a quirk of human vision before the codec proper even starts, and one buried in the middle that is, mechanically, the quality dial on every encoder you have ever used. Knowing which is which explains most of what people find mysterious about video: why re-uploading a clip degrades it, why a screen recording of text looks worse than a video of a face at the same bitrate, why scrubbing snaps to certain moments, and why “this codec is 50% more efficient” is a sentence that cannot be true or false until somebody tells you what they encoded.
The Arithmetic Nobody Runs
A digital frame is a grid of samples. At 8 bits per sample with no chroma subsampling — 4:4:4, one full-resolution luminance value and two full-resolution color-difference values per pixel — each pixel costs 24 bits, or 3 bytes. In the far more common 4:2:0 arrangement, the two color-difference planes are stored at quarter resolution, so a pixel averages 12 bits, or 1.5 bytes. Multiply by pixel count, then by frame rate, and you have the raw bitrate: the number of bits per second a stream consumes, the single most useful number in all of media engineering.
| Format (8-bit 4:2:0) | Pixels per frame | Bytes per frame | Raw bitrate | Raw data per hour |
|---|---|---|---|---|
| 640×480 @ 30 fps | 307,200 | 460,800 | ~111 Mbps | ~50 GB |
| 1280×720 @ 30 fps | 921,600 | 1,382,400 | ~332 Mbps | ~149 GB |
| 1920×1080 @ 30 fps | 2,073,600 | 3,110,400 | ~746 Mbps | ~336 GB |
| 1920×1080 @ 60 fps | 2,073,600 | 3,110,400 | ~1.49 Gbps | ~672 GB |
| 1920×1080 @ 30 fps, 4:4:4 | 2,073,600 | 6,220,800 | ~1.49 Gbps | ~672 GB |
| 3840×2160 (4K) @ 30 fps | 8,294,400 | 12,441,600 | ~2.99 Gbps | ~1.34 TB |
| 3840×2160 (4K) @ 60 fps | 8,294,400 | 12,441,600 | ~5.97 Gbps | ~2.69 TB |
| 7680×4320 (8K) @ 60 fps | 33,177,600 | 49,766,400 | ~23.9 Gbps | ~10.7 TB |
Derived from the pixel-format definitions, not quoted from a vendor. Gigabytes are decimal (10⁹ bytes). Recompute these if bit depth, chroma format, or frame rate change — a 10-bit HDR master is 25% larger again per sample.
Two things in that table are worth sitting with. First, resolution is quadratic in the worst way: 4K is not “a bit more than 1080p,” it is exactly four times the raw data, and 8K is sixteen times. Second, the 4:4:4 row consumes precisely as much as doubling the frame rate — which tells you that the decision to store color at reduced resolution, made before compression starts, is worth as much as halving the frame count.
Against ~746 Mbps of raw 1080p30, Google’s published recommendation for a 1080p SDR upload is 8 Mbps (YouTube Help, “Recommended upload encoding settings”). That is roughly 93 to 1 — but read the conditions before you carry that number anywhere: it compares one specific raw format against one platform’s recommended target for one codec, at one resolution and frame rate, for content the platform considers typical. It is a worked illustration of scale, not a constant. We will come back to exactly why no such constant exists.
Where the Other 99% Goes
Compression works on redundancy — information present in the data that need not be transmitted because the decoder can reconstruct it — and on irrelevance, information the viewer will not miss. Video is unusually rich in both:
- Spatial redundancy. Within a single frame, neighboring pixels are usually similar. A patch of sky, a wall, a cheek: adjacent samples carry nearly the same value, over and over.
- Temporal redundancy. Consecutive frames usually resemble each other. At 30 fps, one frame is 33 milliseconds after the last. Most of the picture has not changed; what has changed has often merely moved.
- Statistical redundancy. After the first two are exploited, what’s left is a symbol stream with a very uneven probability distribution — some values wildly more common than others. That’s compressible by pure information theory, with nothing thrown away.
- Perceptual irrelevance. Human vision resolves fine detail in brightness far better than in color, and tolerates small numerical errors in high-frequency detail far better than in flat areas. Anything the eye cannot resolve does not need to be encoded at all.
The first three are redundancy: removable losslessly, in principle. The fourth is not redundancy at all — it is a deliberate decision to discard real signal because a human being will not notice. Confusing these two categories is the root of nearly every misconception about video quality.
Chroma Subsampling: Discarding Color Before the Codec Starts
Chroma subsampling is the practice of storing the color-difference channels at lower spatial resolution than the brightness channel. It happens at the pixel-format level, before an encoder sees a single frame, and it is the first lossy step in the chain.
It works by first splitting each pixel out of RGB into one luma-like channel (Y′, essentially brightness) and two chroma channels (Cb and Cr, blue-difference and red-difference). Then it simply throws most of the chroma samples away. The rationale is stated cleanly by Xiph.Org, the standards foundation behind Vorbis, Opus, and Theora: “the human eye has far less spatial color resolution than spatial luminosity resolution,” so chroma can be reduced “usually without any significant impact on the apparent raw image quality” (Xiph.org Wiki, “A Digital Media Primer for Geeks”).
The notation reads as a three-number ratio over a conceptual 4×2 block of pixels: 4:4:4 keeps every chroma sample; 4:2:2 halves chroma horizontally; 4:2:0 halves it both horizontally and vertically, keeping one chroma pair per 2×2 group of pixels. That last one is what nearly all consumer video uses, and as the table above shows, it cuts raw data in half — 24 bits per pixel down to 12 — before compression proper has done anything at all.
This is worth naming precisely, because it is the clearest case in the whole pipeline of exploiting the viewer rather than the signal. Nothing about a video frame is mathematically redundant in its chroma planes. The data is simply gone, on the bet that your retina was never going to resolve it. The bet mostly pays: for natural photographic content, 4:2:0 is close to invisible. It fails, visibly, in three places — saturated colored edges (especially red on black), rendered text and UI in screen recordings, and any footage destined for chroma-key compositing or heavy color grading, where a mask or a grade will amplify exactly the color detail that was discarded. Professional acquisition formats keep 4:2:2 or 4:4:4 for that reason, and a chroma-key workflow is one of the few consumer-adjacent tasks where the difference is unmissable.
Spatial Redundancy and the DCT: A Change of Basis, Not a Loss
Inside a frame, a codec first predicts. Intra prediction fills a block by extrapolating from already-decoded neighboring pixels — copying the edge above it downward, or the edge to its left rightward, or interpolating along one of several angular directions. What gets coded is not the block but the residual: the difference between the prediction and the truth. If the prediction was good, that residual is mostly near-zero.
Then comes the transform. Classically a block-based discrete cosine transform (DCT), whose defining useful property is “a strong energy compaction property,” concentrating “most of the signal information… in a few low-frequency components” (Wikipedia, “Discrete cosine transform”). The DCT does not compress anything. It re-describes a block of pixel values as a weighted sum of increasingly fine wave patterns — a smooth overall level, then coarse gradients, then progressively finer ripples. For real image data, the coarse patterns carry almost all the energy and the fine ones carry almost none. You have not lost anything; you have moved the interesting parts into a corner where they are easy to treat differently from the boring parts.
That “you have not lost anything” is a real claim, not a rhetorical one, and modern codecs go out of their way to keep it true. A floating-point DCT would introduce tiny rounding differences between implementations, and those differences would accumulate across predicted frames until the decoder’s picture drifted away from the encoder’s. H.264 solved this by specifying an integer transform exactly — every conforming decoder computes bit-identical results, and the transform stage is exactly invertible by construction (ITU-T Recommendation H.264). HEVC and AV1 use the same strategy with larger and more varied transform sizes.
So the blocking artifacts everyone associates with “DCT compression” are not the DCT’s doing. They come from what happens next.
Temporal Redundancy: I, P, and B Frames
Across frames, the redundancy is enormous and the savings are correspondingly larger. Codecs since MPEG-1 have exploited it with three frame types, arranged in a repeating pattern.
An I-frame (intra-coded) is “a self-contained image… coded without reference to any other frame” — essentially a standalone still, compressed using only the spatial tools above. A P-frame (predicted) lets “each macroblock… be predicted from a region of any previously decoded frame,” so unchanged regions are never re-sent. A B-frame (bi-predictive) goes further, predicting “by combining regions from each of two previously decoded frames,” one earlier and one later in display order (Wikipedia, “Video compression picture types”).
| Frame type | What it stores | What it references | Relative cost | Usable as a seek point? |
|---|---|---|---|---|
| I (intra) | A complete picture: intra predictions plus residuals for every block | Nothing outside itself | Largest by far | Yes — a decoder can start cold here |
| P (predicted) | Motion vectors plus residuals; unchanged regions cost almost nothing | One or more earlier decoded frames | Much smaller than an I-frame | No — its references must already be decoded |
| B (bi-predictive) | Motion vectors into two directions plus a usually tiny residual | Two decoded frames, one earlier and one later in display order | Smallest | No — and it forces decode order to differ from display order |
Relative cost is deliberately qualitative. The actual ratio between an I-frame and a B-frame on your footage depends entirely on your footage — which is this article’s other main point.
The repeating arrangement between I-frames is called a GOP (group of pictures). A closed GOP contains no references reaching outside its own boundaries, which makes each GOP independently decodable and is why streaming formats prefer it; an open GOP allows references across the boundary for slightly better efficiency at the cost of clean cut points. These are live encoder settings, not textbook abstractions: YouTube’s own published recommendations specify “2 consecutive B frames” and a “closed GOP” for its preferred H.264 encode (YouTube Help).
GOP structure is also the direct cause of a behavior every viewer has noticed without explanation. When you drag a scrubber, the player cannot simply start decoding at an arbitrary frame — a P- or B-frame is meaningless without its references. It must locate the nearest preceding I-frame (in H.264, specifically an IDR frame, which flushes the reference buffers and guarantees a clean start) and decode forward from there. Short GOPs make seeking crisp and files larger; long GOPs make files smaller and seeking coarser. That trade-off, not player sloppiness, is why scrubbing snaps.
B-frames cause the second unintuitive behavior: because a B-frame references a frame that comes after it on screen, that later frame must be decoded first. Decode order and display order genuinely differ, which is why every media stream carries two timestamps per packet — a decode timestamp and a presentation timestamp — rather than one.
Motion Compensation: Describing the Movement, Not the Pixels
The mechanism inside P- and B-frames is motion compensation, and it is the single largest source of video’s compression advantage over a folder of still images. The frame is divided into blocks; “each block is predicted from a block of equal size in the reference frame… shift[ed]… represented by a motion vector,” and only “the residual” is compressed (Wikipedia, “Motion compensation”).
Read that literally: the encoder is not sending a picture. It is sending a short instruction — this 16×16 region is the region 7 pixels left and 3 pixels up in the previous frame — plus a small correction for whatever the shift got wrong. A camera pan across a static landscape, which looks like an entirely new image every frame to a human, is to an encoder almost nothing: one motion vector repeated across the whole frame and a near-zero residual.
Three details make this stage worth understanding properly.
It is exact, in itself. A motion vector plus a residual reconstructs the block perfectly. Prediction is not where quality is lost; it merely determines how much there is left to code. Conflating “prediction” with “loss” is the most common way people misplace where an encoder’s quality dial actually acts.
The encoder predicts from what the decoder will have, not from the original. Reference frames used for prediction are the reconstructed ones — already transformed, quantized, and inverse-transformed, artifacts and all. If an encoder predicted from pristine sources, its idea of each frame would drift from the decoder’s a little more with every predicted frame until the picture visibly diverged. Closing that loop is what keeps a hundred-frame GOP stable.
It is why encoding is slow and decoding is fast. Finding a good motion vector means searching a reference frame for the best match, across candidate block sizes, sub-pixel positions, and multiple reference frames. That search is the bulk of an encoder’s work and none of a decoder’s — the decoder is simply told the answer. This asymmetry is fundamental to the design: encode once, expensively; decode a billion times, cheaply.
It is also why content changes everything. Motion search succeeds spectacularly on rigid, predictable movement and fails on anything that isn’t. Fire, rain, water, foliage in wind, film grain, confetti: no block in the next frame is a shifted copy of any block in this one, every residual is large, and the encoder is reduced to coding nearly everything from scratch. Two files at the same resolution, frame rate, codec, and settings can differ by an order of magnitude in size, purely because of what is in them.
Quantization: The One Step That Actually Throws Data Away
Here is the whole lossy heart of a video codec, in one sentence: the transform coefficients are divided by a step size and rounded to integers.
That is quantization. A coefficient of 137 with a step size of 16 becomes 9 (which reconstructs as 144, off by 7). A coefficient of 3 becomes 0 — gone entirely. Nothing recovers those discarded fractions; the division is not invertible. Every artifact you have ever seen in compressed video traces back to this step and this step alone.
The step size is the knob. Make it small and coefficients survive nearly intact, the file gets large, and the output approaches the input. Make it large and most high-frequency coefficients round to zero, the file collapses, and the block is reconstructed from only its coarsest few wave patterns. That is what produces the classic failure modes:
- Blocking — visible squares — is the documented consequence of coarse quantization at high compression ratios (Wikipedia, “Discrete cosine transform”). Adjacent blocks are reconstructed from different surviving coefficients and no longer line up at their edges.
- Ringing and mosquito noise around sharp edges come from discarding the high-frequency coefficients that a crisp edge needs.
- Banding in gradients and dark skies comes from quantization collapsing a smooth ramp onto too few distinct levels.
Every user-facing quality control is a wrapper around this. When you set a constant-rate-factor (CRF) value, or a quantization parameter, or a target bitrate, you are ultimately instructing a rate-control algorithm about how large a quantization step to use, block by block, to land near your goal. That is worth internalizing precisely because it demystifies encoder settings: they are not “quality” in some abstract sense, they are a schedule of rounding coarseness.
One important consequence: quantization is not merely a lossy step, it is the lossy step inside the codec — but it is applied afresh on every encode. Decoding an already-quantized video back to pixels and re-encoding it quantizes a signal that has already been rounded once, compounding the error. That is generation loss, and it is why re-uploading a downloaded clip, or repeatedly re-exporting an edit, degrades it even when nothing about the settings got “worse.”
Entropy Coding: The Final Squeeze, and It Loses Nothing
After quantization, the encoder holds a stream of small integers — mostly zeros, with occasional non-zero values clustered in the low-frequency corner of each block. This is a wildly non-uniform probability distribution, and information theory has an exact answer for those: spend fewer bits on common symbols and more on rare ones.
That is entropy coding, and it is completely lossless — the same category of operation as ZIP, applied to a stream engineered upstream to be extremely compressible. H.264 offers two schemes: CAVLC (context-adaptive variable-length coding) and the stronger CABAC (context-adaptive binary arithmetic coding), which YouTube names directly in its recommended encoding settings (YouTube Help). “Context-adaptive” is the important half: the coder maintains probability models that update as it goes and selects among them based on what neighboring blocks did, so its estimate of “how likely is this symbol here” keeps sharpening over the course of a frame. HEVC uses CABAC exclusively; AV1 uses a multi-symbol arithmetic coder in the same family (AOMedia, AV1 Bitstream & Decoding Process Specification).
Notice the collaboration between these last two stages. Quantization does not just shrink data — it manufactures the statistical skew that entropy coding feeds on, by rounding vast numbers of coefficients to exactly zero. A codec’s final bitrate is produced by a lossless coder operating on a deliberately impoverished signal. The lossy step and the lossless step are designed for each other.
The Spine: Which Steps Lose Information and Which Don’t
| Stage | What it does | Lossless or lossy | What is lost |
|---|---|---|---|
| Chroma subsampling (4:2:0) | Stores the two color-difference planes at quarter resolution | Lossy (pre-codec) | Fine color detail — usually invisible, obvious on saturated edges, text, and keying |
| Intra prediction | Predicts a block from already-decoded neighbors | Lossless in itself | Nothing — only the residual is coded |
| Motion estimation / compensation | Describes a block as a displacement from a reference block | Lossless in itself | Nothing — only the residual is coded |
| Transform (integer DCT) | Re-describes a residual block as frequency coefficients | Lossless by design | Nothing — exactly specified and exactly invertible |
| Quantization | Divides coefficients by a step size and rounds | Lossy — the lossy step | Coefficient precision → blocking, ringing, banding |
| Entropy coding (CABAC) | Packs the quantized symbols with adaptive probability models | Lossless | Nothing |
| Muxing into a container | Writes packets, timestamps, and an index into MP4/MKV/WebM | Lossless | Nothing |
That last row is where a persistent confusion lives, and it is worth stating flatly: a container is not a codec. MP4, MKV, WebM, and MPEG-TS are envelopes holding compressed streams plus timing and indexing metadata; H.264, HEVC, AV1, AAC, and Opus are the compression schemes inside them. Matroska’s own documentation says so about as plainly as a specification can — it is “an audiovisual data container format” and explicitly “NOT a video or audio compression format (codec)” (Matroska Basics).
The practical consequence follows directly from the table. Changing the container touches no codec stage at all, so it is instant and perfectly lossless — FFmpeg’s stream-copy mode is documented as exactly that: “the stream is not to be re-encoded” (ffmpeg.org, ffmpeg documentation). Changing the codec requires decoding to pixels and encoding again, which means quantizing again, which means losing something again. “Convert this video” is two completely different operations depending on which layer you meant, and only one of them costs you quality.
While we’re being precise about layers: H.264, HEVC, and AV1 are standards; x264, x265, SVT-AV1, and libaom are encoders that implement them. x264 and the AV1 decoder dav1d are VideoLAN projects; x265 is MulticoreWare’s, built on x264 code licensed from VideoLAN. Two encoders targeting the same standard can produce very different files from the same source at the same nominal settings — which brings us to the part of this subject that most writing gets wrong.
Every Number in Media Is a Model, Not a Measurement
Return to that 93:1 figure from the opening. It is arithmetically correct and completely conditional. Change the content from a locked-off interview to a handheld shot of falling snow and the same encoder at the same settings will produce a file several times larger. Change the codec generation, the preset, the rate-control mode, or the target device and it moves again. There is no such thing as “the compression ratio of H.264,” and any single number presented as one has quietly hidden its test conditions.
The canonical example is the claim that HEVC is 50% more efficient than H.264. That figure is real and it has a source: the standard’s own overview paper reports roughly a 50% bitrate reduction at equal perceptual quality versus H.264/AVC High Profile, under the standardization body’s own test conditions (G. J. Sullivan, J.-R. Ohm, W.-J. Han, T. Wiegand, “Overview of the High Efficiency Video Coding (HEVC) Standard,” IEEE Transactions on Circuits and Systems for Video Technology, 22(12), 2012; see also ITU-T Recommendation H.265). Those conditions are a specific set of test sequences, evaluated a specific way, with specific reference encoder configurations. Strip them off — as nearly every repetition of the claim does — and “50% more efficient” stops describing a property of the codec and starts describing a measurement someone once made. Encoder maturity, content type, preset, and rate-control mode all move that number substantially, in both directions.
The correct way to compare two encoders is not a single number at all: it is BD-rate (the Bjøntegaard delta), which encodes the same source at several operating points with each encoder, fits both rate-distortion curves, and integrates the area between them to express an average bitrate saving across the whole curve. A single-point comparison tells you about that one bitrate on that one clip and nothing else. And BD-rate itself needs a quality metric to plot against — which introduces a second layer of modeling, because no quality metric is ground truth either. PSNR is mean-squared error on a decibel scale with no model of human vision in it. VMAF is closer, but it is a trained model fit to human ratings, with documented ways to inflate it and multiple viewing-condition variants, which is why any VMAF figure has to state which model version produced it — Netflix released VMAF v1 on June 19, 2026, a materially different generation from the v0 models most published numbers still refer to (Netflix VMAF project).
So the discipline is simple, and it generalizes past video to any benchmark you will ever read: a media number is the output of a model applied to a test condition. When you see one, ask which content, which encoder, which version, which settings, and which metric. If the answer isn’t available, the number isn’t a fact about the world — it is a fact about somebody’s experiment.
What This Explains About Everyday Video
The mechanism, once assembled, quietly explains a whole category of everyday puzzles.
A recording of a person talking to a webcam compresses beautifully because the background is static, the subject moves predictably, and motion compensation eats almost all of it. A screen recording of scrolling text compresses badly and looks bad, because sharp high-contrast edges are exactly what quantization damages first and text is exactly what 4:2:0 chroma subsampling was never designed for. Two “1080p” files can differ tenfold in size and neither is mislabeled, because resolution constrains the pixel grid and says nothing about the bit budget. A video re-shared through three platforms looks soft because each hop re-quantized an already-quantized signal. And a long GOP is why the scrubber lands where it lands.
If you want to watch these stages operate, the open-source stack makes them observable rather than theoretical — the current release is FFmpeg 8.1 “Hoare,” released March 16, 2026 (ffmpeg.org downloads), and the encoders it drives expose the quantization, GOP, and motion-search parameters described above directly. Bitrate is where these decisions meet the network, which is why the same arithmetic underpins how internet bandwidth actually works: 746 Mbps of raw 1080p is not a storage problem so much as a delivery impossibility, and compression is what made streaming video a consumer product rather than a broadcast-engineering one.
❓ Frequently Asked Questions
If compression discards around 99% of the data, why doesn’t video look 99% worse? Because almost none of that 99% was information you could perceive. The bulk of the reduction comes from redundancy — pixels predictable from their neighbors, frames predictable from earlier frames — which is removed by prediction, an exactly reversible operation that costs no quality at all. Only two stages discard real signal: chroma subsampling, which targets a specific documented limit of human vision, and quantization, which rounds away precision in the frequency components the eye is least sensitive to. The design intent throughout is to spend the discarding budget where perception is weakest.
Which step in video compression is actually lossy? Inside the codec, exactly one: quantization — dividing transform coefficients by a step size and rounding them to integers. Prediction, the transform, and entropy coding are all reversible. Chroma subsampling is also lossy but happens earlier, at the pixel-format level, before the encoder is involved. Everything else in the pipeline, up to and including writing the file into an MP4 or MKV container, preserves the bits exactly.
What is a GOP, and why does scrubbing a video snap to certain points? A GOP (group of pictures) is the repeating pattern of frames between self-contained I-frames. Because P- and B-frames are defined only in terms of other frames, a player cannot begin decoding at an arbitrary point — it must start at the nearest preceding I-frame and decode forward. Short GOPs make seeking precise and files larger; long GOPs make files smaller and seek points sparser. The snapping you see is the player finding a legal entry point.
Why is a screen recording of text so much worse than a video of someone talking, at the same settings? Three of the stages work against it simultaneously. Text is high-frequency detail, and quantization discards high-frequency coefficients first, producing ringing around glyph edges. Text is also high-contrast and often colored, and 4:2:0 chroma subsampling has already halved color resolution in both directions before encoding began. And scrolling text defeats motion compensation less gracefully than it looks like it should — sub-pixel scrolling means no block is an exact shifted copy of a previous block, so residuals stay large. Screen content is a genuinely different compression problem from camera content, which is why some codecs ship dedicated screen-content coding tools.
Does converting an MP4 to an MKV re-compress the video? No, provided you remux rather than transcode. Changing containers rewrites the envelope — the packet layout, timestamps, and index — while the compressed streams pass through untouched, which is why it takes seconds on a multi-gigabyte file and costs zero quality. Re-encoding to a different codec is the opposite: it decodes to raw frames and quantizes again, which is inherently lossy no matter how high the quality setting.
Is a codec’s “50% more efficient” claim a real number? It is a real measurement under stated conditions, not a property of the codec. The widely repeated HEVC-versus-H.264 figure comes from the HEVC standard’s own overview paper (Sullivan et al., IEEE TCSVT, 2012) and applies to that paper’s test sequences, quality criterion, and encoder configurations. Content, encoder implementation, preset, and rate-control mode all shift it materially. The rigorous comparison is a BD-rate calculation across whole rate-distortion curves, reported with the metric and test set named — and even then, the metric doing the judging is itself a model.
📚 Further Reading
- Understanding Internet Bandwidth for Engineers and Architects — the delivery side of the same arithmetic: what a bitrate costs once it has to cross a network.
- Why Internet Bandwidth Matters: A Comprehensive Guide to Internet Speed and Efficiency — how the bits-per-second numbers in this article translate into what a connection can actually sustain.
- Ping, Jitter, and Packet Loss — why streaming a compressed video well takes more than raw throughput.
- Screenshots, Screencasts, and Recording your Desktop — the practical side of the screen-content problem described above.
- How Reverse Image Search Works: From Pixels to Meaning — a different route from raw pixels to a compact representation, and a useful contrast with the transform-and-quantize approach.
📚 Recommended Resources
Disclosure: some links below are Amazon affiliate links. If you buy through them, I earn a small commission — at no extra cost to you — which helps keep this site running.
Start with the free material. Video coding is one of the few technical subjects where the best resources cost nothing and the printed shelf is thinner and older than most readers expect — there is no mature book on AV1 or VVC at all, so for those the specifications are the literature. Work through these first; buy a book only if you want the deeper theory in print.
- Digital Video Introduction — Leandro Moreira’s free, illustrated walkthrough of exactly this article’s ground, taken further: frame types, GOP structure, the transform, quantization, and rate control, with runnable examples. It is the single best free primer on video coding, actively maintained, translated into several languages, and the only widely used introduction that treats AV1 as a first-class citizen.
- A Digital Media Primer for Geeks — Xiph.Org’s free primer on sampling, quantization, and chroma, and the clearest available explanation of why the eye tolerates 4:2:0 in the first place. It is the source behind this article’s chroma-subsampling section.
- The ITU-T H.264 Recommendation and the AV1 Bitstream & Decoding Process Specification — the actual standards, free to read. If you want to see the exactly specified integer transform that makes the transform stage bit-identical across decoders, it is in the first of these, not in a blog post.
- Werner Robitza’s CRF guide and the Netflix VMAF project — the two free resources for the measurement half of the story: what the quality dial actually sets, and why every VMAF figure has to name its model version before it means anything.
Introduction to Data Compression
Khalid Sayood · 5th edition, Morgan Kaufmann, 2017 · Paperback, 790 pp
The theory underneath this entire article, taught properly. Sayood builds from information theory through entropy coding, quantization, and transform coding — which is to say, he derives why quantization is the lossy step and entropy coding is not, instead of asserting it the way this article does. It is general compression theory rather than a video-specific text, and better for it: the same ideas explain audio, images, and file archivers. If you buy one book from this list, and you care about the mechanism rather than the codec, buy this one.
Digital Video and HD: Algorithms and Interfaces
Charles Poynton · 2nd edition, Morgan Kaufmann, 2012 · Hardcover, 752 pp
The reference on the signal-and-color side — gamma, color spaces, luma versus luminance, and the interface details that make 4:2:0 behave the way it does. This is the book for the first lossy step in the pipeline, the one that happens before an encoder ever sees a frame. Worth knowing what it is not: a color and signals book, with roughly one part of six on compression and no HEVC or AV1 coverage. Buy it to understand pixels, not codecs.
The H.264 Advanced Video Compression Standard
Iain E. Richardson · 2nd edition, Wiley, 2010 · Hardcover, 352 pp
The book-length version of this article’s middle sections, for the one codec still carrying most of the world’s video. Richardson walks intra prediction, motion estimation, the integer transform, quantization, and the CAVLC/CABAC split at full detail — including the reconstruction loop that keeps an encoder’s reference frames identical to a decoder’s. Dated in the sense that it stops at H.264, which is exactly what it promises; it is not an HEVC or AV1 text.
Coding Video: A Practical Guide to HEVC and Beyond
Iain E. Richardson · Wiley, 2024 · Hardcover, 448 pp
The most current codec text on the shelf, from the same author. It covers the modern block-based pipeline — larger and more varied transform sizes, richer prediction modes, the rate-distortion optimization that decides which of them an encoder picks — at the level this article’s quantization section only gestures at. One honest caveat: the publisher’s own description does not promise substantive AV1 or VVC coverage, so read “and Beyond” as an HEVC-centric book with a forward glance, not a guide to the newer codecs. For those, go back to the free specs above.