How Image Compression Actually Works

I built InstaShrink, which means I spent a lot of evenings staring at DCT coefficients and quantization tables so you don't have to. But if you're curious what actually happens in the half-second between dropping a photo on a compressor and getting a file a tenth the size back, this is the whole story — no hand-waving, no "it's magic".

Two very different meanings of "compress"

Every compression scheme ever invented sits in one of two camps. Lossless compression finds a more efficient way to write down exactly the same data — like writing "a × 1000" instead of a thousand letter a's. Unzip it and you get back every original bit. PNG works this way, which is why a PNG never degrades no matter how many times you re-save it.

Lossy compression cheats. It looks at the image, decides which details your eyes won't miss, and throws them away permanently. That sounds reckless, but it's why a JPEG can be 90% smaller than the raw pixels while looking identical from any normal viewing distance. JPEG and lossy WebP live in this camp.

Neither is "better". Photographs tolerate lossy compression beautifully because they're full of noise and soft gradients. A screenshot with crisp text does not — lossy artifacts smear the letter edges, which is why the right answer is almost always "JPEG for photos, PNG for graphics" and not one format for everything.

Inside a JPEG: five steps from photo to tiny file

JPEG dates from 1992 and it is still, in my view, one of the cleverest pieces of engineering on the web. Every step exploits a specific, measured weakness of human vision.

Step 1: Separate brightness from color

Your camera records red, green, and blue values. JPEG immediately converts these to a different representation called YCbCr: one channel for brightness (Y) and two for color (Cb and Cr). Why bother? Because your retina has far more brightness-sensing cells than color-sensing ones. Once brightness and color are separated, JPEG can be gentle with the channel you're sensitive to and brutal with the ones you're not.

This one conversion sets up roughly half of JPEG's total savings. It's the same trick analog color TV used in the 1950s to squeeze color into a black-and-white signal — good ideas get recycled.

Step 2: Throw away color resolution

Next comes chroma subsampling: keeping brightness at full resolution while storing color at half or quarter resolution. The common patterns:

If you want to see subsampling fail, put small red text on a blue background and save it as a JPEG — the edges turn to mush, because that's pure color contrast with almost no brightness contrast. For photos, though, you will simply never notice.

Step 3: The DCT — turning pixels into frequencies

Here's the mathematical heart. The image is cut into 8×8 pixel blocks, and each block goes through a Discrete Cosine Transform, which re-describes those 64 pixels as a mix of 64 wave patterns — from "one flat color" up to "fine checkerboard".

The audio analogy genuinely helps: a chord is a mix of a few pure notes. The DCT finds which "notes" make up each block of your image. Smooth sky is almost all low notes; grass texture has lots of high ones. Nothing is lost at this step — it's just a change of vocabulary. But it sets up the kill shot, because human eyes are terrible at seeing the high-frequency notes.

Step 4: Quantization — where the loss actually happens

Each of those 64 frequency values gets divided by a number from a quantization table and rounded to the nearest integer. The table is designed so that important low frequencies get divided by small numbers (barely changed) and fine-detail high frequencies get divided by big numbers — which rounds most of them to exactly zero. Deleted, permanently.

When you drag a "quality" slider, this table is the only thing you're changing. Quality 90 divides gently; quality 20 divides so hard that whole blocks collapse to a single average color — that's the blocky look of an over-compressed meme that's been reposted a hundred times.

Step 5: Pack the survivors efficiently

After quantization, each block is mostly zeros. The final stage is classic lossless packing: read the values in a zigzag order so the zeros clump together, store runs of zeros as a single count (run-length encoding), and give the most common values the shortest binary codes (Huffman coding). Free savings, no further quality cost.

Inside a PNG: prediction instead of deletion

PNG takes the opposite philosophy: never delete anything, just describe it more cleverly. Before compressing, PNG runs a prediction filter over each row — it guesses each pixel from its neighbors and stores only the difference between the guess and reality. In a screenshot full of flat color, the guess is almost always right, so the stored differences are almost all zero.

That stream of mostly-zeros then goes through DEFLATE — the same algorithm inside every ZIP file — which hunts for repeated sequences and replaces them with short back-references.

This is why PNG file sizes are so wildly inconsistent, and why I always tell people the format war is really a content war. A 500×500 screenshot with ten colors might be 15 KB as a PNG. A 500×500 photograph might be over a megabyte, because photographic noise is unpredictable and prediction filters can't do anything with it. Same format, hundredfold difference.

WebP: what you get when video engineers do images

WebP came out of Google's VP8 video codec in 2010, and it beats JPEG mainly through one video trick: intra-frame prediction. Instead of compressing each block independently the way JPEG does, WebP predicts each block from its already-decoded neighbors and stores only the error. It also uses flexible block sizes (4×4 up to 16×16), arithmetic coding instead of Huffman, and a deblocking filter that smooths over the seams.

Net result in practice: around 25–35% smaller than an equivalent-quality JPEG, and about 26% smaller than PNG in lossless mode. Not revolutionary, but real — and since every current browser supports it, there's little reason not to use it for web delivery. I compare the three formats head-to-head in WebP vs JPEG vs PNG if you're choosing.

How all of this runs in your browser tab

The part I care most about, because it's the reason InstaShrink exists: none of the above requires a server. WebAssembly lets the same C/C++ compression libraries the pros use — MozJPEG, cwebp — run inside a browser tab at 80–95% of native speed.

When you drop an image onto InstaShrink, the sequence is:

1. The browser's File API reads the image from disk — no network request
2. The file is decoded into raw pixel data in memory
3. The Wasm-compiled encoder runs the full compression pipeline locally
4. The compressed result is assembled as a blob in browser memory
5. Your download link points at that blob — again, no network

What I actually do with this knowledge

Theory is nice; here's how it changes day-to-day decisions. Three things, all consequences of the pipeline above:

• Quality 80 is not "80% as good". The quantization tables aren't linear. Going from 95 to 80 usually cuts the file by more than half with no visible change; going from 80 to 65 saves comparatively little and starts to show. The 75–85 range is the sweet spot for web photos, and it's where I'd start every time.
• Never re-compress a JPEG. Quantization rounds values every save. Round already-rounded data a few times and the errors compound — photographers call it generation loss. Keep an original, compress from that.
• Format choice beats slider position. A logo saved as a JPEG at quality 95 is both bigger and uglier than the same logo as a PNG. Get the format right first; fiddle with quality second.

Keep reading

• WebP vs JPEG vs PNG: how I choose between them
• Reducing JPEG size without visible quality loss
• Why in-browser compression matters for privacy