liveBook · Manning

Huffman encoding

As explained in Math and Architectures of Deep Learning

A prefix-coding method that builds variable-length codes from symbol frequencies to compress data efficiently.

Huffman encoding represents symbols with variable-length, prefix-free bit codes: frequent symbols receive shorter codes, while every code remains immediately decodable. It reduces storage when a distribution is uneven, but can perform worse than fixed-rate coding when symbols are equally probable. section 6.2

G(reen) = 0
R(ed) = 10
B(lue) = 110
Y(ellow) = 111

The prefix code assigns the most frequent color one bit and less frequent colors progressively longer codes. section 6.2

Compressing a 4 × 4 image

For 12 green, 2 red, 1 blue, and 1 yellow pixels, codes of lengths 1, 2, 3, and 3 require 22 bits, or 1.375 bits per pixel, versus 32 bits with two-bit fixed coding. A balanced four-color image instead requires 36 bits, or 2.25 bits per pixel. section 6.2

How Math and Architectures of Deep Learning covers it

How deep it goes
Explains prefix decoding and bit-count calculations, then connects compression effectiveness to entropy without deriving the construction algorithm.
Examples
Uses two 4 × 4 pixel-color images and compares fixed two-bit codes with variable codes.
What you can do after
Evaluate when variable-length coding improves image storage by comparing symbol frequencies, code lengths, and bits per pixel.
How it connects
Connects Huffman encoding with entropy, image compression, probability distributions, and fixed-rate coding.
Cover of Math and Architectures of Deep Learning
liveBookmanning.com
Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

Compare how 4 books explain Huffman encoding →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.