liveBook · Manning

Redundancy

As explained in Timeless Algorithms, The Foundational Ideas

Redundancy is the deliberate inclusion of repeated or structured information to enable reliable recovery of a message despite errors or noise.

Redundancy is repeated or structured information that contributes less new information than its length suggests. When patterns are predictable, removing redundancy can compress a message without losing its information. When transmission is vulnerable to noise, deliberately adding structured redundancy can help detect or correct errors. section 5.1.1 section 5.2.3

Redundancy serves two different purposes:

  • Natural redundancy: predictable patterns, such as AAAAAA, can be represented compactly as A × 6. section 5.1.1
  • Deliberate redundancy: added structure uses more transmitted bits to help a receiver recover a message corrupted by noise. section 5.2.3

Repeating a message

If YES arrives as YXS, the error may be hard to detect. Sending each letter three times—YYYEEESSS—allows recovery by majority voting, though the added redundancy reduces transmission efficiency. Shannon’s noisy channel coding theorem states that below channel capacity, coding schemes can make the probability of error arbitrarily small. section 5.2.3

Figure 5.1 Entropy measures the uncertainty associated with an information source in bits. A fair coin has an entropy of 1 bit, while a biased coin has lower entropy (0.47 bits) because one outcome is much more predictable than the other. A fair die (2.59 bits) and a standard deck of cards (5.7 bits) have progressively higher entropy because they contain larger sets of equally likely outcomes. As entropy increases, preserving the original information becomes increasingly important because there is less redundancy available for compression. Conversely, as entropy decreases, greater redundancy creates more opportunities to compress the source without losing information.

Compare entropy across sources to see how predictability affects the redundancy available for compression. section 5.1.4

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains compression and majority decoding, then states the channel-coding limit without detailing real code designs.
Examples
Uses AAAAAA, W7dX8$, and tripled YES to illustrate compression and recovery from errors.
What you can do after
Evaluate when redundancy can be removed for compression and when adding it improves recovery from transmission errors.
How it connects
Connects redundancy to entropy, compression, channel capacity, noise, and error-correcting codes.

Also explained in

Compare how 13 books explain redundancy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.