liveBook · Manning

Coding theory

As explained in Timeless Algorithms, The Foundational Ideas

Coding is the systematic representation of messages as symbols or codewords for efficient transmission, storage, or error protection.

Noise can corrupt a message even when a channel has enough capacity to carry it. Channel coding addresses this risk by adding structured redundancy: it uses more transmitted symbols to help a receiver detect and correct errors. Shannon’s noisy channel coding theorem says that when the transmission rate stays below channel capacity, codes exist that make the probability of error arbitrarily small. section 5.2.3

Repeated symbols

If YES arrives as YXS, the error may be hard to detect. Sending each letter three times—YYYEEESSS—lets the receiver recover the message by majority vote, even if individual symbols are corrupted. Practical codes are more sophisticated, but use the same principle: trade transmission efficiency for reliability. section 5.2.3

This redundancy differs from redundancy removed during compression: one improves reliability, while the other improves efficiency. Error-correcting codes support reliable transmission and storage in systems including Wi-Fi, QR codes, and satellite links. section 5.2.3

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
States the noisy channel coding theorem and illustrates repetition coding, but leaves sophisticated code designs unexplained.
Examples
Illustrates majority voting with repeated YES symbols and contrasts 1,000-bit input with 700-bit compression.
What you can do after
Understand how deliberately added redundancy can reduce transmission errors at the cost of efficiency.
How it connects
Connects channel coding to channel capacity, data compression, and error-correcting codes.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.