A bit, or binary digit, is the unit for expressing entropy. It represents uncertainty in a choice between two equally likely alternatives: information is measured by the uncertainty it resolves, not by meaning or message length. section 5.1.3 section 5.0
For a fair coin, the two outcomes are equally likely; the entropy calculation gives one bit of uncertainty before the outcome is observed. section 5.1.3
More outcomes or less predictable outcomes require more bits on average. This common scale makes different information sources comparable and helps quantify information and redundancy in communication. section 5.1.3 section 5.6





