Entropy measures how unpredictable a probability distribution is. For outcomes with probabilities p₁ through pₙ, compute it as the negative sum of each probability multiplied by its base-2 logarithm. The result is measured in bits: higher entropy means less ability to predict the next outcome. section 2.1.2
Equation 2.2
\[\begin{aligned} H(X) &= -\sum_{x \in X} p_x \log_2 p_x \notag\\ &= - p_{1} \times \log_2 (p_{1}) - p_{2} \times \log_2 (p_{2}) - ... p_{n} \times \log_2 (p_{n}) \label{equation:entropy-binary-log} \end{aligned}\]Use the binary-log formula to total the uncertainty contributed by every possible outcome in a distribution. section 2.1.2
Biased coin toss
A fair coin has 1 bit of entropy because heads and tails are equally likely. Ten successive fair tosses have 10 bits. Bias reduces entropy: probabilities of 0.75 and 0.25 produce about 0.81 bits, while a coin that always lands tails has 0 bits. Entropy therefore measures uncertainty, not merely the number of named outcomes. section 2.1.2 section 2.1.2
When assessing randomness for cryptography, treat entropy as an engineering requirement:
- Prefer a uniform output distribution, which maximizes an RNG’s entropy. section 2.3
- Model both the entropy rate and generation rate of a physical source; the model assesses quality without predicting its actual output bits. section 2.2.1
- Continuously health-test raw physical output and stop generation if the tests fail. section 2.2.1 section 10.2
- Condition biased raw bits with a randomness extractor, then use the resulting quality entropy to seed and periodically reseed a CSPRNG. section 10.2
Poor entropy can defeat an otherwise sound cryptographic design. Encryption relies on randomness to produce different ciphertexts for the same plaintext, and password storage uses random salts. In ECDSA, guessing only a few bytes of the signing nonce can break security; selecting a weak randomness source nullifies the guarantees expected from the main algorithm. section 10.2
