Public key validation checks that a received public key belongs to the intended cryptographic group. Without it, a malicious Diffie–Hellman key can place the exchange in a small subgroup and reveal information about the private key. section 5.4
A basic validation check works as follows:
- Raise the received public key to the order of the intended subgroup. section 5.4
- Accept it only when the result is the identity element, showing that the key belongs to that subgroup. section 5.4
- For elliptic curves, also account for invalid-curve attacks: prime-order groups do not automatically eliminate validation risks. section 5.4
A one-bit leak
An attacker sends a malicious public key from a small subgroup. The shared secret becomes 1 when the private key is even and −1 when it is odd, revealing its least significant bit; repeated attacks can expose more. section 5.4
Figure 5.15 A small subgroup attack impacts DH groups that have many subgroups. By choosing generators of small subgroups as public keys, an attacker can leak bits of someone’s private key little by little.
Visualizes how choosing small-subgroup generators as public keys leaks private-key bits incrementally. section 5.4
