An invalid curve attack tricks a key-exchange implementation into multiplying a received point from a different elliptic curve. The attacker chooses a related curve with small subgroups, so the resulting exchange can leak pieces of the victim’s private key. section 5.4
The attack follows a short chain:
- Short-Weierstrass scalar-multiplication formulas do not use the curve parameter b, allowing related curves to share implementation formulas. section 5.4
- The attacker sends a point on a related curve containing exploitable small subgroups. section 5.4
- The server multiplies that point during key exchange, unintentionally performing the operation in the wrong group. section 5.4
Defend by validating public keys: reject the point at infinity and verify that the received coordinates satisfy the intended curve equation. For ECDH, X25519 is recommended because its design and implementations account for this threat. section 5.4
Figure 5.15 A small subgroup attack impacts DH groups that have many subgroups. By choosing generators of small subgroups as public keys, an attacker can leak bits of someone’s private key little by little.
Use the small-subgroup diagram to visualize how carefully chosen public points can reveal private-key bits incrementally. section 5.4
