PPO clipping limits how much a policy update benefits from changing an action’s probability. The probability ratio compares the new policy with the old one; with ε = 0.2, the objective clips that ratio to 0.8–1.2. This discourages large policy jumps. section 10.7
The clipped objective shows how PPO compares the unclipped and clipped ratio terms, using the smaller one. section 10.7
A large probability increase
For a positive advantage of 2, a ratio of 1.2 contributes 2.4 to the objective. A ratio of 1.8 is clipped to 1.2, so it contributes no more. Negative advantages likewise limit how sharply a token’s probability can fall in one update. section 10.7
