A clipped straight-through estimator (STE) approximates gradients through fake quantization: it passes gradients for values within the representable range and zeros them for clipped values. section 5.2
The backward pass treats the quantization boundary differently from the basic STE:
- Inside the range, pass the gradient through unchanged. section 5.2
- Outside the range, zero the gradient so optimization does not keep pushing saturated values farther out. section 5.2
Use the clipped variant by default for fake-quantized training. Basic STE passes gradients everywhere; scaled and tanh variants are also described, but have no measured advantage on the stated workloads. section 5.2.1
Figure 5.8 Visualizing the clipped STE gradient function. Top: the fake quantization staircase with "quantizable range" and "clipped range." Bottom: gradient behavior - basic STE (dashed) passes gradient 1 everywhere; clipped STE zeros gradients in the clipped regions.

The plot shows the fake-quantization staircase alongside the clipped STE's gradient: unchanged in range and zero in clipped regions. section 5.2
