The 1F1B schedule is a pipeline-parallel execution strategy that interleaves one forward pass with one backward pass across successive micro-batches. It replaces GPipe’s run-all-forwards-then-run-all-backwards pattern with earlier gradient work, reducing GPU idle time. section 6.5.4 section 6.5.8
A pipeline moves through three operating phases:
- Warm-up fills the pipeline with initial forward passes, leaving a small unavoidable startup bubble. section 6.5.4
- Steady state alternates forward and backward work: after forwarding micro-batch i, a stage processes an available gradient for an earlier micro-batch instead of waiting. section 6.5.4
- Cool-down drains the remaining backward passes after new forward work ends; in a true schedule, these final backward passes run back-to-back. section 6.5.4
GPU 2 in the steady state
GPU 2 may forward micro-batches 1 and 2 during warm-up. Once the gradient for micro-batch 1 arrives from GPU 3, it alternates backward work for that micro-batch with forward work for later micro-batches, keeping its timeline densely occupied. section 6.5.4
Figure 6.7 The 1F1B interleaved schedule. Forward (blue) and backward (green) passes alternate during the steady state. Note: any small gaps visible in this rendered timeline are artifacts of the profiling resolution, not the schedule itself. In a true 1F1B, each GPU executes its remaining backward passes back-to-back during cool-down.

Use the timeline to inspect the alternating forward and backward blocks, the reduced bubble, and the cool-down behavior. section 6.5.4
Estimate ideal schedule efficiency as micro-batches divided by micro-batches plus pipeline stages minus one. section 6.5.4
1F1B reduces, but does not eliminate, the pipeline bubble and activation-memory cost. In mixture-of-experts models, required All-to-All communication can still limit efficiency, motivating folded schedules such as DualPipe. section 6.5.8 section 6.6
