A multistep answer must build on what has already been said. Autoregressive generation produces such a sequence one token at a time: the model predicts a token, appends it to the available context, and uses that expanded context to predict the next. Contextual representation, output logits, softmax, and token selection are repeated as the response grows. section 12.2.3
In a transformer, causal masking prevents a position from attending to later tokens. The mask is added to attention compatibility logits before softmax: allowed positions receive 0, while blocked positions receive −∞. Because e^(−∞) = 0, masked positions receive zero attention weight. Each position can therefore use only itself and earlier context. section 12.2.3
Each row identifies the position being represented; each column identifies a position it might attend to. The upper triangle blocks future positions. section 12.2.3
A train overtaking problem
A slower train travels for one hour at 100 km/h before a faster train leaves at 120 km/h. A response can establish the 100 km head start, calculate the 20 km/h relative speed, and divide distance by relative speed to obtain five hours. Each generated part becomes context for subsequent tokens. section 12.2.3
Transformer training can calculate representations for many positions in parallel while causal masking preserves the next-token objective. Generation remains sequential: a later token depends on the tokens selected before it. Hardware can accelerate each prediction but cannot remove that dependency, so longer responses increase latency and inference cost. section 12.3.2 section 12.4.2
Successive predictions can draw on relationships across the available context, but availability does not guarantee correct use. Relevant details may receive insufficient attention or be interpreted incorrectly; fluent continuations can also be wrong when learned associations are incomplete or conflicting. section 12.1.2 section 12.4.2 section 12.2.2


