Recursion recycles an operation’s output as input for another step until the computation terminates. A recursive function calls itself; an autoregressive model reuses each predicted token to predict the next one. section 9.1
The technique matters because recycling enables sequential prediction while requiring careful control:
- Distinguish recursion from recurrence: recurrence specifically describes RNN units recycling outputs into their own inputs, while recursion also includes systems such as transformers. section 9.1
- Terminate safely by reducing the remaining work at each recycling step; otherwise, recursive computation can continue indefinitely or break a program. section 8.0
- Apply recursive prediction to next-token generation, real-time transcription and translation, algorithmic trading, and code generation. section 8.0
Autoregressive transformer generation
A transformer runs its network to produce one token embedding, recycles that output into the input, and runs again to predict the next token. Unlike an RNN, it contains no recurrent neurons and does not require inner-layer unrolling. section 9.1
Figure 9.1 Autoregressive transformers “recycle” their outputs into their input.
Trace how an autoregressive transformer recycles generated outputs into successive inputs. section 9.1
