Recurrence
A recurrence is a relation or computational process in which each successive value or state is defined using preceding values or states.
Natural Language Processing in Action, Second Edition
A recurrence loop is a computation pattern that repeatedly feeds a step's state or output into the next step of sequence processing.
- How it is used
- Uses recurrent state reuse to process sequences, remember earlier tokens, and predict subsequent tokens or generate text.
- How deep it goes
- Explains recurrence through unrolling, hidden-state flow, termination, and a minimal PyTorch implementation, while contrasting transformer recursion.
- Examples
- Uses token-by-token RNN processing, a recurrence-loop diagram, and a minimal PyTorch RNN returning predictions and hidden states.
- What you can do after
- Build a minimal recurrent neural network in PyTorch that carries hidden state across tokens and produces sequential predictions.
Read how Natural Language Processing in Action, Second Edition explains recurrence →
Sutskever's List
Recurrence is an architectural mechanism that performs repeated state or representation updates across sequential or iterative computation steps.
- How it is used
- Frames sequence order as an architectural bias against which Transformer performance is assessed.
- How deep it goes
- Explains recurrence as an order-related inductive bias and weighs its costs against sequence-task results.
- Examples
- Uses Penn Treebank parsing results and word-reordering tests to compare recurrent models with Transformers.
- What you can do after
- Evaluate recurrence as a useful sequence-order prior rather than a requirement for strong sequence-task performance.
Advanced Algorithms and Data Structures
A recurrence relation is an equation that defines a sequence or function in terms of values for preceding or smaller instances.
- How it is used
- Analyzes recursive algorithm performance by translating branching and subproblem sizes into running-time equations.
- How deep it goes
- Defines recurrences, identifies base cases, and solves a factorial example, but defers broader solution techniques to an appendix.
- Examples
- Uses graph planarity testing, T(n)=nT(n−1) with T(0)=1, and a three-vertex recursion example.
- What you can do after
- Diagnose recursive algorithms by deriving growth rates and recognizing factorial behavior that limits practical input size.
Read how Advanced Algorithms and Data Structures explains recurrence →
Related concepts
- transformer architecturecontrasts with · A transformer architecture is a neural-network architecture that processes sequences or other structured inputs using attention mechanisms and related transformations.
- base caseconfigured by · A base case is a condition in a recursive or inductive computation that can be solved directly without further expansion.
- recurrent neural networkuses · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
- dynamic programmingused by · Dynamic programming is an algorithmic technique that solves problems by decomposing them into overlapping subproblems and storing or reusing their solutions.
- knapsack problemuses · The knapsack problem is an optimization problem that selects items subject to a capacity constraint to maximize their combined value.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.