Recurrence
A recurrence is a relation or computational process in which each successive value or state is defined using preceding values or states.
Natural Language Processing in Action, Second Edition
A recurrence loop is a computation pattern that repeatedly feeds a step's state or output into the next step of sequence processing.
- How it is used
- Uses recurrent state reuse to process sequences, remember earlier tokens, and predict subsequent tokens or generate text.
- How deep it goes
- Explains recurrence through unrolling, hidden-state flow, termination, and a minimal PyTorch implementation, while contrasting transformer recursion.
- Examples
- Uses token-by-token RNN processing, a recurrence-loop diagram, and a minimal PyTorch RNN returning predictions and hidden states.
- What you can do after
- Build a minimal recurrent neural network in PyTorch that carries hidden state across tokens and produces sequential predictions.
Read how Natural Language Processing in Action, Second Edition explains recurrence →
Timeless Algorithms, The Foundational Ideas
A rule that defines a quantity in terms of its values for other instances.
- How it is used
- Expresses a larger problem in terms of smaller subproblems, enabling dynamic programming to combine their solutions.
- How deep it goes
- Defines Fibonacci, knapsack, and coin-change recurrences, then connects recursive decomposition to tabulation and memoization.
- Examples
- Uses Fibonacci numbers, knapsack choices, coin-change combinations, and shortest routes through networks.
- What you can do after
- Understand how to express a problem as a recurrence and why exact evaluation can yield a global optimum when optimal substructure holds.
Read how Timeless Algorithms, The Foundational Ideas explains recurrence →
Sutskever's List
Recurrence is an architectural mechanism that performs repeated state or representation updates across sequential or iterative computation steps.
- How it is used
- Frames sequence order as an architectural bias against which Transformer performance is assessed.
- How deep it goes
- Explains recurrence as an order-related inductive bias and weighs its costs against sequence-task results.
- Examples
- Uses Penn Treebank parsing results and word-reordering tests to compare recurrent models with Transformers.
- What you can do after
- Evaluate recurrence as a useful sequence-order prior rather than a requirement for strong sequence-task performance.
Advanced Algorithms and Data Structures
A recurrence relation is an equation that defines a sequence or function in terms of values for preceding or smaller instances.
- How it is used
- Analyzes recursive algorithm performance by translating branching and subproblem sizes into running-time equations.
- How deep it goes
- Defines recurrences, identifies base cases, and solves a factorial example, but defers broader solution techniques to an appendix.
- Examples
- Uses graph planarity testing, T(n)=nT(n−1) with T(0)=1, and a three-vertex recursion example.
- What you can do after
- Diagnose recursive algorithms by deriving growth rates and recognizing factorial behavior that limits practical input size.
Read how Advanced Algorithms and Data Structures explains recurrence →
What to know first
- convolutional neural networkA convolutional neural network is a neural-network architecture that processes structured inputs with convolutional operations, typically using learned local filters to capture patterns.
- LSTMA long short-term memory (LSTM) network is a recurrent neural-network architecture with gated memory cells that control information flow and model dependencies in sequential data.
Related concepts
- transformer architecturecontrasts with · A transformer is a neural-network architecture that uses attention-based interactions and position-wise transformations to model relationships among elements of sequences or other structured inputs.
- base caseconfigured by · A base case is a condition in a recursive or inductive computation that can be resolved without further recursive expansion.
- recurrent neural networkuses · A recurrent neural network is a neural network that processes ordered inputs while carrying and updating an internal state across successive steps.
- dynamic programmingpart of · Dynamic programming is an algorithmic technique that solves problems by decomposing them into overlapping subproblems and storing or reusing their solutions.
- equationkind of · An equation is a mathematical statement asserting that two expressions are equal.
Questions about recurrence
What is recurrence?
A recurrence is a relation or computational process in which each successive value or state is defined using preceding values or states.
Which Manning books explain recurrence?
Natural Language Processing in Action, Second Edition, Timeless Algorithms, The Foundational Ideas, Sutskever's List and Advanced Algorithms and Data Structures.
What should I know before learning recurrence?
Manning books that teach recurrence usually introduce convolutional neural network and LSTM first.
What is the difference between recurrence and transformer architecture?
A recurrence is a relation or computational process in which each successive value or state is defined using preceding values or states. A transformer is a neural-network architecture that uses attention-based interactions and position-wise transformations to model relationships among elements of sequences or other structured inputs.
What is recurrence a kind of?
Recurrence is a kind of equation. An equation is a mathematical statement asserting that two expressions are equal.
What is recurrence part of?
Recurrence is part of dynamic programming. Dynamic programming is an algorithmic technique that solves problems by decomposing them into overlapping subproblems and storing or reusing their solutions.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



