liveBook · Manning

Hidden state

Also called internal state

An internal condition or representation of a system that is not directly observable but can influence its outputs or subsequent processing.

How 4 Manning books explain it

Build AI Drug Discovery Pipelines

A hidden state is a learned intermediate vector that summarizes sequence information available to a recurrent or sequential model at a particular position.

How it is used
Uses hidden states to carry sequence information through molecular generation and to motivate alternatives for long protein sequences.
How deep it goes
Works through a simple RNN update and GRU gating, then contrasts final-state compression with attention over encoder states.
Examples
Illustrates state updates with acetic acid SMILES and a 500-amino-acid protein sequence.
What you can do after
Builds a simple RNN step from scratch using PyTorch tensor operations and interprets its hidden-state update.

Read how Build AI Drug Discovery Pipelines explains hidden state →

Rearchitecting LLMs

A hidden state is an internal vector representation produced by a neural model layer for the information processed up to that layer.

How it is used
Uses intermediate representations for distillation, expert routing, and inspection of predictions across transformer layers.
How deep it goes
Explains hidden-state extraction, tensor shapes, layer alignment requirements, and logit-lens projection.
Examples
Uses a 14-block student and 18-block teacher for distillation, and a Barcelona prompt to inspect layer predictions.
What you can do after
Diagnose predictions with a logit lens and apply hidden-state alignment during knowledge distillation.

Read how Rearchitecting LLMs explains hidden state →

Machines that Think

An internal recurrent-network representation that carries information from earlier sequence elements to later processing steps.

How it is used
Explains how sequence models retain context and how attention-based models form contextualized token representations.
How deep it goes
Explains state updates, shared weights and memory limitations, without specifying a full implementation.
Examples
Illustrates information loss across intervening phrases and contextualization of “rises” using information from “sun.”
What you can do after
Evaluate how a model carries or recomputes context and why recurrent memory can become difficult to use over long spans.

Read how Machines that Think explains hidden state →

Timeless Algorithms, The Foundational Ideas

A hidden state is an internal representation that a recurrent neural network carries forward to summarize information from earlier sequence elements.

How it is used
Summarizes earlier sequence inputs so recurrent predictions can depend on context beyond the current element.
How deep it goes
Explains fixed-size state updates, information loss, and sequential computation at the architectural level.
Examples
Uses a sentence where “book” must remain linked to “was” across intervening clauses.
What you can do after
Evaluate how recurrent state updates can weaken distant context and constrain parallel sequence processing.

Read how Timeless Algorithms, The Foundational Ideas explains hidden state →

Also covered in

What to know first

Related concepts

Questions about hidden state

What is hidden state?

An internal condition or representation of a system that is not directly observable but can influence its outputs or subsequent processing.

Which Manning books explain hidden state?

Build AI Drug Discovery Pipelines, Rearchitecting LLMs, Machines that Think and Timeless Algorithms, The Foundational Ideas.

What should I know before learning hidden state?

Manning books that teach hidden state usually introduce recurrent neural network, token, foundation model and LSTM first.

What is hidden state a kind of?

Hidden state is a kind of state. State is information describing the condition or configuration of a system or computation at a particular time that may influence its behavior or subsequent transitions.

What is hidden state also called?

Hidden state is also called internal state.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.