liveBook · Manning

State

As explained in Deep Learning with R, Third Edition

A neural-network state is an internal representation that carries information from earlier sequence elements to later processing steps.

A neural-network state is an internal representation that carries information from earlier sequence elements to later processing steps. Unlike feedforward networks, recurrent neural networks update this state as they iterate through a sequence, then reset it between independent sequences. section 13.3.1

For a toy RNN, the state starts at zero. At each timestep, the network combines the current input with the previous state, applies an activation, and stores the result as the next state. The sequence of outputs preserves the state at every timestep.

Generating text one token at a time

During autoregressive generation, a GRU receives one character and an explicitly passed state. A prompt primes the state with information about the entire preceding text; each predicted character is then fed back with the updated state to produce the next character. section 15.1.2

How Deep Learning with R, Third Edition covers it

How deep it goes
Explains initialization, recurrent updates, explicit propagation, and sequence resets, without deriving training dynamics or long-term memory limitations.
Examples
Uses a toy RNN loop, an R implementation, and Shakespeare character generation with a prompted GRU state.
What it assumes
Assumes sequence inputs and neural-network layers, while introducing recurrent memory, state initialization, and state passing operationally.
What you can do after
Build it from scratch with an RNN loop and propagate an explicit GRU state during token-by-token generation.
How it connects
Connects state to recurrent neural networks, RNNs, GRUs, sequence processing, and autoregressive models.
Cover of Deep Learning with R, Third Edition
liveBookmanning.com
Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

Compare how 39 books explain state →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.