A state represents the current situation in a decision process, retaining the information relevant to choosing an action. In an MDP, states connect the agent, actions, rewards, environment, and policy. In an RNN, recurrent state carries context forward as sequence items are processed. section 5.1.1 section 1.1.1
Use state in two related architectures:
- MDP state: identifies the situation the agent currently faces; the policy uses it to select the next action, while the action and resulting events lead to a subsequent state. section 5.1.1
- RNN recurrent state: updates after each sequence item and preserves contextual information about what has already been processed. section 1.1.1
- These roles differ in purpose: MDP states describe decision situations, whereas recurrent states provide sequence-processing memory. section 1.1.1 section 5.1.1
Obstacle-course decisions
An obstacle course supplies the environment, each obstacle is a state, and the dog is the agent. Seeing an obstacle, the dog chooses an action such as jumping, crawling, or sidestepping, then receives positive or negative feedback. section 5.1.1
Figure 1.1 A high-level overview of the general flow of sequential data within an RNN. The encoder in an RNN processes the input sentence one word at a time, updating its internal state with each word. This culminates in a final context vector representing the entire sentence. The decoder then uses this context vector to generate the output sentence, again one word at a time, using its internal (recurrent) state to remember the previously generated words.
Trace how an RNN encoder updates its internal state one word at a time, produces a final context vector, and passes it to a decoder whose recurrent state tracks generated words. section 1.1.1
