liveBook · Manning

Experience

As explained in Applied Reinforcement Learning

Experience is a recorded interaction tuple containing a state, action, reward, and next state.

In reinforcement learning, an experience is a sampled transition: a state, the action taken, the reward received, and the next state. Model-free methods learn from these interactions without relying on a transition function; this can require more interactions, computation, and time. section 4.0

Figure 12.1 The anatomy of a deep reinforcement learning algorithm.

See how the experience source fits among the main parts of a deep reinforcement learning algorithm. section 12.1

How Applied Reinforcement Learning covers it

How deep it goes
Defines the state-action-reward-next-state tuple and contrasts learning from experience with using a model.
What it assumes
Assumes a problem has been represented as a Markov decision process.
What you can do after
Understand how sampled transitions let model-free methods learn without a transition function.
How it connects
Connects experience to Markov decision processes, model-free methods, transition functions, and rewards.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.