liveBook · Manning

Episode

As explained in Applied Reinforcement Learning

A finite sequence of agent-environment interactions beginning from an initial state and ending when a task terminates.

An episode is a complete run of an agent’s interaction with its environment, from a starting state to task termination. Its endpoint can mark success or failure. section 3.2 section 10.3

The endpoint and reward timing depend on the task:

  • In warehouse picking, the robot starts at the depot, collects items, and aims to return before the step limit. section 3.2
  • In language-model fine-tuning, one query prompts a complete response; the episode ends at the end-of-sequence token, and a reward follows. section 10.3

This contrast matters: warehouse interactions can provide rewards along the way, while language-model responses typically receive a single reward at the end. That sparse feedback makes learning harder. section 10.3

Figure 10.2 Language generation mapped as a Markov Decision Process.

See how language generation is represented as a Markov decision process, with a response forming an episode. section 10.3

How Applied Reinforcement Learning covers it

How deep it goes
Defines endpoints through warehouse completion or a response-ending token, and contrasts intermediate with terminal rewards.
Examples
Uses warehouse order picking and one-query, one-response language generation as contrasting episodes.
What you can do after
Understand how task boundaries determine when an episode ends and when rewards arrive.
How it connects
Connects episodes to Markov decision processes, rewards, and reinforcement learning from human feedback.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.