liveBook · Manning

Reinforcement learning

As explained in Timeless Algorithms, The Foundational Ideas

Reinforcement learning is a machine-learning approach in which an agent learns behavior through interaction with an environment and feedback about its actions.

Reinforcement learning lets an agent improve its decisions by interacting with an environment: it observes a state, takes an action, receives a reward, and observes the next state. It is useful when the transition probabilities or rewards are not known in advance. section 6.5.3

Unlike dynamic programming with a known model, reinforcement learning estimates values from experience. It retains Bellman’s recursive logic: a decision’s value depends on its immediate reward and the value of what follows. Q-learning, for example, updates estimates for state-action choices as experience accumulates. section 6.5.3

Because decisions unfold in sequences, an agent can evaluate trajectories of states, actions, and outcomes. Their probabilities combine the conditional probabilities of successive transitions, supporting comparisons between strategies for maximizing long-term reward. section 2.4.2

Robot navigation

A robot in an unfamiliar environment learns without a model of where an action will lead or how rewarding the result will be; it uses observed outcomes to improve later decisions. section 6.5.3

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Traces the shift from known models to experience-based value estimates, but gives no algorithmic implementation.
Examples
Uses robot navigation and state-action-outcome trajectories to illustrate sequential learning.
What you can do after
Understand how agents improve decisions from experience while retaining Bellman’s immediate-reward and future-value logic.
How it connects
Connects reinforcement learning to dynamic programming, Markov decision processes, Bellman equations, Q-learning, and Markov chains.

Also explained in

Compare how 28 books explain reinforcement learning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.