liveBook · Manning

Online learning

As explained in Applied Reinforcement Learning

Online learning updates a model incrementally as data or interaction outcomes become available rather than waiting for a complete dataset or episode.

Online learning updates an agent’s knowledge continuously as new information arrives during interaction, rather than waiting for a complete dataset. section 6.1 section 6.7

Business adaptation

A business can adjust prices, inventory, or marketing in real time as live market signals arrive, instead of waiting to analyze a completed period. section 6.1

In reinforcement learning, temporal-difference methods support online updates by correcting estimates step by step from experience. They learn from a next-step estimate rather than waiting for the final outcome. section 6.1 section 6.7

The timing of updates creates a practical trade-off:

  • Temporal-difference learning updates quickly but can be short-sighted; Monte Carlo learning waits for episode results and has high variance. section 6.5
Figure 6.7 Comparison of TD(0), Monte Carlo and TD(λ) methods.

Compare TD(0), Monte Carlo, and TD(λ) as approaches spanning immediate updates and episode-based evaluation. section 6.5

How Applied Reinforcement Learning covers it

How deep it goes
Contrasts continuous updates with batch training and connects online learning to TD, without specifying a general-purpose procedure.
Examples
Uses real-time business adjustments to prices, inventory, or marketing in response to live market signals.
What you can do after
Apply it by updating decisions as live market signals arrive, such as when adjusting prices, inventory, or marketing.
How it connects
Connects online learning to temporal-difference learning, Monte Carlo methods, SARSA, and Q-learning.

Also explained in

Compare how 2 books explain online learning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.