liveBook · Manning

Forgetting

As explained in Applied Reinforcement Learning

Forgetting is an adaptation technique that reduces the influence of older observations when estimating changing payoff relationships.

Forgetting adapts decision-making to changing markets by reducing the influence of older observations when estimating payoff relationships. If an advertisement's click-through rate changes, relying too heavily on yesterday's evidence can preserve a bad choice. section 5.6

A campaign that has moved on

Customer tastes evolve, creative performance decays, competitors launch campaigns, and seasonality reshapes traffic. Discounting older campaign outcomes helps a bandit remain responsive instead of treating stale evidence as equally relevant. section 5.6

Forgetting belongs to a small set of responses to non-stationarity:

  • Discount older data so recent outcomes carry more influence. section 5.6
  • Use sliding windows that retain only recent outcomes. section 5.6
  • Reset estimates when a meaningful change is detected. section 5.6

How Applied Reinforcement Learning covers it

How deep it goes
Explains the motivation and compares discounting, sliding windows, and resets without detailing their update formulas.
Examples
Uses changing advertisement, email, discount, creative, competitor, and seasonal performance as motivating examples.
What it assumes
Assumes familiarity with bandit decisions and stationary payoffs before introducing changing environments.
What you can do after
Apply it to keep bandit choices responsive when campaign payoffs or customer behavior change over time.
How it connects
Connects forgetting to contextual bandits, non-stationarity, sliding windows, change detection, and exploration.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.