liveBook · Manning

LinUCB

As explained in Applied Reinforcement Learning

A contextual bandit algorithm that models expected reward as a linear function of context and uses upper confidence bounds to balance exploration and exploitation.

When choosing a discount for each customer, a single best-average offer can miss how customer profiles affect revenue. LinUCB uses context to personalize that choice while still exploring uncertain options. section 5.5

For each action, LinUCB learns feature weights that estimate expected reward from the current context. It adds an upper-confidence bonus to favor actions whose rewards remain uncertain, balancing exploitation with exploration. section 5.5

Personalized discounts

Treat discount levels as actions, customer and time signals as context, and transaction revenue as reward. The approach assumes rewards are linear in features and behavior is sufficiently stable; sudden market changes can violate those assumptions. section 5.5

How Applied Reinforcement Learning covers it

How deep it goes
Explains per-action feature weights and confidence-based exploration, then shows agent code without deriving the algorithm mathematically.
Examples
Uses customer and time signals to choose discount levels and learn from transaction revenue.
What you can do after
Apply LinUCB to select personalized discounts and update its estimates from observed rewards.
How it connects
Connects contextual bandits and UCB exploration with an epsilon-greedy baseline.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.