liveBook · Manning

Reinforcement learning

As explained in Applied Reinforcement Learning

Reinforcement learning is a machine-learning paradigm in which an agent learns to choose actions through interactions with an environment and feedback about their consequences.

A business setting prices or managing inventory faces a moving target: each decision affects what happens next, while customers, competitors, and supply conditions keep changing. Reinforcement learning (RL) is a way to learn decisions through interaction and feedback rather than relying only on fixed rules. section 1.0 section 1.7

An agent takes an action in an environment and observes its consequences, including a reward or penalty. Over repeated interactions, it adjusts its policy—the strategy it uses to choose actions. The goal is not just an immediate payoff: a useful policy weighs short-term rewards against longer-term outcomes. section 1.1 section 1.7

Translate a business decision into the parts an agent can learn from:

  • State: the information describing the situation, such as customer or market conditions. section 5.0 section 12.2
  • Actions: the choices available to the agent, such as setting a price or deciding how to invest. section 1.7 section 12.2
  • Reward and constraints: feedback that expresses the objective while accounting for limits and trade-offs. section 1.8 section 12.2

Unlike supervised learning, which learns from labeled examples, RL learns how to act from the consequences of its choices. A simple starting point is the multi-armed bandit: choose an action, observe a reward, and update beliefs. It highlights the trade-off between exploring uncertain choices and exploiting choices that already look promising. A contextual bandit also observes information such as a customer profile before choosing. section 1.1 section 5.0

A safe sandbox for pricing

A dynamic-pricing agent could explore how pricing decisions affect outcomes for perishable products. Because an untrained agent may make many poor choices, a simulated environment can represent essential dynamics, constraints, and feedback before real-world testing. The simulation makes experimentation safer, but its design matters to what the agent learns. section 3.0 section 1.7

RL can also apply when actions are tokens rather than physical moves. In language generation, the growing conversation is the environment, and feedback can reflect human judgments about whether a response is safe and helpful. This targets qualities that next-token prediction alone does not explicitly optimize. section 10.1

RL complements rather than replaces classical optimization. It can learn from experience when a system is difficult to specify in advance, but training may require many interactions or simulations, and decisions can be hard to explain. A working algorithm is only part of a useful system: state representation, reward design, data, and evaluation all matter. For example, using tomorrow’s closing price in a portfolio feature can make a backtest impossible to reproduce live. section 1.8 section 12.2

Table 1.3 Comparing Classical Models and Reinforcement Learning
Metric
Classical Models
Reinforcement Learning
Robustness
Good under fixed assumptions, but sensitive to changes
Learns to handle variation through exposure and experience
Resilience
Needs re-solving or human intervention when disrupted
Recovers and adapts by updating the policy over time
Real-time responsiveness
Depends on model size—can be very slow for mid or large-scale problems
Fast inference once trained; learning takes time upfront
Adaptability
Not adaptive—new scenario = new model
Built for change; adapts to evolving dynamics
Flexibility
Difficult to modify once built
Modular and extendable; constraints and goals can be learned
Generalizability
Problem-specific; often doesn’t transfer well
Can generalize across similar environments with right training setup
Customizability
Rigid logic; requires rebuilding for each variation
Easily tuned for different business KPIs or scenarios
Effort to build
High upfront effort to define everything
High effort too—but more in terms of data, tuning, and experimentation
Effort to operationalize
Often under-estimated; integration into workflows can be painful
Needs proper pipelines, monitoring—but can be API-ready after training
Lifecycle cost
Low maintenance if environment is stable, but needs to be refined once the environment is dynamic
Ongoing maintenance needed; drift, re-training, fine tuning, and updates
Interpretability
Transparent and explainable, since they are mostly biased toward designers mindsets
Often opaque—though this is improving with new research in explainability

Compare classical optimization and RL across adaptability, responsiveness, and other decision-making criteria. section 1.8

How Applied Reinforcement Learning covers it

How deep it goes
Explains interaction and policy learning, then extends the treatment to environment design and system-level reward and evaluation choices.
Examples
Uses dynamic pricing, portfolio management, and language generation to illustrate agents, feedback, and design choices.
What you can do after
Evaluate/understand when reinforcement learning fits a decision problem and what simulation, reward, and evaluation choices matter.
How it connects
Connects reinforcement learning with supervised learning, classical optimization, contextual bandits, and large language models.

Also explained in

Compare how 28 books explain reinforcement learning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.