liveBook · Manning

Reinforcement learning

Also called RL

Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.

How 28 Manning books explain it

Reinforcement Learning from Human Feedback

Reinforcement learning is a machine learning paradigm in which an agent learns behavior by optimizing rewards obtained through interaction with an environment.

How it is used
Uses reinforcement learning to optimize language-model behavior from preference, verifiable, or environment feedback.
How deep it goes
Develops the trajectory and discounted-return formulation, then explains policy-gradient trade-offs and practical RLHF training loops.
Examples
Illustrates reinforcement learning with a thermostat, language-model completions scored by a reward model, and arithmetic and navigation generalization tests.
What you can do after
Evaluate reinforcement learning objectives and post-training trade-offs, including reward design, infrastructure cost, stability, and generalization.

Read how Reinforcement Learning from Human Feedback explains reinforcement learning →

Applied Reinforcement Learning

Reinforcement learning is a machine-learning paradigm in which an agent learns to choose actions through interactions with an environment and feedback about their consequences.

How it is used
Frames reinforcement learning as a way to improve sequential business decisions under uncertainty and changing conditions.
How deep it goes
Explains interaction and policy learning, then extends the treatment to environment design and system-level reward and evaluation choices.
Examples
Uses dynamic pricing, portfolio management, and language generation to illustrate agents, feedback, and design choices.
What you can do after
Evaluate/understand when reinforcement learning fits a decision problem and what simulation, reward, and evaluation choices matter.

Read how Applied Reinforcement Learning explains reinforcement learning →

Optimization Algorithms: AI techniques for design, planning, and control problems

Reinforcement learning is a machine-learning paradigm in which an agent learns action policies from interactions and evaluative feedback.

How it is used
Applies reinforcement learning to search and optimization by learning behavior through environmental interaction.
How deep it goes
Defines reward-driven learning, distinguishes problem and algorithm classes, and explains PPO clipping at a conceptual level.
Examples
Uses contextual bandit choices to illustrate how the best action can change with side information.
What you can do after
Evaluate and understand whether a decision problem calls for an MDP or a bandit formulation.

Read how Optimization Algorithms: AI techniques for design, planning, and control problems explains reinforcement learning →

Build a DeepSeek Model (From Scratch)

Reinforcement learning is a machine-learning paradigm in which an agent improves its behavior by optimizing rewards received from interactions or evaluations.

How it is used
Uses reinforcement learning to post-train a pretrained language model toward useful reasoning behavior.
How deep it goes
Moves from the interaction framework through policy gradients, PPO and GRPO to PyTorch training excerpts, while excluding massive-scale distributed infrastructure.
Examples
Uses a tennis-ball arithmetic rollout, the R1-Zero and R1 pipelines, and a Qwen2.5-32B comparison with distillation.
What you can do after
Build a PyTorch GRPO training path using verifier rewards and training steps.

Read how Build a DeepSeek Model (From Scratch) explains reinforcement learning →

Evolutionary Deep Learning: Genetic algorithms and neural networks

Reinforcement learning is a machine-learning paradigm in which an agent learns behavior by interacting with an environment and receiving rewards or other evaluative feedback.

How it is used
Frames reward-driven agent learning and supplies Gym environments for testing decision-making agents.
How deep it goes
Explains the state–action–reward loop and implements tabular Q-learning, while deferring advanced deep Q-learning treatment.
Examples
Uses pet training, a 4-by-4 FrozenLake agent, Gym environment exercises, and reward-scored NEAT agents.
What you can do after
Build it from scratch as a tabular Q-learning agent and evaluate it in a Gym environment.

Read how Evolutionary Deep Learning: Genetic algorithms and neural networks explains reinforcement learning →

Sutskever's List

Machine learning in which an agent learns behavior through rewards or penalties from interaction with an environment.

How it is used
Positions reinforcement learning as an early OpenAI research direction pursued through a research platform and game-playing milestones.
How deep it goes
Mentions the approach and its applications but does not explain training algorithms or reward design.
Examples
Cites OpenAI Gym and AI-driven Dota 2 gameplay as early milestones.
What you can do after
Evaluate/understand it as an early research direction at OpenAI rather than implement a reinforcement learning algorithm.

Read how Sutskever's List explains reinforcement learning →

Deep Learning with R, Third Edition

Reinforcement learning is a machine-learning technique in which an agent learns behavior through interaction with an environment and feedback such as rewards.

How it is used
Uses reinforcement learning to improve language-model responses through preference feedback or verifiable answers.
How deep it goes
Explains simplified training loops and reward modeling, while noting that practical gradient updates are more complex.
Examples
Illustrates preference-ranked chatbot responses, checked math solutions, and generated code tested by execution.
What you can do after
Evaluate and understand simplified reinforcement-learning loops for language-model training and their practical limitations.

Read how Deep Learning with R, Third Edition explains reinforcement learning →

Build AI Drug Discovery Pipelines

Reinforcement learning is a machine-learning approach in which an agent learns sequential actions through feedback from rewards or penalties.

How it is used
Uses sequential reward-driven exploration to steer generative models toward molecules satisfying drug-discovery objectives.
How deep it goes
Explains agents, actions, rewards, latent-space exploration, and policy updates, while deferring formal RL development to an appendix.
Examples
Demonstrates latent molecular optimization through MOLRL, VAE-CYC, GENTRL, and the TNIK-focused rentosertib program.
What you can do after
Apply reinforcement learning to configure rewards and optimize generated molecules against multiple predicted properties.

Read how Build AI Drug Discovery Pipelines explains reinforcement learning →

Build a Robo-Advisor with Python (From Scratch)

Reinforcement learning is a machine-learning approach in which an agent learns a decision policy through interactions with an environment and feedback from rewards or penalties.

How it is used
Applies reinforcement learning to multiperiod financial planning decisions involving investing, spending, retirement, and withdrawals.
How deep it goes
Works through state grids, simulated transitions, epsilon exploration, episodes, and tabular Q-learning updates; defers broader algorithm variants.
Examples
Uses a wealth-and-time grid, stock/bond allocation actions, simulated wealth transitions, and repeated Q-learning episodes.
What you can do after
Build it from scratch as a tabular Q-learning policy for simulated multiperiod wealth and allocation decisions.

Read how Build a Robo-Advisor with Python (From Scratch) explains reinforcement learning →

Grokking AI Algorithms, Second Edition

Reinforcement learning is a machine-learning paradigm in which an agent learns behavior through rewards for useful actions and penalties for harmful or unsuccessful actions in an environment.

How it is used
Uses reinforcement learning for goal-directed decisions learned through interaction and feedback.
How deep it goes
Explains the feedback loop and simulated problem setup, noting that the parking-lot environment simplifies real-world complexity.
Examples
Illustrates learning with dog training, a simulated parking lot, and strategy games.
What you can do after
Evaluate when reinforcement learning suits a problem with a known goal, available actions, and feedback.

Read how Grokking AI Algorithms, Second Edition explains reinforcement learning →

The AI Pocket Book

A learning approach in which an agent improves by trying actions and using feedback from the results.

How it is used
Frames machine learning as feedback-driven trial and error for discovering which actions produce useful results.
How deep it goes
Explains random actions, feedback, exploration, exploitation, and their tradeoff, but does not detail algorithms or implementation.
Examples
Uses randomized advertisements, click feedback, and later preference-based ad selection as its concrete example.
What you can do after
Evaluate when feedback-driven learning is useful and recognize its exploration, exploitation, and data-efficiency tradeoffs.

Read how The AI Pocket Book explains reinforcement learning →

Fight Fraud with Machine Learning

Reinforcement learning is machine learning in which an agent learns parameter or action choices from reward signals associated with outcomes.

How it is used
Uses human-ranked answers to further tune ChatGPT after supervised fine-tuning.
How deep it goes
Explains the reward-model and backpropagation loop, but defers algorithmic reinforcement-learning details.
Examples
Uses question-answer candidates ranked by labelers and compares training to a video game agent.
What you can do after
Evaluate/understand how human feedback can tune GPT parameters toward higher-scoring answers.

Read how Fight Fraud with Machine Learning explains reinforcement learning →

Build a Reasoning Model (From Scratch)

A training approach where a model learns from rewards or feedback to improve its behavior.

How it is used
Improves LLM reasoning through training-time scaling, typically after pretraining and sometimes instruction fine-tuning.
How deep it goes
Explains reward signals, response groups, policy updates, and RLHF-versus-RLVR distinctions, then points toward GRPO implementation.
Examples
Uses reasoning LLMs, task-correctness rewards, and classic agents trained for chess, Go, or video games as examples.
What you can do after
Apply RL to LLM reasoning by selecting reward signals, sampling response groups, and updating weights with group-based policy optimization.

Read how Build a Reasoning Model (From Scratch) explains reinforcement learning →

Introduction to Generative AI, Second Edition

Reinforcement learning is machine learning in which an agent learns behavior from rewards or penalties produced by interactions with an environment.

How it is used
Positions reinforcement learning as trial-and-error guidance for model behavior, including large language model fine-tuning.
How deep it goes
Defines rewards, penalties, and comparison paradigms at an intuitive level, but stops short of algorithmic or implementation detail.
Examples
Uses model outputs, numerical rewards and penalties, and human-ranked responses as teaching examples.
What you can do after
Evaluate/understand how feedback shapes model behavior and why human preferences can guide language-model responses.

Read how Introduction to Generative AI, Second Edition explains reinforcement learning →

Grokking AI Applications

Reinforcement learning is a machine learning approach in which an agent learns through trial and error while interacting with an environment to achieve goals.

How it is used
Positions reinforcement learning as trial-and-error training for achieving goals and refining conversational outputs.
How deep it goes
Defines trial and error and its role in ChatGPT refinement, but does not explain algorithms, rewards, or implementation.
Examples
Uses a child learning to walk and ChatGPT’s conversational refinement as examples.
What you can do after
Understand reinforcement learning’s role among ChatGPT’s three training stages.

Read how Grokking AI Applications explains reinforcement learning →

LLM Evaluation and Alignment, The Foundational Ideas

Reinforcement learning is a machine-learning paradigm in which an agent learns behavior by optimizing cumulative reward from interactions with an environment.

How it is used
Uses scalar evaluation scores as training signals for optimizing model behavior after supervised fine-tuning.
How deep it goes
Defines the policy, reward, trajectory, and update loop, then flags credit assignment without detailing optimizer mechanics.
Examples
Uses financial summarization, Atari, chess, simulated robots, and Pong to illustrate rewards, trajectories, and behavior optimization.
What you can do after
Understand how policies learn from scored responses and diagnose why reward design and credit assignment complicate language-model training.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains reinforcement learning →

Deep Learning with Python, Third Edition

A learning setup where an agent improves by interacting with an environment and receiving feedback, used here as a way to keep generating fresh training problems for LLMs.

How it is used
Uses interaction and reward signals to improve LLM behavior when manually written training responses become a bottleneck.
How deep it goes
Defines the RLHF workflow through reward modeling and pseudocode, but stops short of deeper reinforcement-learning details.
Examples
Uses ranked response preferences, scalar reward predictions, cutoff filtering, and self-contained environments that generate new problems.
What you can do after
Evaluate how RLHF converts human preferences into reward-guided LLM tuning and recognize its simplified training loop.

Read how Deep Learning with Python, Third Edition explains reinforcement learning →

Grokking Machine Learning

A machine-learning approach in which an agent learns to choose actions by interacting with an environment and receiving rewards or punishments.

How it is used
Uses agent–environment interaction to teach a machine-learning approach distinct from supervised and unsupervised learning.
How deep it goes
Defines agents, environments, goals, and rewards, then sketches learning from scores; it stops short of further detail.
Examples
Uses a grid-world robot that seeks treasure, avoids a dragon, cannot cross a mountain, and loses points for each step.
What you can do after
Understand how an agent can use accumulated scores to choose a path toward a goal and away from hazards.

Read how Grokking Machine Learning explains reinforcement learning →

Grokking Machine Learning, Second Edition

A machine-learning approach in which an agent learns to choose actions by interacting with an environment and receiving rewards or punishments.

How it is used
Uses interaction with an environment to teach an agent how to reach goals through rewards and punishments.
How deep it goes
Explains the agent, environment, goals, and feedback through a grid-world example, then defers deeper treatment.
Examples
Uses a robot grid world with a treasure chest, dragon, mountain, learned scores, and alternative paths.
What you can do after
Understand how an agent can learn a goal-directed path from environmental feedback.

Read how Grokking Machine Learning, Second Edition explains reinforcement learning →

Acing the CCNA Exam: Volume 2 Advanced Networking and Security

Reinforcement learning is a machine-learning approach in which an agent learns behavior through interactions with an environment and feedback about its actions.

How it is used
Classifies a machine-learning approach within the broader field of artificial intelligence.
How deep it goes
Defines learning through interaction and feedback but does not describe algorithms, implementation, or operational configuration.

Read how Acing the CCNA Exam: Volume 2 Advanced Networking and Security explains reinforcement learning →

Robotics for Programmers

Reinforcement learning is machine learning in which an agent learns behavior by interacting with an environment and receiving rewards.

How it is used
Deploys simulation-based learning to develop robot behavior at scale while limiting physical damage and development cost.
How deep it goes
Defines the agent–environment–reward–policy loop and simulation transfer, but defers learning algorithms and implementation details.
Examples
Uses simulated robots for large-scale policy training before controlling target scenarios on real robots.
What you can do after
Operate a robot with a learned policy that maps observations or states to actions in target scenarios.

Read how Robotics for Programmers explains reinforcement learning →

Causal AI

A machine-learning paradigm in which an agent learns actions or policies by interacting with an environment and receiving rewards or penalties.

How it is used
Uses sequential interaction and rewards to automate decisions whose consequences unfold over time.
How deep it goes
Develops policies, cumulative reward, Bellman formulations, counterfactual credit assignment, and causal DAG representations of MDPs and POMDPs.
Examples
Uses portfolio backtesting, saved-game policy simulation, regret comparisons, and Pavlovian reward associations as concrete illustrations.
What you can do after
Evaluate/understand when causal policy evaluation can outperform correlation-based reinforcement learning.

Read how Causal AI explains reinforcement learning →

Inside AI

Reinforcement learning is a machine-learning technique in which an agent learns behavior by interacting with an environment and receiving feedback for its actions.

How it is used
Applies trial-and-error learning where actions can receive clearly defined rewards.
How deep it goes
Defines trial-and-error learning and reinforcement signals, then illustrates self-play without detailing implementation.
Examples
Uses Arthur Samuel’s 1956 self-playing checkers program as its concrete example.
What you can do after
Evaluate whether an application has sufficiently clear rewards to justify reinforcement learning and identify the need for oversight.

Read how Inside AI explains reinforcement learning →

LLM Customization and Fine-Tuning

A machine-learning approach in which an agent learns a policy through reward signals from interactions or sampled outcomes.

How it is used
Uses reward-based training to elicit reasoning behaviors for enterprise tasks such as debugging, compliance checks, and root-cause analysis.
How deep it goes
Explains behavior emergence through GRPO and recommends narrow applications, while treating DeepSeek-style training as a case study rather than a default.
Examples
Uses login-failure root-cause analysis and GSM8K math reasoning to illustrate verification behavior and parameter-efficient training.
What you can do after
Evaluate whether to reserve GRPO for narrow, high-value reasoning tasks instead of routine classification, extraction, or generation.

Read how LLM Customization and Fine-Tuning explains reinforcement learning →

Retrieval Augmented Generation, The Foundational Ideas

Reinforcement learning is a machine-learning paradigm in which an agent learns a policy from reward signals produced by interactions with an environment.

How it is used
Trains retrieval or compression policies against final-answer or downstream-task rewards.
How deep it goes
Explains two-phase training, candidate grouping, normalized advantages, and critic-free optimization, but does not develop general RL theory.
Examples
Uses Search-R1, ReSearch, CORE, RECOMP, and GRPO as examples of answer-level search or task-oriented compression.
What you can do after
Apply it via policy optimization to align retrieval or compression with answer accuracy and task-specific evaluation.

Read how Retrieval Augmented Generation, The Foundational Ideas explains reinforcement learning →

Data Without Labels

A machine-learning paradigm in which an agent improves its behavior through sequential interactions, trial and error, and feedback from rewards or penalties.

How it is used
Frames reinforcement learning as an ML approach for dynamic decision-making and self-improving behavior.
How deep it goes
Defines trial and error and reward signals, but stops short of algorithms, environments, or implementation details.
Examples
Uses computer chess, self-driving cars, and modern video games to illustrate sequential learning and action correction.
What you can do after
Evaluate reinforcement learning as a fit for dynamic decision-making and explain its reward-driven feedback loop.

Read how Data Without Labels explains reinforcement learning →

Data-Driven Cybersecurity

Reinforcement learning is a machine-learning approach in which an agent learns decisions by interacting with an environment and optimizing cumulative feedback.

How it is used
Automates cybersecurity incident response by selecting actions from observed security states and their past outcomes.
How deep it goes
Explains the agent-environment-reward loop and demonstrates Q-learning with states, actions, rewards, and a learned Q-table.
Examples
Uses quarantine, traffic blocking, and escalation across normal, suspicious, and attack states, including a Python Q-table exercise.
What you can do after
Apply Q-learning to model adaptive incident-response choices and interpret learned action values for security states.

Read how Data-Driven Cybersecurity explains reinforcement learning →

Timeless Algorithms, The Foundational Ideas

Reinforcement learning is a machine-learning approach in which an agent learns behavior through interaction with an environment and feedback about its actions.

How it is used
Models agents learning decisions through environmental interaction when transition probabilities or rewards are unknown.
How deep it goes
Traces the shift from known models to experience-based value estimates, but gives no algorithmic implementation.
Examples
Uses robot navigation and state-action-outcome trajectories to illustrate sequential learning.
What you can do after
Understand how agents improve decisions from experience while retaining Bellman’s immediate-reward and future-value logic.

Read how Timeless Algorithms, The Foundational Ideas explains reinforcement learning →

Also covered in

What to know first

Related concepts

Questions about reinforcement learning

What is reinforcement learning?

Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.

Which Manning books explain reinforcement learning?

Reinforcement Learning from Human Feedback, Applied Reinforcement Learning, Optimization Algorithms: AI techniques for design, planning, and control problems, Build a DeepSeek Model (From Scratch), Evolutionary Deep Learning: Genetic algorithms and neural networks, Sutskever's List, Deep Learning with R, Third Edition, Build AI Drug Discovery Pipelines and 20 more.

What should I know before learning reinforcement learning?

Manning books that teach reinforcement learning usually introduce language model, environment, machine learning and AI agent first.

What is the difference between reinforcement learning and fine-tuning?

Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards. Fine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.

What is reinforcement learning a kind of?

Reinforcement learning is a kind of machine learning. Machine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.

What is reinforcement learning also called?

Reinforcement learning is also called RL.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.