Reinforcement learning
Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.
Reinforcement Learning from Human Feedback
Reinforcement learning is a machine learning paradigm in which an agent learns behavior by optimizing rewards obtained through interaction with an environment.
- How it is used
- Uses reinforcement learning to optimize language-model behavior from preference, verifiable, or environment feedback.
- How deep it goes
- Develops the trajectory and discounted-return formulation, then explains policy-gradient trade-offs and practical RLHF training loops.
- Examples
- Illustrates reinforcement learning with a thermostat, language-model completions scored by a reward model, and arithmetic and navigation generalization tests.
- What you can do after
- Evaluate reinforcement learning objectives and post-training trade-offs, including reward design, infrastructure cost, stability, and generalization.
Read how Reinforcement Learning from Human Feedback explains reinforcement learning →
Applied Reinforcement Learning
Reinforcement learning is a machine-learning paradigm in which an agent learns to choose actions through interactions with an environment and feedback about their consequences.
- How it is used
- Frames reinforcement learning as a way to improve sequential business decisions under uncertainty and changing conditions.
- How deep it goes
- Explains interaction and policy learning, then extends the treatment to environment design and system-level reward and evaluation choices.
- Examples
- Uses dynamic pricing, portfolio management, and language generation to illustrate agents, feedback, and design choices.
- What you can do after
- Evaluate/understand when reinforcement learning fits a decision problem and what simulation, reward, and evaluation choices matter.
Read how Applied Reinforcement Learning explains reinforcement learning →
Optimization Algorithms: AI techniques for design, planning, and control problems
Reinforcement learning is a machine-learning paradigm in which an agent learns action policies from interactions and evaluative feedback.
- How it is used
- Applies reinforcement learning to search and optimization by learning behavior through environmental interaction.
- How deep it goes
- Defines reward-driven learning, distinguishes problem and algorithm classes, and explains PPO clipping at a conceptual level.
- Examples
- Uses contextual bandit choices to illustrate how the best action can change with side information.
- What you can do after
- Evaluate and understand whether a decision problem calls for an MDP or a bandit formulation.
Build a DeepSeek Model (From Scratch)
Reinforcement learning is a machine-learning paradigm in which an agent improves its behavior by optimizing rewards received from interactions or evaluations.
- How it is used
- Uses reinforcement learning to post-train a pretrained language model toward useful reasoning behavior.
- How deep it goes
- Moves from the interaction framework through policy gradients, PPO and GRPO to PyTorch training excerpts, while excluding massive-scale distributed infrastructure.
- Examples
- Uses a tennis-ball arithmetic rollout, the R1-Zero and R1 pipelines, and a Qwen2.5-32B comparison with distillation.
- What you can do after
- Build a PyTorch GRPO training path using verifier rewards and training steps.
Read how Build a DeepSeek Model (From Scratch) explains reinforcement learning →
Evolutionary Deep Learning: Genetic algorithms and neural networks
Reinforcement learning is a machine-learning paradigm in which an agent learns behavior by interacting with an environment and receiving rewards or other evaluative feedback.
- How it is used
- Frames reward-driven agent learning and supplies Gym environments for testing decision-making agents.
- How deep it goes
- Explains the state–action–reward loop and implements tabular Q-learning, while deferring advanced deep Q-learning treatment.
- Examples
- Uses pet training, a 4-by-4 FrozenLake agent, Gym environment exercises, and reward-scored NEAT agents.
- What you can do after
- Build it from scratch as a tabular Q-learning agent and evaluate it in a Gym environment.
Sutskever's List
Machine learning in which an agent learns behavior through rewards or penalties from interaction with an environment.
- How it is used
- Positions reinforcement learning as an early OpenAI research direction pursued through a research platform and game-playing milestones.
- How deep it goes
- Mentions the approach and its applications but does not explain training algorithms or reward design.
- Examples
- Cites OpenAI Gym and AI-driven Dota 2 gameplay as early milestones.
- What you can do after
- Evaluate/understand it as an early research direction at OpenAI rather than implement a reinforcement learning algorithm.
Deep Learning with R, Third Edition
Reinforcement learning is a machine-learning technique in which an agent learns behavior through interaction with an environment and feedback such as rewards.
- How it is used
- Uses reinforcement learning to improve language-model responses through preference feedback or verifiable answers.
- How deep it goes
- Explains simplified training loops and reward modeling, while noting that practical gradient updates are more complex.
- Examples
- Illustrates preference-ranked chatbot responses, checked math solutions, and generated code tested by execution.
- What you can do after
- Evaluate and understand simplified reinforcement-learning loops for language-model training and their practical limitations.
Read how Deep Learning with R, Third Edition explains reinforcement learning →
Build AI Drug Discovery Pipelines
Reinforcement learning is a machine-learning approach in which an agent learns sequential actions through feedback from rewards or penalties.
- How it is used
- Uses sequential reward-driven exploration to steer generative models toward molecules satisfying drug-discovery objectives.
- How deep it goes
- Explains agents, actions, rewards, latent-space exploration, and policy updates, while deferring formal RL development to an appendix.
- Examples
- Demonstrates latent molecular optimization through MOLRL, VAE-CYC, GENTRL, and the TNIK-focused rentosertib program.
- What you can do after
- Apply reinforcement learning to configure rewards and optimize generated molecules against multiple predicted properties.
Read how Build AI Drug Discovery Pipelines explains reinforcement learning →
Build a Robo-Advisor with Python (From Scratch)
Reinforcement learning is a machine-learning approach in which an agent learns a decision policy through interactions with an environment and feedback from rewards or penalties.
- How it is used
- Applies reinforcement learning to multiperiod financial planning decisions involving investing, spending, retirement, and withdrawals.
- How deep it goes
- Works through state grids, simulated transitions, epsilon exploration, episodes, and tabular Q-learning updates; defers broader algorithm variants.
- Examples
- Uses a wealth-and-time grid, stock/bond allocation actions, simulated wealth transitions, and repeated Q-learning episodes.
- What you can do after
- Build it from scratch as a tabular Q-learning policy for simulated multiperiod wealth and allocation decisions.
Read how Build a Robo-Advisor with Python (From Scratch) explains reinforcement learning →
Grokking AI Algorithms, Second Edition
Reinforcement learning is a machine-learning paradigm in which an agent learns behavior through rewards for useful actions and penalties for harmful or unsuccessful actions in an environment.
- How it is used
- Uses reinforcement learning for goal-directed decisions learned through interaction and feedback.
- How deep it goes
- Explains the feedback loop and simulated problem setup, noting that the parking-lot environment simplifies real-world complexity.
- Examples
- Illustrates learning with dog training, a simulated parking lot, and strategy games.
- What you can do after
- Evaluate when reinforcement learning suits a problem with a known goal, available actions, and feedback.
Read how Grokking AI Algorithms, Second Edition explains reinforcement learning →
The AI Pocket Book
A learning approach in which an agent improves by trying actions and using feedback from the results.
- How it is used
- Frames machine learning as feedback-driven trial and error for discovering which actions produce useful results.
- How deep it goes
- Explains random actions, feedback, exploration, exploitation, and their tradeoff, but does not detail algorithms or implementation.
- Examples
- Uses randomized advertisements, click feedback, and later preference-based ad selection as its concrete example.
- What you can do after
- Evaluate when feedback-driven learning is useful and recognize its exploration, exploitation, and data-efficiency tradeoffs.
Read how The AI Pocket Book explains reinforcement learning →
Fight Fraud with Machine Learning
Reinforcement learning is machine learning in which an agent learns parameter or action choices from reward signals associated with outcomes.
- How it is used
- Uses human-ranked answers to further tune ChatGPT after supervised fine-tuning.
- How deep it goes
- Explains the reward-model and backpropagation loop, but defers algorithmic reinforcement-learning details.
- Examples
- Uses question-answer candidates ranked by labelers and compares training to a video game agent.
- What you can do after
- Evaluate/understand how human feedback can tune GPT parameters toward higher-scoring answers.
Read how Fight Fraud with Machine Learning explains reinforcement learning →
Build a Reasoning Model (From Scratch)
A training approach where a model learns from rewards or feedback to improve its behavior.
- How it is used
- Improves LLM reasoning through training-time scaling, typically after pretraining and sometimes instruction fine-tuning.
- How deep it goes
- Explains reward signals, response groups, policy updates, and RLHF-versus-RLVR distinctions, then points toward GRPO implementation.
- Examples
- Uses reasoning LLMs, task-correctness rewards, and classic agents trained for chess, Go, or video games as examples.
- What you can do after
- Apply RL to LLM reasoning by selecting reward signals, sampling response groups, and updating weights with group-based policy optimization.
Read how Build a Reasoning Model (From Scratch) explains reinforcement learning →
Introduction to Generative AI, Second Edition
Reinforcement learning is machine learning in which an agent learns behavior from rewards or penalties produced by interactions with an environment.
- How it is used
- Positions reinforcement learning as trial-and-error guidance for model behavior, including large language model fine-tuning.
- How deep it goes
- Defines rewards, penalties, and comparison paradigms at an intuitive level, but stops short of algorithmic or implementation detail.
- Examples
- Uses model outputs, numerical rewards and penalties, and human-ranked responses as teaching examples.
- What you can do after
- Evaluate/understand how feedback shapes model behavior and why human preferences can guide language-model responses.
Read how Introduction to Generative AI, Second Edition explains reinforcement learning →
Grokking AI Applications
Reinforcement learning is a machine learning approach in which an agent learns through trial and error while interacting with an environment to achieve goals.
- How it is used
- Positions reinforcement learning as trial-and-error training for achieving goals and refining conversational outputs.
- How deep it goes
- Defines trial and error and its role in ChatGPT refinement, but does not explain algorithms, rewards, or implementation.
- Examples
- Uses a child learning to walk and ChatGPT’s conversational refinement as examples.
- What you can do after
- Understand reinforcement learning’s role among ChatGPT’s three training stages.
Read how Grokking AI Applications explains reinforcement learning →
LLM Evaluation and Alignment, The Foundational Ideas
Reinforcement learning is a machine-learning paradigm in which an agent learns behavior by optimizing cumulative reward from interactions with an environment.
- How it is used
- Uses scalar evaluation scores as training signals for optimizing model behavior after supervised fine-tuning.
- How deep it goes
- Defines the policy, reward, trajectory, and update loop, then flags credit assignment without detailing optimizer mechanics.
- Examples
- Uses financial summarization, Atari, chess, simulated robots, and Pong to illustrate rewards, trajectories, and behavior optimization.
- What you can do after
- Understand how policies learn from scored responses and diagnose why reward design and credit assignment complicate language-model training.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains reinforcement learning →
Deep Learning with Python, Third Edition
A learning setup where an agent improves by interacting with an environment and receiving feedback, used here as a way to keep generating fresh training problems for LLMs.
- How it is used
- Uses interaction and reward signals to improve LLM behavior when manually written training responses become a bottleneck.
- How deep it goes
- Defines the RLHF workflow through reward modeling and pseudocode, but stops short of deeper reinforcement-learning details.
- Examples
- Uses ranked response preferences, scalar reward predictions, cutoff filtering, and self-contained environments that generate new problems.
- What you can do after
- Evaluate how RLHF converts human preferences into reward-guided LLM tuning and recognize its simplified training loop.
Read how Deep Learning with Python, Third Edition explains reinforcement learning →
Grokking Machine Learning
A machine-learning approach in which an agent learns to choose actions by interacting with an environment and receiving rewards or punishments.
- How it is used
- Uses agent–environment interaction to teach a machine-learning approach distinct from supervised and unsupervised learning.
- How deep it goes
- Defines agents, environments, goals, and rewards, then sketches learning from scores; it stops short of further detail.
- Examples
- Uses a grid-world robot that seeks treasure, avoids a dragon, cannot cross a mountain, and loses points for each step.
- What you can do after
- Understand how an agent can use accumulated scores to choose a path toward a goal and away from hazards.
Read how Grokking Machine Learning explains reinforcement learning →
Grokking Machine Learning, Second Edition
A machine-learning approach in which an agent learns to choose actions by interacting with an environment and receiving rewards or punishments.
- How it is used
- Uses interaction with an environment to teach an agent how to reach goals through rewards and punishments.
- How deep it goes
- Explains the agent, environment, goals, and feedback through a grid-world example, then defers deeper treatment.
- Examples
- Uses a robot grid world with a treasure chest, dragon, mountain, learned scores, and alternative paths.
- What you can do after
- Understand how an agent can learn a goal-directed path from environmental feedback.
Read how Grokking Machine Learning, Second Edition explains reinforcement learning →
Acing the CCNA Exam: Volume 2 Advanced Networking and Security
Reinforcement learning is a machine-learning approach in which an agent learns behavior through interactions with an environment and feedback about its actions.
- How it is used
- Classifies a machine-learning approach within the broader field of artificial intelligence.
- How deep it goes
- Defines learning through interaction and feedback but does not describe algorithms, implementation, or operational configuration.
Robotics for Programmers
Reinforcement learning is machine learning in which an agent learns behavior by interacting with an environment and receiving rewards.
- How it is used
- Deploys simulation-based learning to develop robot behavior at scale while limiting physical damage and development cost.
- How deep it goes
- Defines the agent–environment–reward–policy loop and simulation transfer, but defers learning algorithms and implementation details.
- Examples
- Uses simulated robots for large-scale policy training before controlling target scenarios on real robots.
- What you can do after
- Operate a robot with a learned policy that maps observations or states to actions in target scenarios.
Read how Robotics for Programmers explains reinforcement learning →
Causal AI
A machine-learning paradigm in which an agent learns actions or policies by interacting with an environment and receiving rewards or penalties.
- How it is used
- Uses sequential interaction and rewards to automate decisions whose consequences unfold over time.
- How deep it goes
- Develops policies, cumulative reward, Bellman formulations, counterfactual credit assignment, and causal DAG representations of MDPs and POMDPs.
- Examples
- Uses portfolio backtesting, saved-game policy simulation, regret comparisons, and Pavlovian reward associations as concrete illustrations.
- What you can do after
- Evaluate/understand when causal policy evaluation can outperform correlation-based reinforcement learning.
Inside AI
Reinforcement learning is a machine-learning technique in which an agent learns behavior by interacting with an environment and receiving feedback for its actions.
- How it is used
- Applies trial-and-error learning where actions can receive clearly defined rewards.
- How deep it goes
- Defines trial-and-error learning and reinforcement signals, then illustrates self-play without detailing implementation.
- Examples
- Uses Arthur Samuel’s 1956 self-playing checkers program as its concrete example.
- What you can do after
- Evaluate whether an application has sufficiently clear rewards to justify reinforcement learning and identify the need for oversight.
LLM Customization and Fine-Tuning
A machine-learning approach in which an agent learns a policy through reward signals from interactions or sampled outcomes.
- How it is used
- Uses reward-based training to elicit reasoning behaviors for enterprise tasks such as debugging, compliance checks, and root-cause analysis.
- How deep it goes
- Explains behavior emergence through GRPO and recommends narrow applications, while treating DeepSeek-style training as a case study rather than a default.
- Examples
- Uses login-failure root-cause analysis and GSM8K math reasoning to illustrate verification behavior and parameter-efficient training.
- What you can do after
- Evaluate whether to reserve GRPO for narrow, high-value reasoning tasks instead of routine classification, extraction, or generation.
Read how LLM Customization and Fine-Tuning explains reinforcement learning →
Retrieval Augmented Generation, The Foundational Ideas
Reinforcement learning is a machine-learning paradigm in which an agent learns a policy from reward signals produced by interactions with an environment.
- How it is used
- Trains retrieval or compression policies against final-answer or downstream-task rewards.
- How deep it goes
- Explains two-phase training, candidate grouping, normalized advantages, and critic-free optimization, but does not develop general RL theory.
- Examples
- Uses Search-R1, ReSearch, CORE, RECOMP, and GRPO as examples of answer-level search or task-oriented compression.
- What you can do after
- Apply it via policy optimization to align retrieval or compression with answer accuracy and task-specific evaluation.
Read how Retrieval Augmented Generation, The Foundational Ideas explains reinforcement learning →
Data Without Labels
A machine-learning paradigm in which an agent improves its behavior through sequential interactions, trial and error, and feedback from rewards or penalties.
- How it is used
- Frames reinforcement learning as an ML approach for dynamic decision-making and self-improving behavior.
- How deep it goes
- Defines trial and error and reward signals, but stops short of algorithms, environments, or implementation details.
- Examples
- Uses computer chess, self-driving cars, and modern video games to illustrate sequential learning and action correction.
- What you can do after
- Evaluate reinforcement learning as a fit for dynamic decision-making and explain its reward-driven feedback loop.
Read how Data Without Labels explains reinforcement learning →
Data-Driven Cybersecurity
Reinforcement learning is a machine-learning approach in which an agent learns decisions by interacting with an environment and optimizing cumulative feedback.
- How it is used
- Automates cybersecurity incident response by selecting actions from observed security states and their past outcomes.
- How deep it goes
- Explains the agent-environment-reward loop and demonstrates Q-learning with states, actions, rewards, and a learned Q-table.
- Examples
- Uses quarantine, traffic blocking, and escalation across normal, suspicious, and attack states, including a Python Q-table exercise.
- What you can do after
- Apply Q-learning to model adaptive incident-response choices and interpret learned action values for security states.
Read how Data-Driven Cybersecurity explains reinforcement learning →
Timeless Algorithms, The Foundational Ideas
Reinforcement learning is a machine-learning approach in which an agent learns behavior through interaction with an environment and feedback about its actions.
- How it is used
- Models agents learning decisions through environmental interaction when transition probabilities or rewards are unknown.
- How deep it goes
- Traces the shift from known models to experience-based value estimates, but gives no algorithmic implementation.
- Examples
- Uses robot navigation and state-action-outcome trajectories to illustrate sequential learning.
- What you can do after
- Understand how agents improve decisions from experience while retaining Bellman’s immediate-reward and future-value logic.
Read how Timeless Algorithms, The Foundational Ideas explains reinforcement learning →
Also covered in
- How Large Language Models Work
- Causal Inference for Data Science
- Advanced Algorithms and Data Structures
- Machines that ThinkIn early access (MEAP)
- LLMs in Production
- Generative AI in Action
- The Complete Obsolete Guide to Generative AI
- Transformers in Action
- Deep Learning with JAX
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- environmentAn environment is the external context, system, or conditions within which an agent or application operates and interacts.
- machine learningMachine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
- AI agentAn AI agent is a computational system that uses artificial intelligence to interpret information and select or perform actions toward goals while interacting with an environment.
Related concepts
- reward signaluses · A reward signal is a value or feedback indicating the desirability, benefit, or cost associated with an action, outcome, or behavior.
- machine learningkind of · Machine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
- AI agentuses · An AI agent is a computational system that uses artificial intelligence to interpret information and select or perform actions toward goals while interacting with an environment.
- environmentuses · An environment is the external context, system, or conditions within which an agent or application operates and interacts.
- RLHFused by · Reinforcement learning from human feedback is a model-training approach that uses human evaluations or preferences as feedback to optimize model behavior.
- fine-tuningcontrasts with · Fine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
Questions about reinforcement learning
What is reinforcement learning?
Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.
Which Manning books explain reinforcement learning?
Reinforcement Learning from Human Feedback, Applied Reinforcement Learning, Optimization Algorithms: AI techniques for design, planning, and control problems, Build a DeepSeek Model (From Scratch), Evolutionary Deep Learning: Genetic algorithms and neural networks, Sutskever's List, Deep Learning with R, Third Edition, Build AI Drug Discovery Pipelines and 20 more.
What should I know before learning reinforcement learning?
Manning books that teach reinforcement learning usually introduce language model, environment, machine learning and AI agent first.
What is the difference between reinforcement learning and fine-tuning?
Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards. Fine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
What is reinforcement learning a kind of?
Reinforcement learning is a kind of machine learning. Machine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
What is reinforcement learning also called?
Reinforcement learning is also called RL.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



























