Reinforcement learning
Reinforcement learning is a machine-learning paradigm in which an agent learns actions or a policy through interaction with an environment and evaluative feedback, such as rewards.
Reinforcement Learning from Human Feedback
Reinforcement learning is a machine learning paradigm in which an agent learns behavior by optimizing rewards obtained through interaction with an environment.
- How it is used
- Optimizes language-model completions against preference-model, verifiable, or environment-derived rewards during post-training.
- How deep it goes
- Defines policies, trajectories, dynamics, discounted returns, and KL-regularized objectives, then connects them to policy-gradient training trade-offs.
- Examples
- Uses a 70°F thermostat, prompt-completion reward scoring, tool-use trajectories, arithmetic card games, and visual navigation tasks.
- What you can do after
- Evaluate/understand RL objectives, policy updates, reward signals, KL constraints, and their post-training trade-offs.
Read how Reinforcement Learning from Human Feedback explains reinforcement learning →
Optimization Algorithms: AI techniques for design, planning, and control problems
Reinforcement learning is a machine-learning paradigm in which an agent learns action policies from interactions and evaluative feedback.
- How it is used
- Frames sequential decision-making as policy optimization for cumulative reward, including applications to search and optimization problems.
- How deep it goes
- Defines agent feedback and decision settings, then distinguishes model-based, model-free, on-policy, and off-policy methods and explains PPO clipping.
- Examples
- Contrasts repeated arm choices with actions that change future states, and identifies contextual bandits as choices informed by side information.
- What you can do after
- Evaluate/understand it by deciding whether an optimization decision needs a bandit formulation or an MDP with future-state effects.
Evolutionary Deep Learning: Genetic algorithms and neural networks
Reinforcement learning is a machine-learning paradigm in which an agent learns behavior by interacting with an environment and receiving rewards or other evaluative feedback.
- How it is used
- Frames reward-driven agent learning and supplies Gym environments for testing decision-making agents.
- How deep it goes
- Explains the state–action–reward loop and implements tabular Q-learning, while deferring advanced deep Q-learning treatment.
- Examples
- Uses pet training, a 4-by-4 FrozenLake agent, Gym environment exercises, and reward-scored NEAT agents.
- What you can do after
- Build it from scratch as a tabular Q-learning agent and evaluate it in a Gym environment.
Sutskever's List
Machine learning in which an agent learns behavior through rewards or penalties from interaction with an environment.
- How it is used
- Positions reinforcement learning as an early OpenAI research direction pursued through a research platform and game-playing milestones.
- How deep it goes
- Mentions the approach and its applications but does not explain training algorithms or reward design.
- Examples
- Cites OpenAI Gym and AI-driven Dota 2 gameplay as early milestones.
- What you can do after
- Evaluate/understand it as an early research direction at OpenAI rather than implement a reinforcement learning algorithm.
Deep Learning with R, Third Edition
Reinforcement learning is a machine-learning technique in which an agent learns behavior through interaction with an environment and feedback such as rewards.
- How it is used
- Uses reinforcement learning to improve language-model responses through human preferences or automatically checked answers.
- How deep it goes
- Works through a simplified generate-score-train loop and a verifiable-answer procedure, while leaving practical gradient updates unspecified.
- Examples
- Uses reward-filtering pseudocode, a labeled math answer, and executable code as examples of feedback.
- What you can do after
- Evaluate/understand it by tracing how generated responses receive preference scores or answer checks and feed into further training.
Read how Deep Learning with R, Third Edition explains reinforcement learning →
Build a Robo-Advisor with Python (From Scratch)
Reinforcement learning is a machine-learning approach in which an agent learns a decision policy through interactions with an environment and feedback from rewards or penalties.
- How it is used
- Applies reinforcement learning to multiperiod financial planning decisions involving investing, spending, retirement, and withdrawals.
- How deep it goes
- Works through state grids, simulated transitions, epsilon exploration, episodes, and tabular Q-learning updates; defers broader algorithm variants.
- Examples
- Uses a wealth-and-time grid, stock/bond allocation actions, simulated wealth transitions, and repeated Q-learning episodes.
- What you can do after
- Build it from scratch as a tabular Q-learning policy for simulated multiperiod wealth and allocation decisions.
Read how Build a Robo-Advisor with Python (From Scratch) explains reinforcement learning →
Grokking AI Algorithms, Second Edition
Reinforcement learning is a machine-learning paradigm in which an agent learns behavior through rewards for useful actions and penalties for harmful or unsuccessful actions in an environment.
- How it is used
- Guides an agent toward a goal when available actions are known but the best sequence must be learned through feedback.
- How deep it goes
- Explains rewards, cumulative outcomes, and a simulated training life cycle, without detailing an implementation in the supplied evidence.
- Examples
- Uses dog training, simulated parking-lot navigation, and strategic games to illustrate feedback and decisions over time.
- What you can do after
- Evaluate/understand it by identifying an agent, its available actions, a goal, and feedback for cumulative outcomes.
Read how Grokking AI Algorithms, Second Edition explains reinforcement learning →
The AI Pocket Book
A learning approach in which an agent improves by trying actions and using feedback from the results.
- How it is used
- Frames machine learning as feedback-driven trial and error for discovering which actions produce useful results.
- How deep it goes
- Explains random actions, feedback, exploration, exploitation, and their tradeoff, but does not detail algorithms or implementation.
- Examples
- Uses randomized advertisements, click feedback, and later preference-based ad selection as its concrete example.
- What you can do after
- Evaluate when feedback-driven learning is useful and recognize its exploration, exploitation, and data-efficiency tradeoffs.
Read how The AI Pocket Book explains reinforcement learning →
Build a Reasoning Model (From Scratch)
A training approach where a model learns from rewards or feedback to improve its behavior.
- How it is used
- Improves LLM reasoning through training-time scaling, typically after pretraining and sometimes instruction fine-tuning.
- How deep it goes
- Explains reward signals, response groups, policy updates, and RLHF-versus-RLVR distinctions, then points toward GRPO implementation.
- Examples
- Uses reasoning LLMs, task-correctness rewards, and classic agents trained for chess, Go, or video games as examples.
- What you can do after
- Apply RL to LLM reasoning by selecting reward signals, sampling response groups, and updating weights with group-based policy optimization.
Read how Build a Reasoning Model (From Scratch) explains reinforcement learning →
Introduction to Generative AI, Second Edition
Reinforcement learning is machine learning in which an agent learns behavior from rewards or penalties produced by interactions with an environment.
- How it is used
- Positions reinforcement learning as trial-and-error guidance for model behavior, including large language model fine-tuning.
- How deep it goes
- Defines rewards, penalties, and comparison paradigms at an intuitive level, but stops short of algorithmic or implementation detail.
- Examples
- Uses model outputs, numerical rewards and penalties, and human-ranked responses as teaching examples.
- What you can do after
- Evaluate/understand how feedback shapes model behavior and why human preferences can guide language-model responses.
Read how Introduction to Generative AI, Second Edition explains reinforcement learning →
Grokking AI Applications
Reinforcement learning is a machine learning approach in which an agent learns through trial and error while interacting with an environment to achieve goals.
- How it is used
- Positions reinforcement learning as trial-and-error training for achieving goals and refining conversational outputs.
- How deep it goes
- Defines trial and error and its role in ChatGPT refinement, but does not explain algorithms, rewards, or implementation.
- Examples
- Uses a child learning to walk and ChatGPT’s conversational refinement as examples.
- What you can do after
- Understand reinforcement learning’s role among ChatGPT’s three training stages.
Read how Grokking AI Applications explains reinforcement learning →
Deep Learning with Python, Third Edition
A learning setup where an agent improves by interacting with an environment and receiving feedback, used here as a way to keep generating fresh training problems for LLMs.
- How it is used
- Uses interaction and reward signals to improve LLM behavior when manually written training responses become a bottleneck.
- How deep it goes
- Defines the RLHF workflow through reward modeling and pseudocode, but stops short of deeper reinforcement-learning details.
- Examples
- Uses ranked response preferences, scalar reward predictions, cutoff filtering, and self-contained environments that generate new problems.
- What you can do after
- Evaluate how RLHF converts human preferences into reward-guided LLM tuning and recognize its simplified training loop.
Read how Deep Learning with Python, Third Edition explains reinforcement learning →
Grokking Machine Learning
A machine-learning approach in which an agent learns to choose actions by interacting with an environment and receiving rewards or punishments.
- How it is used
- Uses agent–environment interaction to teach a machine-learning approach distinct from supervised and unsupervised learning.
- How deep it goes
- Defines agents, environments, goals, and rewards, then sketches learning from scores; it stops short of further detail.
- Examples
- Uses a grid-world robot that seeks treasure, avoids a dragon, cannot cross a mountain, and loses points for each step.
- What you can do after
- Understand how an agent can use accumulated scores to choose a path toward a goal and away from hazards.
Read how Grokking Machine Learning explains reinforcement learning →
Acing the CCNA Exam: Volume 2 Advanced Networking and Security
Reinforcement learning is a machine-learning approach in which an agent learns behavior through interactions with an environment and feedback about its actions.
- How it is used
- Classifies a machine-learning approach within the broader field of artificial intelligence.
- How deep it goes
- Defines learning through interaction and feedback but does not describe algorithms, implementation, or operational configuration.
Causal AI
A machine-learning paradigm in which an agent learns actions or policies by interacting with an environment and receiving rewards or penalties.
- How it is used
- Uses sequential interaction and rewards to automate decisions whose consequences unfold over time.
- How deep it goes
- Develops policies, cumulative reward, Bellman formulations, counterfactual credit assignment, and causal DAG representations of MDPs and POMDPs.
- Examples
- Uses portfolio backtesting, saved-game policy simulation, regret comparisons, and Pavlovian reward associations as concrete illustrations.
- What you can do after
- Evaluate/understand when causal policy evaluation can outperform correlation-based reinforcement learning.
Inside AI
Reinforcement learning is a machine-learning technique in which an agent learns behavior by interacting with an environment and receiving feedback for its actions.
- How it is used
- Applies trial-and-error learning where actions can receive clearly defined rewards.
- How deep it goes
- Defines trial-and-error learning and reinforcement signals, then illustrates self-play without detailing implementation.
- Examples
- Uses Arthur Samuel’s 1956 self-playing checkers program as its concrete example.
- What you can do after
- Evaluate whether an application has sufficiently clear rewards to justify reinforcement learning and identify the need for oversight.
Data Without Labels
A machine-learning paradigm in which an agent improves its behavior through sequential interactions, trial and error, and feedback from rewards or penalties.
- How it is used
- Frames reinforcement learning as an ML approach for dynamic decision-making and self-improving behavior.
- How deep it goes
- Defines trial and error and reward signals, but stops short of algorithms, environments, or implementation details.
- Examples
- Uses computer chess, self-driving cars, and modern video games to illustrate sequential learning and action correction.
- What you can do after
- Evaluate reinforcement learning as a fit for dynamic decision-making and explain its reward-driven feedback loop.
Read how Data Without Labels explains reinforcement learning →
Data-Driven Cybersecurity
Reinforcement learning is a machine-learning approach in which an agent learns decisions by interacting with an environment and optimizing cumulative feedback.
- How it is used
- Automates cybersecurity incident response by selecting actions from observed security states and their past outcomes.
- How deep it goes
- Explains the agent-environment-reward loop and demonstrates Q-learning with states, actions, rewards, and a learned Q-table.
- Examples
- Uses quarantine, traffic blocking, and escalation across normal, suspicious, and attack states, including a Python Q-table exercise.
- What you can do after
- Apply Q-learning to model adaptive incident-response choices and interpret learned action values for security states.
Read how Data-Driven Cybersecurity explains reinforcement learning →
Related concepts
- reward signaluses · A reward signal is an evaluative feedback value representing the desirability or payoff of an action, outcome, or state.
- machine learningkind of · Machine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
- AI agentuses · An AI agent is a computational system that uses artificial intelligence to interpret information, select actions, and pursue goals through interaction with an environment.
- environmentuses · An environment is the external context, including its entities and conditions, in which a system, process, or agent exists or operates.
- reinforcement learning with human feedbackused by · Reinforcement learning with human feedback is a model-training approach that uses human evaluations or preferences to guide optimization of model behavior.
- fine-tuningcontrasts with · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.