Reward signal
A reward signal is an evaluative feedback value representing the desirability or payoff of an action, outcome, or state.
Optimization Algorithms: AI techniques for design, planning, and control problems
A numerical feedback signal indicating the immediate value of an agent's action in a state.
- How it is used
- Provides action feedback for learning policies that maximize cumulative reward.
- How deep it goes
- Works through sampled payoffs, per-step training feedback, and a noisy reward function, without deriving a general reward-design procedure.
- Examples
- Uses CartPole’s +1-per-step feedback, sampled slot-machine payoffs, and noisy truck-selection scores.
- What you can do after
- Build it from scratch in a Python CartPole training loop or apply it via Vowpal Wabbit to truck selection.
Grokking AI Algorithms, Second Edition
A score is a numerical evaluation assigned to an output or state to indicate its favorability according to an evaluator.
- How it is used
- Guides an agent’s trial-and-error decisions by providing feedback after actions in an environment.
- How deep it goes
- Works through a simulated reward schedule and a reward-design failure, then gives simple training measures rather than a general scoring formula.
- Examples
- Uses dog training, parking-lot navigation, game-state scoring, and an overweight knapsack to make favorable and unfavorable feedback concrete.
- What you can do after
- Build it from scratch in a simple simulator, then evaluate whether its rewards encourage the intended behavior.
Read how Grokking AI Algorithms, Second Edition explains reward signal →
Build a Robo-Advisor with Python (From Scratch)
A reward is a numerical feedback signal representing the benefit or cost an agent receives after taking an action.
- How it is used
- Defines the feedback objective that guides investor allocation and spending decisions in reinforcement learning.
- How deep it goes
- Works through binary terminal rewards, discounted consumption utility, dynamic programming, and Q-learning updates.
- Examples
- Uses a $2 million wealth goal, a $1 million initial state, stock-bond allocations, and consumption utility.
- What you can do after
- Apply it via dynamic programming or Q-learning to encode goal success or discounted consumption utility.
Read how Build a Robo-Advisor with Python (From Scratch) explains reward signal →
Reinforcement Learning from Human Feedback
A reward is a scalar evaluative signal indicating the desirability of an action or outcome for an agent.
- How it is used
- Provides the scalar objective that RL policies maximize and RLHF uses to improve generated completions.
- How deep it goes
- Defines discounted return and develops pairwise scalar reward-model scoring, while deferring training-pipeline specifics to library documentation.
- Examples
- Uses a target-temperature thermostat, CartPole balancing, and chosen-versus-rejected text completions scored by a Bradley–Terry reward model.
- What you can do after
- Evaluate/understand how scalar rewards define RL objectives and how preference models supply optimization signals for RLHF.
Read how Reinforcement Learning from Human Feedback explains reward signal →
Causal AI
A reward is a scalar feedback signal that evaluates the desirability of an agent's outcome or action in reinforcement learning.
- How it is used
- Quantifies desirable outcomes so agents can compare actions and policies under uncertainty.
- How deep it goes
- Defines utility and negative loss, then uses reward distributions in decision rules, bandit feedback, and a grid-world decision process.
- Examples
- Uses investment returns, medication decisions, a slot-machine analogy, and positive and negative grid-world rewards.
- What you can do after
- Evaluate/understand reward-based choices by comparing utility under candidate actions while accounting for uncertain outcomes.
Evolutionary Deep Learning: Genetic algorithms and neural networks
A reward is a numerical feedback signal returned by a reinforcement-learning environment after an agent takes an action.
- How it is used
- Drives action selection and learning by feeding environment outcomes back to reinforcement-learning agents.
- How deep it goes
- Explains reward through state-action loops, a Q-learning update, and accumulated-fitness evaluation, while deferring deep Q-learning.
- Examples
- Uses FrozenLake goals and holes, Q-table training episodes, and Cart Pole genome simulations.
- What you can do after
- Apply it via OpenAI Gym to train a Q-learning agent or score NEAT genomes from environment feedback.
Read how Evolutionary Deep Learning: Genetic algorithms and neural networks explains reward signal →
Experimentation for Engineers
A reward is the observed outcome assigned to a contextual-bandit action, represented here by the user's viewing time.
- How it is used
- Measures the immediate business outcome used to choose and evaluate contextual-bandit actions.
- How deep it goes
- Defines reward, models its measurement, logs observations, evaluates averages, and distinguishes short-term rewards from multistep returns.
- Examples
- Uses post viewing time, ad clicks, song play clicks, a noisy simulator, and logs containing context, action, and reward.
- What you can do after
- Evaluate contextual-bandit decisions with observed business metrics and recognize when a metric is unsuitable as a single-action reward.
Read how Experimentation for Engineers explains reward signal →
Transformers in Action
A reward signal is a numerical feedback signal that indicates the desirability of an action, output, or state in reinforcement learning.
- How it is used
- Guides sequential decisions in an MDP and supplies output scores for language-model reinforcement learning.
- How deep it goes
- Defines rewards within an MDP and implements output-level scoring functions, but does not derive reward optimization mathematically.
- Examples
- Uses dog-training feedback, length-and-diversity scores for Financial Phrasebank sentences, and graded rewards for structured completions.
- What you can do after
- Apply it via trlX or TRL by defining an output-scoring function and supplying rewards for training.
Introduction to Generative AI, Second Edition
A reward is a numerical feedback value representing progress toward a task and reinforcing a model behavior.
- How it is used
- Guides model behavior by scoring outcomes during reinforcement learning and post-training alignment.
- How deep it goes
- Defines numerical feedback and explains its behavioral effect, but stops short of implementation details or formal optimization.
- Examples
- Uses model completions, human preference judgments, AI-rated responses, and the reinforcement learning cycle as teaching examples.
- What you can do after
- Evaluate how rewards shape model behavior and distinguish human-judged from AI-judged alignment feedback.
Read how Introduction to Generative AI, Second Edition explains reward signal →
Deep Learning with Python, Third Edition
Feedback value used in reinforcement learning to indicate how desirable an action or output is.
- How it is used
- Uses response scores to provide feedback for reinforcement learning and preference-based language-model tuning.
- How deep it goes
- Defines scalar response scoring and sketches filtering, while deferring specialized gradient-update algorithms and deeper reinforcement-learning details.
- Examples
- Demonstrates prompt-response scoring, cutoff filtering, and iterative regeneration with improved reward models.
- What you can do after
- Apply it via a reward model to score generated responses and select or steer training examples.
Read how Deep Learning with Python, Third Edition explains reward signal →
LLMs in Production
A reward system is a mechanism that assigns feedback signals to actions or outcomes to guide an agent's behavior.
- How it is used
- Guides reinforcement learning by scoring actions instead of supplying predefined labels.
- How deep it goes
- Explains reward functions, intermediate incentives, and reward hacking, while only naming reward models and PPO.
- Examples
- Uses StarCraft winning, army growth, and Pylon-building to demonstrate delayed feedback and reward hacking.
- What you can do after
- Evaluate reward designs by checking whether intermediate incentives support the intended outcome.
Related concepts
- reinforcement learningused by · Reinforcement learning is a machine-learning paradigm in which an agent learns actions or a policy through interaction with an environment and evaluative feedback, such as rewards.
- regretmeasured by · Regret is the difference between the reward or utility obtained from a choice and that obtainable from the best alternative under the same circumstances, often accumulated across decisions.
- reward modelproduced by · A reward model is a learned model that assigns scores to outputs, actions, or outcomes to represent their desirability according to preferences or evaluation criteria.
- environmentproduced by · An environment is the external context, including its entities and conditions, in which a system, process, or agent exists or operates.
- actionproduced by · An action is a choice or operation selected or performed by an actor that can affect a system or environment or produce an outcome.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.