A reward function assigns numerical feedback to decisions and their outcomes. In reinforcement learning, it translates goals into a signal that steers an agent’s behavior. For a factory, that signal might combine sales revenue with the costs of production, inventory, and unmet orders. section 2.5 section 3.1
A reward can depend on the current state, action, and resulting state. An environment typically calculates it as part of a step, alongside the transition to the next state. Rewards accumulated over a trajectory let policies be compared over time, rather than judging a single decision in isolation. section 3.1 section 2.5 section 4.4
Scoring a production day
A factory chooses whether to produce scooters, produce bikes, or idle. Its daily reward adds revenue from fulfilled demand and subtracts production, holding, setup, and backorder costs. Switching products incurs a $200 setup cost, so a change must be worthwhile enough to offset it. The score captures trade-offs among sales, inventory, and service rather than rewarding production volume alone. section 2.5
Listing 2.7 Reward function
def reward_function(state, action, demand,params:FactoryProblemDefinition,
produced_scooter, produced_bike, \\
backorder_scooter, backorder_bike, next_state):
_day, _inv_scooter, _inv_bike, current_setup = state #A
_next_day, next_inv_scooter, next_inv_bike, _next_setup = next_state #A
demand_scooter, demand_bike = demand #A
revenue = (demand_scooter - backorder_scooter) * \\
params.price_scooter + (demand_bike - backorder_bike) * \\
params.price_bike #B
production_cost = produced_scooter * params.cost_scooter + #C
produced_bike * params.cost_bike #C
setup_penalty = params.setup_cost \\ #D
if (action in (0, 1) and action != current_setup) else 0 #D
holding_penalty = params.holding_cost * \\
(next_inv_scooter + next_inv_bike) #E
backorder_penalty = params.backorder_cost * \\ #F
(backorder_scooter + backorder_bike) #F
total_reward = revenue - production_cost - \\ #G
setup_penalty - holding_penalty - backorder_penalty #G
return total_reward
Inspect how the factory reward uses demand, production, setup, backorders, and the next state. section 2.5
Practical reward design starts with the business decision:
- Identify the objectives and translate them into measurable reward terms. Include costs and undesirable outcomes, not just the activity the agent should increase. section 2.6 section 2.5
- Choose when feedback arrives and balance term scales. Stepwise feedback can guide intermediate behavior, while a terminal reward or penalty can reflect the final outcome. section 2.7 section 3.3
- Distinguish hard constraints from soft trade-offs. Use action masking alongside penalties for hard constraints; soft constraints can be handled through penalties and dynamic trade-offs. section 2.7
- Test what the agent actually learns. Simulations, audits, and scenario tests can reveal shortcuts that improve the score without advancing the real objective. section 2.6.6
Useful output versus a proxy
Rewarding machine runtime alone can encourage a machine to run continuously and create unwanted inventory. Similarly, paying for recommendation clicks can favor clickbait over sales and user satisfaction. Combining clicks with revenue and a penalty for bounces is one way to make the signal reflect more of the intended outcome. section 2.6.6
Warehouse and pricing rewards
A warehouse robot receives a small penalty for each valid move, larger penalties for walls and obstacles, rewards for pickups, and a large reward for returning to the depot after collecting everything. In perishable pricing, each period’s sales revenue is rewarded, while leftover inventory receives a terminal penalty. Both designs balance immediate feedback with completion of the wider task. section 3.2 section 3.3
Table 3.1 Key reward signals used in warehouse order picking environment
|
Agent behavior
|
Reward
|
Purpose
|
|
Move to a valid (empty) cell
|
–0.1
|
Small step penalty to encourage shorter, more efficient paths
|
|
Hit wall (outside grid)
|
–0.5
|
Penalize invalid moves outside the grid limits
|
|
Attempt to move into a blocked cell (hit obstacle)
|
–0.7
|
Stronger penalty to discourage unsafe or infeasible paths
|
|
Successfully pick up an item
|
+15.0
|
Significant reward for achieving one of the primary objectives
|
|
Return to the depot
after collecting all items
|
+100.0
|
Large terminal reward for completing the full task successfully
|
|
Episode ends without full completion (e.g., time limit reached)
|
–20.0
|
Penalty for failing to complete the task within the allowed steps
|
Compare the warehouse reward values and the behavior each is intended to encourage. section 3.2
Not every useful reward is an explicit business formula. For language-model feedback, qualities such as helpfulness and safety can be difficult to express directly, so a reward can be learned from human preferences and applied to a completed response. For verifiable answers, a verifier can instead extract an answer and compare it with a reference to produce a scalar score. section 10.3 section 11.7.4


