A pricing agent needs feedback after it sets a price. Daily profit can provide that feedback as a reward signal: a single number indicating how desirable an outcome was. Rewards steer the agent toward decisions that improve cumulative outcomes, not just immediate gains. section 2.4 section 1.1
In a Markov decision process, an agent chooses an action in its current state, receives a reward, and moves to a next state. The action affects both the immediate reward and the probabilities of future states. Learning which earlier decisions contributed to success or failure is the credit-assignment challenge. section 2.3 section 1.1
Reward engineering translates business goals into a signal an agent can learn from. A useful design may combine objectives such as profit and service quality with penalties for undesirable outcomes. Give feedback at appropriate steps and watch for mismatched scales between terms. For hard limits, use action masking alongside penalties; soft constraints can be handled with penalties and trade-offs. section 2.6 section 2.7
Rewards for business decisions
An online store can reward daily profit: sales revenue minus associated costs. A call center can instead penalize customer waiting time and labor expense. In an ad campaign, the signal can be simpler: 1 for a click and 0 for no click; the agent uses observed clicks to learn which ads perform well. section 2.4 section 5.3
Table 2.1 Comparing Different Business Problems Modeled as Markov Decision Processes
|
Example
|
State design
|
Action design
|
Reward design
|
Non-Makrov if…
|
|
Dynamic Pricing in Online Store
|
Inventory level, current price, demand forecast, days left in month
|
Set daily price
|
Daily profit (revenue – costs)
|
Ignoring inventory level or pending backorders
|
|
Workforce Scheduling in Call Center
|
Queue length, available agents, forecasted call arrivals
|
Choose number of agents to schedule
|
Negative cost: waiting time penalties + salaries
|
Ignoring agent fatigue or cumulative workload history
|
|
Preventive Maintenance for Machines
|
Machine condition, time since last maintenance, environmental factors
|
Choose whether to maintain each machine
|
Cost of maintenance vs cost of breakdowns
|
Omitting time since last maintenance or operational history
|
|
Customer Retention Campaign
|
Active status, weeks since last engagement, recent activity, subscription tenure
|
Choose between offer, reminder, or silence
|
+1 for retention, 0 for churn
|
Missing recent activity history or engagement recency
|
Compare how rewards, states, and actions differ across business decisions. section 2.4
A reward can also encourage the wrong behavior. Paying for machine runtime may produce unwanted inventory, while rewarding clicks alone can favor clickbait over sales and user satisfaction. Choose proxies carefully, combine measures when appropriate, and use simulations, audits, and scenario testing to look for shortcuts. section 2.6.6
The algorithm does not decide the business objective for you. For example, a portfolio reward must balance returns and risk without inadvertently encouraging the agent never to invest. Reward design is part of designing and evaluating the whole decision system. section 12.2















