A retailer deciding tomorrow’s price has several choices, each with uncertain consequences. In reinforcement learning, an action is the choice an agent makes in its current state. It can influence both the immediate reward and the probability of moving to a particular next state. section 2.3 section 2.4
The agent observes a state, selects an action, receives feedback, and moves to a new state. A policy maps states to actions; a stochastic policy maps states to probability distributions over actions. The aim is not simply to pick the best immediate outcome, but to account for consequences over time. section 1.7 section 10.3 section 2.7
Setting a daily price
An online store chooses a price from a set of possible values each day. Its state can include inventory, current price, expected demand, and days remaining in the month. Daily profit is the reward; purchases influenced by the chosen price affect the next day’s inventory. Omitting inventory or pending backorders can leave the agent without information needed to predict what happens next. section 2.4
Action design should reflect the business decision and its constraints. For example, scheduling actions choose staffing levels, while maintenance actions decide whether to maintain each machine. Hard constraints can be enforced with action masking and penalties; soft constraints can be handled with penalties and trade-offs. section 2.4 section 2.7
Table 2.1 Comparing Different Business Problems Modeled as Markov Decision Processes
|
Example
|
State design
|
Action design
|
Reward design
|
Non-Makrov if…
|
|
Dynamic Pricing in Online Store
|
Inventory level, current price, demand forecast, days left in month
|
Set daily price
|
Daily profit (revenue – costs)
|
Ignoring inventory level or pending backorders
|
|
Workforce Scheduling in Call Center
|
Queue length, available agents, forecasted call arrivals
|
Choose number of agents to schedule
|
Negative cost: waiting time penalties + salaries
|
Ignoring agent fatigue or cumulative workload history
|
|
Preventive Maintenance for Machines
|
Machine condition, time since last maintenance, environmental factors
|
Choose whether to maintain each machine
|
Cost of maintenance vs cost of breakdowns
|
Omitting time since last maintenance or operational history
|
|
Customer Retention Campaign
|
Active status, weeks since last engagement, recent activity, subscription tenure
|
Choose between offer, reminder, or silence
|
+1 for retention, 0 for churn
|
Missing recent activity history or engagement recency
|
Compare action choices alongside states, rewards, and missing-state risks across business problems. section 2.4
Choosing an advertisement
In an ad campaign, each action shows one of five ads. A click yields reward 1 and no click yields 0. The agent learns from observed outcomes while balancing trying ads with showing those that appear effective. section 5.3
Actions need not be business controls. In language generation, an action selects the next token from the vocabulary. The state is the prompt plus the tokens generated so far, and each selected token extends that state. The model’s policy supplies probabilities over the possible next tokens. section 10.3
















