liveBook · Manning

Action

As explained in Applied Reinforcement Learning

An action is a choice made by an agent that can affect the subsequent state of its environment.

A retailer deciding tomorrow’s price has several choices, each with uncertain consequences. In reinforcement learning, an action is the choice an agent makes in its current state. It can influence both the immediate reward and the probability of moving to a particular next state. section 2.3 section 2.4

The agent observes a state, selects an action, receives feedback, and moves to a new state. A policy maps states to actions; a stochastic policy maps states to probability distributions over actions. The aim is not simply to pick the best immediate outcome, but to account for consequences over time. section 1.7 section 10.3 section 2.7

Setting a daily price

An online store chooses a price from a set of possible values each day. Its state can include inventory, current price, expected demand, and days remaining in the month. Daily profit is the reward; purchases influenced by the chosen price affect the next day’s inventory. Omitting inventory or pending backorders can leave the agent without information needed to predict what happens next. section 2.4

Action design should reflect the business decision and its constraints. For example, scheduling actions choose staffing levels, while maintenance actions decide whether to maintain each machine. Hard constraints can be enforced with action masking and penalties; soft constraints can be handled with penalties and trade-offs. section 2.4 section 2.7

Table 2.1 Comparing Different Business Problems Modeled as Markov Decision Processes
Example
State design
Action design
Reward design
Non-Makrov if…
Dynamic Pricing in Online Store
Inventory level, current price, demand forecast, days left in month
Set daily price
Daily profit (revenue – costs)
Ignoring inventory level or pending backorders
Workforce Scheduling in Call Center
Queue length, available agents, forecasted call arrivals
Choose number of agents to schedule
Negative cost: waiting time penalties + salaries
Ignoring agent fatigue or cumulative workload history
Preventive Maintenance for Machines
Machine condition, time since last maintenance, environmental factors
Choose whether to maintain each machine
Cost of maintenance vs cost of breakdowns
Omitting time since last maintenance or operational history
Customer Retention Campaign
Active status, weeks since last engagement, recent activity, subscription tenure
Choose between offer, reminder, or silence
+1 for retention, 0 for churn
Missing recent activity history or engagement recency

Compare action choices alongside states, rewards, and missing-state risks across business problems. section 2.4

Choosing an advertisement

In an ad campaign, each action shows one of five ads. A click yields reward 1 and no click yields 0. The agent learns from observed outcomes while balancing trying ads with showing those that appear effective. section 5.3

Actions need not be business controls. In language generation, an action selects the next token from the vocabulary. The state is the prompt plus the tokens generated so far, and each selected token extends that state. The model’s policy supplies probabilities over the possible next tokens. section 10.3

How Applied Reinforcement Learning covers it

How deep it goes
Defines actions and their effects on rewards and transitions, then works through business and language-model mappings.
Examples
Illustrates actions through daily pricing, staffing, ad selection, and next-token generation.
What you can do after
Builds a Python ad-campaign simulation from scratch to compare action-selection strategies.
How it connects
Connects actions to states, rewards, transitions, policies, action masking, and multi-armed bandits.

Also explained in

Compare how 17 books explain action →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.