Dynamic pricing adjusts prices over time as demand, inventory, expiration, or competitors’ prices change. For perishable goods, the decision is recurring and measurable: choose prices that make the most of inventory before it loses value. section 1.4
The trade-off is straightforward but uncertain. A high price can leave inventory unsold; a low price can sell out too quickly, missing revenue from customers willing to pay more. Each decision affects what remains available later. section 3.3
A simple reinforcement learning environment turns the selling window into a sequence of decisions:
- State: remaining sales periods and inventory. section 3.3
- Action: choose a price from a predefined list. section 3.3
- Reward: revenue from the period’s sales, with a penalty for inventory left after the final period. section 3.3
- Transition: demand responds to price, grows slightly with urgency, and includes random variation; sales cannot exceed remaining inventory. section 3.3
Table 3.2 Elements of Markov decision process for perishable dynamic pricing
|
Markov decision process elements
|
Definitions
|
|
State
|
How many sales periods are left.
How many items are left to sell.
|
|
Action
|
The agent chooses a
price from a predefined list, e.g., [8, 10, 12, 15, 18].
|
|
Reward
|
Calculated as the revenue from sales in that period.
Penalty if items remain unsold after the last period.
|
|
Transition Dynamics
|
Demand drops as the price increases (price sensitivity).
Demand slightly increases over time due to
urgency (people rush to buy near the end).
Random noise is added to simulate uncertainty.
|
Compare the state, action, reward, and demand assumptions in the perishable-goods model. section 3.3
State design depends on the business problem. An online store maximizing monthly profit might also track the current price, expected demand, and days remaining. Omitting inventory or pending backorders can make future outcomes depend on information missing from the recorded state, undermining the Markov property. section 2.4
Testing a perishable-goods environment
A simulation starts with 100 items and 15 selling periods, with prices chosen from 8, 10, 12, 15, and 18. A random agent selects a price each period to sanity-check the environment. This checks the mechanics; it does not establish an optimized pricing policy. section 3.3
Listing 3.9 Environment Step Logic and Demand Model Function
def _calculate_demand(self, price_chosen): #A
base_price_effect = max(0, self.base_demand_per_period - self.price_sensitivity * price_chosen) #B
urgency_factor = 1.0 + (self.total_time_periods - self.time_periods_left) / (2.0 * self.total_time_periods) #C
noise = np.random.uniform(-0.1, 0.1) * base_price_effect #D
potential_demand = int(round(max(0, base_price_effect * urgency_factor + noise))) #D
return potential_demand #D
def step(self, action_idx): #E
price_chosen = self.price_levels[action_idx] #F
units_sold_this_period = 0
potential_demand = 0
if self.current_inventory > 0:
potential_demand = self._calculate_demand(price_chosen) #G
units_sold_this_period = min(potential_demand, #H
self.current_inventory) #H
self.current_inventory -= units_sold_this_period #I
revenue_this_period = price_chosen * units_sold_this_period #I
reward = revenue_this_period #I
self.time_periods_left -= 1 #J
self.current_time_period += 1 #J
done = self.time_periods_left <= 0 or self.current_inventory <= 0 #K
if done and self.time_periods_left <= 0: #L
penalty_for_unsold = self.current_inventory * self.unsold_penalty_per_item #L
reward -= penalty_for_unsold #L
info = {"price_chosen": price_chosen, "units_sold": units_sold_this_period} #M
self._update_history(price_chosen, units_sold_this_period, revenue_this_period) #M
return self._get_state(), reward, done, info
Inspect how simulated demand, price sensitivity, urgency, noise, and available inventory interact in a period. section 3.3
