liveBook · Manning

Dynamic pricing

As explained in Applied Reinforcement Learning

Dynamic pricing is the adjustment of prices over time in response to changing demand, inventory conditions, expiration, or competitive factors.

Dynamic pricing adjusts prices over time as demand, inventory, expiration, or competitors’ prices change. For perishable goods, the decision is recurring and measurable: choose prices that make the most of inventory before it loses value. section 1.4

The trade-off is straightforward but uncertain. A high price can leave inventory unsold; a low price can sell out too quickly, missing revenue from customers willing to pay more. Each decision affects what remains available later. section 3.3

A simple reinforcement learning environment turns the selling window into a sequence of decisions:

  • State: remaining sales periods and inventory. section 3.3
  • Action: choose a price from a predefined list. section 3.3
  • Reward: revenue from the period’s sales, with a penalty for inventory left after the final period. section 3.3
  • Transition: demand responds to price, grows slightly with urgency, and includes random variation; sales cannot exceed remaining inventory. section 3.3
Table 3.2 Elements of Markov decision process for perishable dynamic pricing
Markov decision process elements
Definitions
State
How many sales periods are left.
How many items are left to sell.
Action
The agent chooses a price from a predefined list, e.g., [8, 10, 12, 15, 18].
Reward
Calculated as the revenue from sales in that period.
Penalty if items remain unsold after the last period.
Transition Dynamics
Demand drops as the price increases (price sensitivity).
Demand slightly increases over time due to urgency (people rush to buy near the end).
Random noise is added to simulate uncertainty.

Compare the state, action, reward, and demand assumptions in the perishable-goods model. section 3.3

State design depends on the business problem. An online store maximizing monthly profit might also track the current price, expected demand, and days remaining. Omitting inventory or pending backorders can make future outcomes depend on information missing from the recorded state, undermining the Markov property. section 2.4

Testing a perishable-goods environment

A simulation starts with 100 items and 15 selling periods, with prices chosen from 8, 10, 12, 15, and 18. A random agent selects a price each period to sanity-check the environment. This checks the mechanics; it does not establish an optimized pricing policy. section 3.3

Listing 3.9 Environment Step Logic and Demand Model Function
def _calculate_demand(self, price_chosen): #A
    base_price_effect = max(0, self.base_demand_per_period - self.price_sensitivity * price_chosen) #B
    urgency_factor = 1.0 + (self.total_time_periods - self.time_periods_left) / (2.0 * self.total_time_periods) #C
 
    noise = np.random.uniform(-0.1, 0.1) * base_price_effect #D
    potential_demand = int(round(max(0, base_price_effect * urgency_factor + noise))) #D
    return potential_demand #D
 
def step(self, action_idx): #E
    price_chosen = self.price_levels[action_idx] #F
    units_sold_this_period = 0
    potential_demand = 0
 
    if self.current_inventory > 0:
        potential_demand = self._calculate_demand(price_chosen) #G
        units_sold_this_period = min(potential_demand, #H
self.current_inventory) #H
    
    self.current_inventory -= units_sold_this_period #I
    revenue_this_period = price_chosen * units_sold_this_period #I
    reward = revenue_this_period #I
    
    self.time_periods_left -= 1 #J
    self.current_time_period += 1 #J
    
    done = self.time_periods_left <= 0 or self.current_inventory <= 0 #K
    
    if done and self.time_periods_left <= 0: #L
        penalty_for_unsold = self.current_inventory *  self.unsold_penalty_per_item #L
        reward -= penalty_for_unsold #L
        
    info = {"price_chosen": price_chosen, "units_sold": units_sold_this_period} #M
    self._update_history(price_chosen, units_sold_this_period, revenue_this_period) #M
    
    return self._get_state(), reward, done, info

Inspect how simulated demand, price sensitivity, urgency, noise, and available inventory interact in a period. section 3.3

How Applied Reinforcement Learning covers it

How deep it goes
Models states, actions, rewards, and stochastic transitions, then implements and sanity-checks a simplified environment without detailed domain-specific complexities.
Examples
Uses an online store’s monthly profit problem and a perishable-inventory simulation with a random agent.
What you can do after
Build a simplified pricing environment from scratch and sanity-check its mechanics with a random agent.
How it connects
Connects dynamic pricing to reinforcement learning, Markov decision processes, state design, and price-sensitive demand.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.