A state is a snapshot of the information relevant to a decision at a particular moment. For an online store setting prices, it might include inventory, the current price, expected demand, and time remaining in the sales period. Choosing what to include determines what the agent can use when it acts. section 2.1 section 2.4
In reinforcement learning, the agent observes a state, chooses an action, receives a reward, and moves to a new state. The action can affect both the immediate reward and the likelihood of different next states. A policy maps states to actions with long-term outcomes in mind. section 1.7 section 2.3
A useful state captures enough information to predict what happens next without needing omitted history: this is the Markov property. For example, leaving pending backorders out of a pricing state can make future outcomes depend on information the agent cannot see. State design is therefore a modeling choice, not just a list of convenient features. section 2.4 section 2.7
Pricing a product
Set the state to inventory, current price, demand forecast, and days remaining. The action is a daily price; the reward can be daily revenue minus costs. Purchases then change inventory and influence the next state. Include relevant backorders if they affect future decisions. section 2.4
States can also expose changing constraints. A scheduling state might include employee availability; a data-center state might include remaining power budget and maintenance windows. Making these conditions visible helps the agent make context-aware choices, but visibility alone does not enforce a hard constraint. Action masking can prevent an invalid choice. section 2.6.2 section 2.6.4
Generating a response
For language generation, the state is the prompt combined with all tokens generated so far. Choosing the next token is the action; appending it extends the state for the next decision. section 10.3
Representation matters when a system underperforms: an agent cannot act on information its state has discarded. In a portfolio environment, separate market, news, portfolio, and execution features make it possible to switch inputs off and test their contribution. Use only information available at decision time; future data in the state creates leakage. section 12.9.1 section 12.4 section 12.11
Listing 12.2 The state: four feature blocks with a news switch
def _market_features(self, day):
lb = self.cfg.lookback
price_win = self.prices[day - lb + 1: day + 1] #A
...
return feats.reshape(-1)
def _news_features(self, day):
if not self.cfg.use_news or self.signal_matrix is None:
return np.zeros(self.news_dim) #B
return self.signal_matrix[day] #C
def _get_state(self) -> np.ndarray: #D
blocks = []
if self.cfg.include_market_features:
blocks.append(self._market_features(self.day))
blocks.append(self._news_features(self.day))
if self.cfg.include_portfolio_features:
blocks.append(self._portfolio_features())
if self.cfg.include_execution_features:
blocks.append(self._execution_features())
return np.concatenate(blocks).astype(np.float32) See how the portfolio state is assembled from feature blocks, including a switch for the news signal. section 12.4



















































