liveBook · Manning

Fixed-width state representation

As explained in Applied Reinforcement Learning

A state representation that keeps a constant number of features even when some features are disabled or contain no information.

In a portfolio agent, a fixed-width state keeps the news feature block present even when news is disabled, filling it with zeros. The state remains 63 numbers wide, so the same agent architecture can run across news modes and comparisons stay consistent. section 12.4

Listing 12.2 The state: four feature blocks with a news switch
def _market_features(self, day):
    lb = self.cfg.lookback
    price_win = self.prices[day - lb + 1: day + 1]  #A
    ...
    return feats.reshape(-1)
 
def _news_features(self, day):
    if not self.cfg.use_news or self.signal_matrix is None:
        return np.zeros(self.news_dim)  #B
    return self.signal_matrix[day] #C
 
def _get_state(self) -> np.ndarray:    #D
        blocks = []
        if self.cfg.include_market_features:
            blocks.append(self._market_features(self.day))
        blocks.append(self._news_features(self.day))    
        if self.cfg.include_portfolio_features:
            blocks.append(self._portfolio_features())
        if self.cfg.include_execution_features:
            blocks.append(self._execution_features())
        return np.concatenate(blocks).astype(np.float32) 

See how the news switch returns zeros while the state is assembled from feature blocks. section 12.4

How Applied Reinforcement Learning covers it

How deep it goes
Represents state as feature blocks and zeroes disabled news rather than changing input width.
Examples
Uses a zero-filled news block to preserve a 63-number state when news is disabled.
What you can do after
Evaluate news modes with the same agent architecture and a consistent state width.
How it connects
Connects state representation to feature ablation and reinforcement learning agent architecture.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.