In reinforcement learning, a neural network is a parameterized function that maps inputs to outputs. It replaces a lookup table when the state space is too large, and can generalize across states. section 8.6
An RL network can represent:
- A value function that estimates how good each action is in a state. section 12.1
- A policy that estimates how likely each action is in a state. section 12.1
- Both a policy and a value function, as in the actor–critic pattern. section 12.1
Networks stack linear transformations and nonlinear activations to learn features. Backpropagation sends gradients backward through the layers to update their parameters. Two to four hidden layers suit many RL problems; high-dimensional images or sequences may call for CNNs or LSTMs. section 8.6
Value estimate or portfolio policy
A DQN network estimates action values from a state. In a portfolio manager, a PPO network instead represents a policy, assigning likelihoods to eight allocation choices for the day's state. section 12.1
Figure 12.1 The anatomy of a deep reinforcement learning algorithm.

Use the algorithm anatomy diagram to see the network as one component in a larger deep RL system. section 12.1










