liveBook · Manning

Neural network

As explained in Applied Reinforcement Learning

A neural network is a parameterized computational model composed of interconnected transformations that map inputs to outputs.

In reinforcement learning, a neural network is a parameterized function that maps inputs to outputs. It replaces a lookup table when the state space is too large, and can generalize across states. section 8.6

An RL network can represent:

  • A value function that estimates how good each action is in a state. section 12.1
  • A policy that estimates how likely each action is in a state. section 12.1
  • Both a policy and a value function, as in the actor–critic pattern. section 12.1

Networks stack linear transformations and nonlinear activations to learn features. Backpropagation sends gradients backward through the layers to update their parameters. Two to four hidden layers suit many RL problems; high-dimensional images or sequences may call for CNNs or LSTMs. section 8.6

Value estimate or portfolio policy

A DQN network estimates action values from a state. In a portfolio manager, a PPO network instead represents a policy, assigning likelihoods to eight allocation choices for the day's state. section 12.1

Figure 12.1 The anatomy of a deep reinforcement learning algorithm.

Use the algorithm anatomy diagram to see the network as one component in a larger deep RL system. section 12.1

How Applied Reinforcement Learning covers it

How deep it goes
Details stacked transformations, activations, backpropagation, and typical hidden-layer counts.
Examples
Illustrates DQN action-value estimates and a PPO portfolio policy over eight allocation choices.
What you can do after
Understand how networks represent RL functions and when deeper architectures suit high-dimensional inputs.
How it connects
Connects neural networks to function approximation, value functions, policies, and actor–critic methods.

Also explained in

Compare how 11 books explain neural network →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.