liveBook · Manning

Deep neural network

As explained in Applied Reinforcement Learning

A deep neural network is a neural network with multiple learned layers between its input and output.

A deep neural network is a parameterized function approximator. It learns layer-by-layer transformations of raw inputs, rather than relying on hand-designed features, to produce predictions such as action values. section 8.2

In deep reinforcement learning, a network can:

  • Process each layer with a weighted sum and bias, followed by a nonlinear activation that lets stacked layers model complex relationships. section 8.2
  • Learn useful representations from high-dimensional states and output one Q-value per discrete action. section 8.5
  • Update parameters by reducing prediction error, with backpropagation computing gradients through the layers. section 8.2 section 8.6

From raw input to action values

An image or sensor vector can enter the network as a raw state; learned layers transform it before the network estimates Q-values. section 8.2 section 8.5

Figure 8.4 End-to-end learning in deep neural networks

Trace the forward pass, prediction, error, and parameter-update loop. section 8.2

How Applied Reinforcement Learning covers it

How deep it goes
Explains layered weighted transformations, nonlinear activations, learned representations, and gradient-based updates.
Examples
Uses images, sensor vectors, and medical records as high-dimensional inputs, with Q-values as network outputs.
What it assumes
Builds on linear function approximation and temporal-difference error while explaining learned feature transformations through network layers.
What you can do after
Apply deep networks to approximate Q-values for high-dimensional states and select among discrete actions.
How it connects
Connects neural networks to function approximation, reinforcement learning, Q-learning, backpropagation, and temporal-difference learning.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.