liveBook · Manning

Deep reinforcement learning

As explained in Applied Reinforcement Learning

A reinforcement-learning approach that uses deep neural networks to approximate policies, value functions, or related decision components.

Deep reinforcement learning uses neural networks to approximate decision functions when an agent faces too many possible states for a table of values. A network can represent action values, a policy, or both. section 8.0 section 12.1

Algorithms differ in how they fill the shared structure:

  • DQN learns action values and acts greedily; REINFORCE learns a policy directly; PPO combines a policy and critic; GRPO uses groups of sampled answers as a baseline. section 12.1
  • To compare methods, look for which part changes while the broader learning loop remains. section 12.1

Sequential decisions

Cancer treatment can be framed as adjusting doses over multiple sessions as tumor response and side effects evolve. A portfolio agent can instead use a policy to choose among allocation options. section 8.0 section 12.1

Figure 12.1 The anatomy of a deep reinforcement learning algorithm.

Use the algorithm anatomy figure to orient the shared parts and loop. section 12.1

How Applied Reinforcement Learning covers it

How deep it goes
Explains neural networks as value functions, policies, or both, and frames algorithms as components wired into a loop.
Examples
Uses uncertain tumor-dose adjustments and an eight-choice allocation policy for a six-ETF portfolio.
What you can do after
Understand algorithm differences by identifying which shared component each method changes.
How it connects
Connects deep reinforcement learning to Q-learning, DQN, REINFORCE, PPO, GRPO, and actor–critic methods.

Also explained in

Compare how 2 books explain deep reinforcement learning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.