liveBook · Manning

Target network

As explained in Applied Reinforcement Learning

A target network is a delayed copy of an action-value network whose parameters are updated periodically and used to compute learning targets.

A target network is a delayed copy of the action-value network. Its frozen parameters compute learning targets while the online network learns. section 8.6

Without this separation, changes to the network also shift the targets it is trying to match, creating feedback that can amplify errors. Periodic updates make those targets more stable. section 8.3 section 8.6

Set the target network’s update interval, then tune it against training behavior:

  • Update every C steps; a typical interval is 1,000–10,000 steps. section 8.6
  • Increase C if loss oscillates; decrease C if learning stalls and Q-values change slowly. section 8.6
Figure 8.6 Deep Q-Networks learning systems

Use the DQN system diagram to follow how the target network fits into the learning flow. section 8.3

How Applied Reinforcement Learning covers it

How deep it goes
Defines frozen parameters, periodic updates, and tuning signals, but does not provide a target-network implementation.
Examples
Illustrates update intervals of 1,000–10,000 steps and uses oscillating loss or slow Q-value changes as tuning signals.
What it assumes
Contrasts writable Q-learning table entries with learned parameters in a neural action-value network.
What you can do after
Configure the update interval and adjust it in response to loss oscillation or stalled Q-value changes.
How it connects
Connects target networks to DQN, experience replay, and Double DQN.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.