A target network is a delayed copy of the action-value network. Its frozen parameters compute learning targets while the online network learns. section 8.6
Without this separation, changes to the network also shift the targets it is trying to match, creating feedback that can amplify errors. Periodic updates make those targets more stable. section 8.3 section 8.6
Set the target network’s update interval, then tune it against training behavior:
- Update every C steps; a typical interval is 1,000–10,000 steps. section 8.6
- Increase C if loss oscillates; decrease C if learning stalls and Q-values change slowly. section 8.6
Figure 8.6 Deep Q-Networks learning systems
Use the DQN system diagram to follow how the target network fits into the learning flow. section 8.3
