An accumulating trace tracks eligibility for credit in TD(λ). Each step, traces decay by γλ; when a state is visited, add 1 to its decayed trace. Repeated visits can raise a trace above 1, giving frequently visited states stronger credit. section 6.5
A repeated visit
With γ = 0.9 and λ = 0.8, a trace of 0.5 decays to 0.36. Visiting that state adds 1, making its trace 1.36. section 6.5
Figure 6.8 Update mechanism of temporal difference learning with eligibility traces.
See the eligibility-trace update mechanism in temporal-difference learning. section 6.5
