Dynamic programming (DP) solves a multiperiod financial decision by working backward from a terminal outcome. It represents wealth at each time as discrete states, evaluates available actions, weights next-period values by transition probabilities, and stores the best expected value and action at every state. For a goals-based plan, the terminal value is 1 when wealth exceeds the goal and 0 otherwise. section 6.2.1 section 6.2
A practical implementation first builds a wealth-time grid and estimates how each asset allocation moves wealth between grid points. Transition probabilities come from the probability that next-period wealth falls between adjacent state midpoints. The illustrated calculation assumes normally distributed stock returns, while allowing other return distributions to replace that assumption. section 6.2.1 section 6.2
Backward recursion follows a repeatable procedure:
- Set terminal values from the objective, such as reaching a $2 million goal. section 6.2.1
- For each earlier wealth-time state and candidate allocation, compute the expected next-period value using transition probabilities. section 6.2.1
- Keep the maximum expected value and the allocation that produces it; then move one period backward. section 6.2.1
- Extend the action set when the objective includes consumption: evaluate both allocation and spending, add current consumption utility, and discount future value. section 6.3.2
Goal investing and lifetime spending
For a 10-period goal, DP can choose among discrete stock-bond allocations at each wealth state to maximize the chance of finishing above a target. In a spending version, consumption becomes an additional action: wealth is reduced by current consumption, the terminal value equals utility of final wealth, and each earlier choice trades current utility against discounted future utility. section 6.2.1 section 6.3.2
DP provides a clear benchmark for sequential planning, but it requires transition probabilities and becomes difficult as states and actions multiply. Reinforcement learning is contrasted as an alternative that can learn without a known next-state probability distribution. section 6.6
Listing 6.3 DP solution for goals-based investing
G = 2000000 #1
nA = 21 #2
V = np.zeros((nW, T+1)) #3
A = np.zeros((nW, T)) #4
EV = np.zeros(nA) #5
W_midpts = compute_midpoints(W)
for j in range(nW): #6
if W[j, T] > G: #6
V[j, T] = 1 #6
for t in range(T-1, -1, -1):
for j in range(nW):
for a in range(nA):
W_tplus1_dist = compute_transition_probs(W[j,t], t, a,
W_midpts)
EV[a] = np.dot(W_tplus1_dist, V[:, t+1])
V[j,t] = EV.max() #7
A[j,t] = EV.argmax() #8
Inspect the nested backward loop that initializes terminal goal values, evaluates every allocation, and records the maximizing value and action. section 6.2.1
