An action is a decision an agent makes that can change its environment’s state. In goals-based investing, the agent is an investor, the state combines current wealth and time, and an action is the percentage of wealth allocated to a stock fund rather than a risk-free bond. Rewards or penalties from resulting outcomes shape the policy that chooses actions. section 6.2 section 6.2
Actions are evaluated in context, not as universally good choices. A policy maps states to actions: the appropriate stock allocation can depend on both remaining time and current wealth. This differs from a fixed glide path that changes only with time. section 6.2 section 6.2.1
Dynamic programming selects an action by working backward from the investment goal:
- Assign terminal reward 1 when wealth exceeds the goal and 0 otherwise. section 6.2.1
- For each wealth state and possible asset allocation, compute expected value from transition probabilities and next-period values. section 6.2.1
- Store the highest expected value as V(W,t) and the corresponding action as A(W,t), then repeat for earlier periods. section 6.2.1
Choosing an allocation at a wealth state
At each discrete wealth-time node, the procedure tests every available allocation. With 21 action levels, an action index maps to a stock weight from 0% through 100%; the remainder goes to bonds. The selected action is the allocation with the greatest expected probability of reaching the terminal wealth goal. section 6.2.1 section 6.2.1
Listing 6.3 DP solution for goals-based investing
G = 2000000 #1
nA = 21 #2
V = np.zeros((nW, T+1)) #3
A = np.zeros((nW, T)) #4
EV = np.zeros(nA) #5
W_midpts = compute_midpoints(W)
for j in range(nW): #6
if W[j, T] > G: #6
V[j, T] = 1 #6
for t in range(T-1, -1, -1):
for j in range(nW):
for a in range(nA):
W_tplus1_dist = compute_transition_probs(W[j,t], t, a,
W_midpts)
EV[a] = np.dot(W_tplus1_dist, V[:, t+1])
V[j,t] = EV.max() #7
A[j,t] = EV.argmax() #8
The implementation loops over wealth states and discrete asset-allocation actions, compares their expected values, and records the maximizing action in A. section 6.2.1
