A policy maps each state to an action. In goals-based investing, it tells an investor how much wealth to allocate to stocks versus a risk-free bond for a given wealth level and time. The objective is to maximize cumulative reward, defined here by meeting a financial goal over the planning horizon. section 6.2
The policy is state-dependent rather than a fixed glide path. With state (W, t), an optimal action can change when wealth differs even at the same date: an investor ahead of the goal can reduce stock exposure, while one behind it may need greater risk exposure. section 6.2.1
Two methods produce a policy from modeled investment choices:
- Dynamic programming computes expected values backward from the goal date, then records the action with the highest expected value at every wealth-time state in A(W,t). section 6.2.1
- Q-learning estimates Q(W,t,a), the value of each state-action pair, through many simulated episodes; choosing the action with the largest Q value yields the learned policy. section 6.2.2
Goal-based stock allocation
For a $1 million starting portfolio aiming to double in 10 years, the dynamic-programming policy begins with an 85% stock allocation. Along a medium-return wealth path, stock allocation declines; near the goal date, enough wealth can make a risk-free-bond allocation sufficient. section 6.2.1
Figure 6.4 Asset allocations for the three paths of wealth
Compare the policy’s stock allocations across good, medium, and bad wealth paths to see why time alone does not determine the action. section 6.2.1
