chapter six

6 Financial Planning Using Reinforcement Learning

 

This chapter covers

  • Solving a goals-based investing problem using dynamic programming
  • Solving the same goals-based problem using reinforcement learning (AI)
  • Discussing how utility functions can be used in financial planning
  • Applying reinforcement learning to optimize spending using utility functions
  • Extending the model to include longevity risk in order to answer questions like when to take Social Security

When we looked at asset allocation earlier, we were essentially performing a single-period optimization. However, most financial planning decisions involve making decisions over multiple periods. And the decisions made today - not only how to allocate assets but how much to spend, whether to claim Social Security, when to retire, what accounts to withdraw from, etc. - affect decisions in the future. These multiperiod, dynamic models, which economists sometimes call “lifecycle models,” are much more complicated to optimize.

6.1 A Goals-Based Investing Example

6.2 An Introduction to Reinforcement Learning

6.2.1 Solution Using Dynamic Programming

6.2.2 Solution Using Q-Learning

6.3 Utility Function Approach

6.3.1 Understanding Utility Functions

6.3.2 Optimal Spending Using Utility Functions

6.4 Longevity Risk

6.5 Other Extensions

6.6 Summary