liveBook · Manning

Discount rate

As explained in Build a Robo-Advisor with Python (From Scratch)

A discount rate in reinforcement learning is a parameter that determines how strongly future rewards are weighted relative to immediate rewards.

The discount rate, γ, controls how strongly Q-learning values future rewards relative to immediate rewards. It ranges from 0 to 1: γ = 0 is myopic, while γ = 1 gives current and future rewards equal weight. This subjective preference differs from an interest rate. section 6.2.2

Financial planning example

For asset-allocation decisions, Q-learning updates the current action value using the observed reward plus γ times the best estimated value at the next wealth state. A smoker’s preference for an immediate cigarette illustrates why subjective discounting can differ from bond discounting. section 6.2.2

How Build a Robo-Advisor with Python (From Scratch) covers it

How deep it goes
Defines γ, gives its boundary cases, distinguishes it from interest, and shows its placement in the Q-learning update.
Examples
Uses asset allocation, a smoker’s immediate craving, and bond discounting to contrast subjective and financial discounting.
What it assumes
Assumes familiarity with states, actions, rewards, Q-values, and iterative learning, while defining γ from its role in the update.
What you can do after
Apply it via Q-learning by selecting γ to represent an agent’s preference for immediate versus future rewards.
How it connects
Connects discount rate with Q-learning, temporal-difference learning, rewards, learning rate, and asset allocation.
Cover of Build a Robo-Advisor with Python (From Scratch)
liveBookmanning.com
Read Build a Robo-Advisor with Python (From Scratch) on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Build a Robo-Advisor with Python (From Scratch). The section links point to the parts of the book it draws on.