Q-learning
Q-learning is a model-free reinforcement-learning algorithm that estimates state-action values through updates based on rewards and subsequent state values.
Evolutionary Deep Learning: Genetic algorithms and neural networks
Q-learning is a model-free reinforcement-learning algorithm that learns action values by updating estimates of expected return for state-action pairs.
- How it is used
- Uses action-value estimates to train an agent to navigate reward-based Gym environments.
- How deep it goes
- Walks through a runnable Q-table implementation, exploration, update rule, training loop, and evaluation before deferring complex environments to DQN.
- Examples
- Demonstrates a 4-by-4 Frozen Lake agent and relates deep Q-learning to CartPole, Atari, and Lunar Lander.
- What you can do after
- Build it from scratch as a Gym-based Q-table agent with epsilon exploration, reward updates, and greedy evaluation.
Read how Evolutionary Deep Learning: Genetic algorithms and neural networks explains Q-learning →
Grokking AI Algorithms, Second Edition
Q-learning is a reinforcement-learning algorithm that learns action values for states without requiring a model of the environment's transition dynamics.
- How it is used
- Uses state–action scores to guide an agent toward favorable actions while it learns from simulated outcomes.
- How deep it goes
- Explains the Q-table and action-selection cycle, then outlines repeated simulation in Python without developing a theoretical proof.
- Examples
- Uses parking-lot positions and moves, Q-table figures, update illustrations, and a Python simulation listing.
- What you can do after
- Build it from scratch using a state–action table, simulated moves, rewards, and a choice between highest-scoring and random actions.
Read how Grokking AI Algorithms, Second Edition explains Q-learning →
Build a Robo-Advisor with Python (From Scratch)
Q-learning is a model-free reinforcement-learning algorithm that estimates the value of taking each action in each state and uses those estimates to derive a policy.
- How it is used
- Deploys repeated simulated episodes to learn asset-allocation policies for goals-based financial planning.
- How deep it goes
- Explains the state-action update and implements exploration, transitions, initialization, and episode generation.
- Examples
- Uses wealth states, time horizons, asset-allocation actions, simulated transitions, terminal goals, and repeated episodes.
- What you can do after
- Apply it via Python to estimate Q values and derive policies from simulated financial episodes.
Read how Build a Robo-Advisor with Python (From Scratch) explains Q-learning →
Related concepts
- reinforcement learningkind of · Reinforcement learning is a machine-learning paradigm in which an agent learns a policy through interaction with an environment and evaluative feedback about its actions or outcomes.
- epsilon-greedyuses · Epsilon-greedy is an action-selection strategy that chooses a random action with probability epsilon and otherwise selects the currently highest-valued action.
- deep reinforcement learningcontrasts with · Deep reinforcement learning is reinforcement learning that uses deep neural networks to represent policies, value functions, or other decision-making components.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.