Action-value function

As explained in Evolutionary Deep Learning: Genetic algorithms and neural networks

A Q-function maps a state and action to an estimate of the expected cumulative reward from taking that action and following a policy thereafter.

How Evolutionary Deep Learning: Genetic algorithms and neural networks covers it

How deep it goes

Explains table representation and update mechanics, but deliberately stops short of a deeper Q-learning treatment.

Examples

Uses a Gym FrozenLake environment, a 16-by-4 Q-table, training episodes, and greedy simulated episodes.

What it assumes

Assumes discrete states, actions, rewards, and environment stepping while making the Q-table and training loop explicit.

What you can do after

Build it from scratch as a tabular Q-learning agent and evaluate its learned actions in an environment.

How it connects

Connects reinforcement learning with Q-learning, Q-tables, policies, rewards, discount factors, and exploration–exploitation.

liveBookmanning.com

Read Evolutionary Deep Learning: Genetic algorithms and neural networks on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA Q-function maps a state-action pair to the expected cumulative reward obtained by taking the action and subsequently following a policy.
• 

  liveBookmanning.com
  Optimization Algorithms: AI techniques for design, planning, and control problemsAn action-value function estimates the expected cumulative future reward from taking a specified action in a state and then following a policy.
• 

  liveBookmanning.com
  Build a Robo-Advisor with Python (From Scratch)A Q-function maps a state and an action to the estimated cumulative reward obtained by taking that action and following a policy afterward.
• 

  liveBookmanning.com
  Reinforcement Learning from Human FeedbackA Q-function maps a state-action pair to a scalar estimate of expected future reward.

Compare how 5 books explain action-value function →

Related concepts

• Q-learningused by · Q-learning is a model-free reinforcement-learning algorithm that estimates state-action values through updates based on rewards and subsequent state values.
• value functionkind of · A value function assigns or estimates the expected cumulative reward, utility, or quality associated with a state or action under a policy or decision rule.
• Bellman equationkind of · A Bellman equation recursively relates a state or action value to an immediate reward and the expected discounted value of successor states or actions.
• Q-tablepart of · A Q-table is a lookup table storing estimated action values for state-action pairs in a tabular reinforcement-learning system.
• advantage functionused by · An advantage function in reinforcement learning estimates how much better or worse an action’s expected return is than a baseline value for the corresponding state or policy.

This explainer was generated by Manning's knowledge graph from the text of Evolutionary Deep Learning: Genetic algorithms and neural networks. The section links point to the parts of the book it draws on.