Frozen lake problem

As explained in Evolutionary Deep Learning: Genetic algorithms and neural networks

A frozen lake environment is a discrete grid-world environment containing traversable frozen squares, terminal holes, and a goal location.

The frozen lake problem is a discrete 4-by-4 grid-world in which an agent navigates safe frozen squares, avoids terminal holes, and reaches a rewarded goal. section 11.1.1

Figure 11.2 Frozen lake environment

Use the environment diagram to inspect the grid layout, safe squares, holes, and goal location. section 11.1.1

A Q-learning agent solves the environment by:

• Selecting up, down, left, or right from each state. section 11.1.1
• Exploring through trial and error while estimating action quality in a Q-table. section 11.1.1
• Updating values from rewards and future-state estimates, then reducing random exploration over 20,000 episodes. section 11.1.1

Minimal Gym implementation

Create FrozenLake-v0 with Gym, derive its action and state counts, initialize Q to zeros, train with choose_action, env.step, and learn, then exercise the greedy policy for five episodes. section 11.1.1

How Evolutionary Deep Learning: Genetic algorithms and neural networks covers it

How deep it goes

Explains grid rules, Q-table updates, exploration decay, training, and evaluation without deriving Q-learning theoretically.

Examples

Uses a 4-by-4 lake, terminal holes, a bottom-right goal, Gym listings, Q-table output, and five evaluation episodes.

What you can do after

Apply Q-learning via OpenAI Gym to train and evaluate an agent on a discrete navigation problem.

How it connects

Connects reinforcement learning with Q-learning, Q-tables, reward signals, exploration, and OpenAI Gym.

liveBookmanning.com

Read Evolutionary Deep Learning: Genetic algorithms and neural networks on liveBook Get the book at manning.com

Related concepts

• OpenAI Gymused by · OpenAI Gym is an open-source toolkit providing standardized interfaces and benchmark environments for developing and evaluating reinforcement-learning agents.

This explainer was generated by Manning's knowledge graph from the text of Evolutionary Deep Learning: Genetic algorithms and neural networks. The section links point to the parts of the book it draws on.