State space

As explained in Evolutionary Deep Learning: Genetic algorithms and neural networks

A state space is the set of states that describe the situations an agent can encounter in an environment.

A state space describes the possible situations an agent may encounter in an environment. In Gym, the environment exposes its observation/state space so an implementation can determine how state is represented and how large the problem is. FrozenLake uses a discrete state space: its size is the number of available discrete state values. Other environments use continuous or ranged values. section 11.1.1 section 11.2

State representation directly shapes the learning mechanism. For tabular Q-learning, the state-space size and action-space size define a Q-table: each row corresponds to a state, and each column records an action’s learned quality in that state. This is practical for a small discrete environment, but continuous spaces make a table unsuitable; deep reinforcement learning instead uses a neural network in place of the state or Q-table. section 11.1.1 section 11.2

Inspect a state space before choosing an agent design:

• Create the Gym environment and query observation_space and action_space to identify the available representation and action count. section 11.1.1 section 11.2
• Use a zero-filled array sized as (state_size, action_size) when states and actions are discrete and a tabular Q-learner is appropriate. section 11.1.1
• Treat more possible states as greater environment complexity; the illustrated comparison raises state-space size to the action-space size. section 11.2
• Pass observation size into a DQN when replacing a table with a network; its input accepts the state space and its output covers the action space. section 11.5

From FrozenLake table to LunarLander network

FrozenLake initializes one Q-table row per discrete state and selects values from that row for available actions. For LunarLander, the agent reads the observation-space shape, reshapes each state for training, and feeds it to a three-layer network whose final layer has one output per action. The contrast shows why state representation matters: table lookup fits the small discrete case, while a neural model handles the richer observation input. section 11.1.1 section 11.5

Figure 11.5 A comparison of Gym environments

Compare Gym environments by their action and state spaces and by the resulting relative-complexity estimate before selecting a tabular or deep-learning approach. section 11.2

How Evolutionary Deep Learning: Genetic algorithms and neural networks covers it

How deep it goes

Explains discrete and continuous representations through table sizing, complexity comparison, and neural-network replacement rather than formal state modeling.

Examples

Uses FrozenLake’s discrete Q-table plus CartPole, MountainCar, LunarLander, and a DQN LunarLander agent.

What it assumes

Assumes basic Python notebook use while providing explicit Gym setup and runnable environment inspection.

What you can do after

Apply it via OpenAI Gym by reading observation and action spaces to size a Q-table or DQN.

How it connects

Connects state spaces to action spaces, Q-learning, Q-tables, deep reinforcement learning, and deep Q-networks.

liveBookmanning.com

Read Evolutionary Deep Learning: Genetic algorithms and neural networks on liveBook Get the book at manning.com

Related concepts

• dynamic programmingused by · Dynamic programming is an algorithmic technique that solves problems by decomposing them into overlapping subproblems and storing or reusing their solutions.
• constraintconfigured by · A constraint is a condition or rule that restricts the permissible states, values, actions, or solutions in a system or problem.
• problem solvingused by · Problem solving is the process of analyzing a problem and developing, selecting, or applying a solution to address it.
• knapsack problempart of · The knapsack problem is an optimization problem that selects items subject to a capacity constraint to maximize their combined value.
• statehas part · State is information or a configuration describing the condition of a system, entity, or computation at a particular time and potentially influencing subsequent behavior or decisions.

This explainer was generated by Manning's knowledge graph from the text of Evolutionary Deep Learning: Genetic algorithms and neural networks. The section links point to the parts of the book it draws on.