In reinforcement learning, an environment is the setting an agent acts in and receives feedback from. Think of a dog learning to sit at home: a spoken cue is part of its surroundings, and a treat rewards the action. For a learning agent, the environment similarly provides circumstances in which actions have consequences. section 10.1
An environment matters when the goal is known but the best sequence of actions is not. The agent tries actions, observes rewards or penalties, and learns which choices help over time. A single action need not look best immediately if it helps achieve a larger goal. section 10.3
A parking-lot environment
Picture a self-driving car trying to reach its owner while avoiding parked cars and pedestrians. The car is the agent; the parking lot and its obstacles form the environment. It can move north, south, east, or west, one block at a time. Each move changes its position and has an outcome: reaching the owner is desirable, while collisions bring penalties. Repeated attempts let the agent compare routes, not just individual moves. section 10.3 section 10.5
To train without driving a real car, use a simulator that models the environment separately from the agent learning what to do. It must reset to a starting state, report the current state, apply an action, and calculate the resulting reward. This makes it possible to practice and measure progress toward the goal. section 10.4 section 10.5
class Simulator:
def __init__(self, road, road_size_x, road_size_y,
➥agent_start_x, agent_start_y):
self.road = road
self.road_size_x = road_size_x
self.road_size_y = road_size_y
self.agent_x = agent_start_x
self.agent_y = agent_start_y #A
def move_agent(self, action):
next_x, next_y = self.agent_x, self.agent_y
if action == COMMAND_NORTH: #B
next_x -= 1
elif action == COMMAND_SOUTH:
next_x += 1
elif action == COMMAND_EAST:
next_y += 1
elif action == COMMAND_WEST:
next_y -= 1
reward_update = 0
if self.is_within_bounds(next_x, next_y): #C
reward_update = self.cost_movement(next_x, next_y) #D
self.agent_x, self.agent_y = next_x, next_y
else:
reward_update = ROAD_OUT_OF_BOUNDS_REWARD #E
return reward_update
The simulator listing shows a move changing the agent’s position and producing a reward, with a bounds check for moves beyond the road. section 10.5
The feedback rules need care. If moving between empty spaces repeatedly earns rewards, an agent might keep doing that instead of reaching the goal. A simulated environment is also a simplification: real settings can offer more actions and more varied consequences. section 10.5 section 10.4
A market sandbox
A trading agent can practice in a simulated market built from historical data. Its state includes market data, its holdings, and its cash balance; it chooses to buy, sell, or hold and receives feedback from changes in portfolio value. The sandbox allows practice without risking real money. section 10.15
