A step function applies an agent’s action to the current environment state, computes a reward, updates the state, and returns the next state, reward, episode status, and an empty information dictionary. The template refuses calls after the environment is done; its example marks the episode done after each step. section 3.1
Listing 3.1 Template for any reinforcement learning environment
class CustomEnv:
def __init__(self): #A
self.current_step = 0 #A
self.done = False #A
self.state = None #A
self.action_space = [0,1] #B
self.observation_space = np.zeros(3, dtype=np.float32) #B
def reset(self): #C
self.current_step = 0 #C
self.done = False #C
self.state = self._get_initial_state() #C
return self.state #C
def step(self, action):
if self.done: #D
raise Exception("Environment is done. Call reset().") #D
next_state = self._transition(self.state, action) #E
reward = self._compute_reward(self.state, action, next_state) #F
self.state = next_state #G
self.current_step += 1 #G
self.done = True #G
return next_state, reward, self.done, {} #H
def _get_initial_state(self): #I
raise NotImplementedError #I
def _transition(self, state, action): #J
raise NotImplementedError #J
def _compute_reward(self, state, action, next_state): #K
raise NotImplementedError #K
See the environment template for the step method’s transition, reward, state-update, and return sequence. section 3.1
