An instinct function is a reusable neural-network function or layer that can be placed in models for different tasks or environments. In instinctual deep reinforcement learning, several DQN networks share a pool of middle-layer instincts while retaining environment-specific input and output layers. section 12.6
An instinctual model is assembled by:
- Splitting the network into functional chunks with the Keras functional API. section 12.6
- Keeping the first and last layers specific to each environment while selecting reusable middle layers from an instinct pool. section 12.6
Instinctual DQN
A model receives an environment-specific input layer, applies three selected dense instincts, concatenates their outputs, and produces actions through an environment-specific output layer.
Listing 12.32 EDL_12_6_Instinctual_DQN_GYMS.ipynb: Building the model
def _build_model(self, instincts):
inputs = k.Input(shape=(self.state_size,)) #1
dense_in = instincts[0] #2
dense1 = instincts[1]
dense2 = instincts[2] #2
dense3 = instincts[3]
dense_out = instincts[4] #2
x = dense_in(inputs)
x1 = dense1(x) #3
x2 = dense2(x)
x3 = dense3(x)
x = kl.concatenate([x1, x2, x3]) #4
outputs = dense_out(x)
model = k.Model(inputs=inputs, outputs=outputs) #5
model.compile(loss='mse', optimizer=k.optimizers.Adam(learning_rate=self.learning_rate))
return model
Inspect the functional model-building pattern that turns selected layers into reusable instinct slots. section 12.6
