Reinforcement learning trains an agent to choose actions by observing an environment, receiving rewards or penalties, and maximizing cumulative reward over time. In cybersecurity, it supports automated incident response by learning when to quarantine machines, block traffic, or escalate alerts. section 12.6
A security response agent evaluates outcomes over repeated interactions:
- Blocking traffic after an attack succeeds can earn a reward and increase the likelihood of that action later. section 12.6
- False positives or delayed escalation create penalties, discouraging responses that increase operational risk. section 12.6
- The learned policy balances known effective actions with continued exploration of alternatives. section 12.6.1
Adaptive incident response
A Q-learning exercise uses normal, suspicious, and attack states with quarantine, block-traffic, and escalate actions. Its learned Q-table favors escalation in normal conditions, blocking traffic in suspicious conditions, and blocking traffic during attacks. section 12.6.1
Listing 12.6 Using Q-learning for automated incident response
import numpy as np #1
# Define states and actions
states = ['normal', 'suspicious', 'attack'] #2
actions = ['quarantine', 'block_traffic', 'escalate'] #3
# Initialize the Q-table
Q_table = np.zeros((len(states), len(actions))) #4
# Define the rewards for each action in each state
rewards = { #5
('normal', 'quarantine'): -10,
('normal', 'block_traffic'): -5,
('normal', 'escalate'): -1,
('suspicious', 'quarantine'): 5,
('suspicious', 'block_traffic'): 10,
('suspicious', 'escalate'): 1,
('attack', 'quarantine'): 20,
('attack', 'block_traffic'): 30,
('attack', 'escalate'): 15,
}
# Parameters for Q-learning
alpha = 0.1 #Learning rate #6
gamma = 0.9 #Discount factor #7
epsilon = 0.1 #Exploration rate #8
# Simulate the learning process
for episode in range(1000): #9
state = np.random.choice(states) #10
# Choose an action (explore or exploit)
if np.random.uniform(0, 1) < epsilon: #11
action = np.random.choice(actions) #12
else:
action = actions[np.argmax(Q_table[states.index(state)])] #13
# Get the reward and update Q-values
reward = rewards[(state, action)] #14
new_state = np.random.choice(states)
# Randomly simulate a new state #15
Q_table[states.index(state), actions.index(action)] +=
alpha * (reward + gamma * np.max(Q_table[
states.index(new_state)]) - Q_table[
states.index(state), actions.index(action)]) #16
# Print the learned Q-table
print("Learned Q-table:") #17
print(Q_table) #18
Inspect the Python listing to see states, actions, rewards, and Q-learning parameters assembled for automated incident response. section 12.6.1
