Radiotherapy treats cancer with focused high-energy radiation that damages cancer-cell DNA, preventing division and eventually causing cell death. The challenge is to deliver enough dose to control the tumor while limiting exposure to healthy organs at risk, such as the heart or lungs. section 8.4
In fractionated radiotherapy, the total dose is divided into smaller doses called fractions, typically delivered over 20–40 sessions. Beam aim and intensity are sequential decisions: dose accumulates, and the tumor may shrink or grow as treatment progresses. section 8.4
Treatment planning as a simulation
A simplified treatment environment represents cumulative doses, remaining tumor dose, organ safety margins, tumor size, and treatment progress. Actions select a target and intensity; the reward favors progress toward the tumor dose while penalizing organ-limit violations and tumor size. A deep Q-network can learn decisions within this simulated setting. section 8.4
Listing 8.5 Computing the reward signal
def _compute_reward(
self,
dose_change: np.ndarray,
done: bool,
success: bool,
failure: bool,
prev_tumor_dose: float,
):
tumor_dose = self.cum_doses[0]
oar_doses = self.cum_doses[1:]
oar_limits = self.organ_limits[1:]
oar_risk = self.organ_risk_weights[1:]
prev_error = np.abs(prev_tumor_dose - self.target_dose) #A
curr_error = np.abs(tumor_dose - self.target_dose) #A
tumor_error = curr_error / self.target_dose #B
tumor_reward = -1.5 * tumor_error #B
if curr_error < prev_error: #C
direction_bonus = 0.3 #C
else:
direction_bonus = -0.15 #C
violations = np.maximum(0.0, oar_doses - oar_limits) #D
weighted_violations = violations * oar_risk #E
organ_penalty = -0.3 * np.sum(weighted_violations) #E
tumor_size_penalty = -0.4 * self.tumor_size #F
step_penalty = -self.step_cost #G
reward = (
tumor_reward
+ direction_bonus
+ organ_penalty
+ tumor_size_penalty
+ step_penalty
) #H
if done:
if success:
reward += 200.0 #I
if failure:
reward -= 200.0 #J
return float(reward)
The reward calculation makes the tumor-dose and organ-safety trade-off explicit. section 8.4
