liveBook · Manning

Radiotherapy treatment planning

As explained in Applied Reinforcement Learning

Radiotherapy treatment planning is the process of selecting radiation delivery parameters over treatment sessions to control a tumor while limiting exposure to healthy tissue.

Radiotherapy treatment planning balances tumor control against radiation exposure to healthy organs. Treatment is typically divided into 20 to 40 sessions, or fractions. Dose accumulates, and the tumor may respond between sessions, making each beam and intensity decision relevant to later choices. section 8.4

A reinforcement learning simulation represents each decision using cumulative doses, remaining tumor dose, organ dose margins, tumor size, and treatment progress. An action selects a target and intensity. The reward favors progress toward the tumor dose target while penalizing organ-limit violations and tumor size. section 8.4

A simulated treatment

One environment models 30 fractions, five organs, and three intensity levels. It adds dose spill to neighboring organs and noise to delivered doses, then updates tumor size according to the dose received. A Double DQN agent learns decisions within this simulated setting; the example is not evidence of clinical effectiveness. section 8.4

Listing 8.5 Computing the reward signal
def _compute_reward(
    self,
    dose_change: np.ndarray,
    done: bool,
    success: bool,
    failure: bool,
    prev_tumor_dose: float,
):
    tumor_dose = self.cum_doses[0]
    oar_doses = self.cum_doses[1:]
    oar_limits = self.organ_limits[1:]
    oar_risk = self.organ_risk_weights[1:]
 
    prev_error = np.abs(prev_tumor_dose - self.target_dose)  #A
    curr_error = np.abs(tumor_dose - self.target_dose)  #A
    tumor_error = curr_error / self.target_dose  #B
    tumor_reward = -1.5 * tumor_error  #B
 
    if curr_error < prev_error:  #C
        direction_bonus = 0.3  #C
    else:
        direction_bonus = -0.15  #C
 
    violations = np.maximum(0.0, oar_doses - oar_limits)  #D
    weighted_violations = violations * oar_risk  #E
    organ_penalty = -0.3 * np.sum(weighted_violations)  #E
 
    tumor_size_penalty = -0.4 * self.tumor_size  #F
 
    step_penalty = -self.step_cost  #G
 
    reward = (
        tumor_reward
        + direction_bonus
        + organ_penalty
        + tumor_size_penalty
        + step_penalty
    )  #H
 
    if done:
        if success:
            reward += 200.0  #I
        if failure:
            reward -= 200.0  #J
 
    return float(reward)

The reward calculation makes the trade-off between tumor dose, organ exposure, and tumor size explicit. section 8.4

How Applied Reinforcement Learning covers it

How deep it goes
Works through a simulated treatment environment and Double DQN implementation, including state, actions, rewards, success checks, and training.
Examples
Uses a 30-fraction simulation with five organs, three intensity levels, dose spill, and changing tumor size.
What it assumes
Assumes prior understanding of DQN theory, stability, and limitations before applying it to treatment planning.
What you can do after
Builds a simulated treatment-planning environment and Double DQN agent from scratch.
How it connects
Connects fractionated radiotherapy with reinforcement learning, dose constraints, reward design, and Double DQN.

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.