Data leakage occurs when a model or its evaluation uses information that would not legitimately be available when the model is used. In portfolio reinforcement learning, leakage can make a backtest dishonest. section 12.0 section 12.2
Prevent look-ahead by ordering each environment step so the agent’s allocation is chosen before the next period’s returns are applied. Evaluation should also guard against leakage, unfair comparisons, and misleading metrics. section 12.2 section 12.4
Listing 12.4 The step function, ordered to prevent look-ahead (summarized)
def step_target_weights(self, target_w, action_name="custom"):
target_w = np.asarray(target_w,
➥ dtype=np.float64).copy()
turnover = 0.5 * np.sum(np.abs(target_w - self.weights)) #A
transaction_cost = self.cfg.tx_cost_rate * turnover #B
next_ret = np.zeros(self.n_slots)
next_ret[: self.n_assets] = self.returns[self.day + 1] #C
gross_return = float(np.dot(target_w, next_ret)) #D
net_return = gross_return - transaction_cost
self.value *= (1.0 + net_return)
self.peak_value = max(self.peak_value, self.value)
drawdown = self.value / self.peak_value - 1.0 #E
grown = target_w * (1.0 + next_ret)
denom = grown.sum()
self.weights = grown / denom if denom >
➥ 1e-12 else target_w
port_vol = float(np.std(self.returns_history)) if
➥ len(self.returns_history) >= 2 else 0.0
reward = (gross_return
- transaction_cost
- self.cfg.turnover_penalty_coef * turnover
- self.cfg.drawdown_penalty_coef * (-drawdown)
- self.cfg.vol_penalty_coef * port_vol) *
➥ self.cfg.reward_scale #F
self.day += 1 #G
self.step_count += 1
done = self.step_count >= self.cfg.horizon
return self._get_state(), float(reward), done, infoInspect the summarized step function’s ordering for an example of preventing look-ahead. section 12.4











