liveBook · Manning

Data leakage

As explained in Applied Reinforcement Learning

The unintended use of information in a model or evaluation that would not legitimately be available when the model is used.

Data leakage occurs when a model or its evaluation uses information that would not legitimately be available when the model is used. In portfolio reinforcement learning, leakage can make a backtest dishonest. section 12.0 section 12.2

Prevent look-ahead by ordering each environment step so the agent’s allocation is chosen before the next period’s returns are applied. Evaluation should also guard against leakage, unfair comparisons, and misleading metrics. section 12.2 section 12.4

Listing 12.4 The step function, ordered to prevent look-ahead (summarized)
def step_target_weights(self, target_w, action_name="custom"):
    target_w = np.asarray(target_w,
    ➥ dtype=np.float64).copy()
 
    turnover = 0.5 * np.sum(np.abs(target_w - self.weights))  #A
    transaction_cost = self.cfg.tx_cost_rate * turnover  #B
 
    next_ret = np.zeros(self.n_slots)
    next_ret[: self.n_assets] = self.returns[self.day + 1]  #C
    gross_return = float(np.dot(target_w, next_ret))  #D
 
    net_return = gross_return - transaction_cost
    self.value *= (1.0 + net_return)
    self.peak_value = max(self.peak_value, self.value)
    drawdown = self.value / self.peak_value - 1.0  #E
    
    grown = target_w * (1.0 + next_ret)
    denom = grown.sum()
    self.weights = grown / denom if denom >
    ➥ 1e-12 else target_w
 
    port_vol = float(np.std(self.returns_history)) if  
    ➥ len(self.returns_history) >= 2 else 0.0
    reward = (gross_return
              - transaction_cost
              - self.cfg.turnover_penalty_coef * turnover
              - self.cfg.drawdown_penalty_coef * (-drawdown)
              - self.cfg.vol_penalty_coef * port_vol) * 
              ➥ self.cfg.reward_scale  #F
 
    self.day += 1  #G
    self.step_count += 1
    done = self.step_count >= self.cfg.horizon
    return self._get_state(), float(reward), done, info

Inspect the summarized step function’s ordering for an example of preventing look-ahead. section 12.4

How Applied Reinforcement Learning covers it

How deep it goes
Shows leakage as a systems risk and gives an ordered environment step to prevent look-ahead, without detailing broader leakage audits.
Examples
Uses a portfolio environment that applies next-period returns after target allocations are set.
What you can do after
Diagnose/debug with it by checking that an environment applies returns only after the agent selects its allocation.
How it connects
Connects data leakage to backtesting, evaluation, and environment design.

Also explained in

Compare how 12 books explain data leakage →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.