Evaluation tests whether an agent’s apparent success reflects useful behavior or luck. In portfolio management, assess it on later, unseen market history, using consistent conditions and measures that account for risk—not return alone. section 12.7 section 12.9.5
A credible comparison should:
- Compare against challenging baselines, change one variable at a time, and aggregate results across many starting points or episodes. section 12.7 section 12.9.5
- Report measures such as Sharpe ratio, worst drawdown, and turnover rather than relying on bare returns. section 12.7 section 12.9.5
- Measure intermediate links in the causal chain: for example, compare each signal with an oracle using mean error and directional accuracy. section 12.7
When identical results are a warning
If agents given different signals produce byte-for-byte identical results, do not assume the system is reproducible and sound. Check that the signal inputs really differ, then investigate whether the policy is ignoring them. section 12.7




























