Reward hacking occurs when an optimizing system exploits weaknesses in its scoring objective, achieving a high score without satisfying the intended goal. In generative chemistry, a model may propose molecules that score well through weak property predictors but are unstable, impossible to synthesize, or unrealistic. section 1.3.2 section 10.1.3
The risk is especially clear in reinforcement learning: the agent improves whatever reward it receives. If that reward comes from a learned property predictor, optimization can target the predictor’s blind spots rather than genuine molecular quality. section 10.1.3
Reduce reward-hacking risk by combining objectives with practical safeguards:
- Balance potency, solubility, selectivity, synthesizability, and safety instead of optimizing one predicted property. section 10.1.3
- Add synthetic-accessibility or retrosynthesis filters, novelty and diversity terms, undesirable-substructure penalties, and scaffold-similarity constraints where appropriate. section 10.1.3
- Treat high predicted scores as an in-silico shortlist, then validate proposed molecules through the design-make-test cycle. section 10.1.3
Targeted molecular generation
A latent-space reinforcement-learning system encodes an input molecule, navigates toward regions associated with properties such as activity and low toxicity, decodes new structures, and feeds their predicted properties back as rewards. Without safeguards, the agent can exploit predictor errors. section 10.1.3




