liveBook · Manning

Reinforcement learning

As explained in Build AI Drug Discovery Pipelines

Reinforcement learning is a machine-learning approach in which an agent learns sequential actions through feedback from rewards or penalties.

Reinforcement learning (RL) trains an agent to make sequential decisions by rewarding useful outcomes and penalizing poor ones. In molecular design, the agent explores a generative model’s latent space and learns to favor molecules with desired properties. section 10.1.3

A targeted molecular-generation workflow typically combines:

  • A pretrained generative model that encodes molecules into continuous latent vectors and decodes optimized vectors into new structures. section 10.1.3
  • An agent whose actions move through latent space or select the next token. section 10.1.3
  • A reward function that scores generated molecules using objectives such as binding affinity, drug-likeness, selectivity, solubility, CYP inhibition, and synthetic accessibility. section 10.1.3 section 13.3.1
  • Repeated exploration and policy updates that make high-reward choices more likely, while requiring checks for reward hacking. section 10.1.3

Rentosertib molecular design

For the TNIK program, Chemistry42 used more than 40 generative models in parallel. GENTRL combined a variational autoencoder with RL: the agent explored chemical latent space while property predictors supplied rewards. Early weighting favored TNIK affinity but produced poorly soluble or impractical molecules; later cycles increased emphasis on drug-like properties. section 13.3.1

Figure 10.1 Overview of targeted molecular generation with latent reinforcement learning. The system begins with an input molecule that is encoded into a compressed latent vector representation. An RL agent operates within this latent space, proposing optimization actions to navigate toward regions associated with desired molecular properties (such as "Active" and "Non-toxic" regions shown). The decoder then reconstructs these optimized latent vectors back into novel molecular structures. Rewards are calculated based on the properties of the generated molecules and fed back to improve the RL agent's decision-making. This approach enables continuous optimization in latent space rather than discrete structural modifications, allowing guided generation of molecules with specific target properties.

Trace the loop from input-molecule encoding through latent-space actions, molecular decoding, property-based rewards, and agent improvement. section 10.1.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains agents, actions, rewards, latent-space exploration, and policy updates, while deferring formal RL development to an appendix.
Examples
Demonstrates latent molecular optimization through MOLRL, VAE-CYC, GENTRL, and the TNIK-focused rentosertib program.
What it assumes
Assumes a pretrained generative model, a molecular latent space, and property predictors; defines agent, action, environment, and reward through the application.
What you can do after
Apply reinforcement learning to configure rewards and optimize generated molecules against multiple predicted properties.
How it connects
Connects reinforcement learning with generative models, variational autoencoders, latent spaces, policy gradients, and multi-objective optimization.

Also explained in

Compare how 28 books explain reinforcement learning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.