liveBook · Manning

Exploration-exploitation trade-off

As explained in Build AI Drug Discovery Pipelines

The exploration-exploitation trade-off is the choice between gathering information from unfamiliar regions and favoring options already predicted to perform well.

The exploration–exploitation trade-off guides active-learning decisions in drug discovery. Exploration tests unfamiliar chemical regions, while exploitation favors molecules already predicted to have strong binding affinity. Balancing both can reveal novel active compounds without ignoring promising molecular families. section 9.2.5

Common acquisition strategies make the balance computational:

  • Greedy acquisition selects molecules with the best predicted binding affinity, emphasizing exploitation. section 9.2.5
  • Uncertainty sampling selects molecules where the model is least certain, emphasizing exploration. section 9.2.5
  • Monte Carlo dropout estimates neural-network uncertainty by collecting predictions from repeated inference passes and measuring their standard deviation. section 9.2.5
Listing 9.8 Greedy Acquisition and Uncertainty Sampling
def greedy_acquisition(model, X_pool, device, batch_size=1):
    predictions = model.predict(X_pool, device)
    best_indices = np.argsort(predictions)[:batch_size]
    return best_indices
 
def uncertainty_sampling(
    model, X_pool, device, batch_size=1, n_samples=100
):
    model.train()    #A
    all_predictions = np.zeros((n_samples, len(X_pool)))
    
    for i in range(n_samples):    #B
        inputs = torch.tensor(X_pool, dtype=torch.float32).to(device)
        
        with torch.no_grad():    #C
            outputs = model(inputs)            
        all_predictions[i, :] = outputs.cpu().numpy().flatten()
    
    uncertainties = np.std(all_predictions, axis=0)    #D
    best_indices = np.argsort(
        uncertainties
    )[::-1][:batch_size]    #E
    return best_indices
 
def select_next_molecule(
    model, X_pool, smiles_pool, device, best_score, batch_size=1, 
    acquisition_function="greedy", xi=0.01
):    #F
    if acquisition_function == "greedy":
        best_indices = greedy_acquisition(model, X_pool, device, batch_size)
    elif acquisition_function == "uncertainty":
        best_indices = uncertainty_sampling(model, X_pool, device)
    elif acquisition_function == "pi":
        best_indices = probability_of_improvement(
            model, X_pool, device, best_score, xi
        )
    elif acquisition_function == "ei":
        best_indices = expected_improvement(
            model, X_pool, device, best_score, xi
        )
    else:
        raise ValueError(f"Unknown acquisition function: {acquisition_function}")
 
    best_smiles = [smiles_pool[i] for i in best_indices]
    return best_indices, best_smiles

Runnable Python functions compare greedy acquisition with uncertainty sampling and select the next molecular batch from a candidate pool. section 9.2.5

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Defines exploration and exploitation, then implements greedy and uncertainty-based acquisition without detailing PI or EI.
Examples
Uses apple varieties as an analogy and molecular pools, predicted affinity, and dropout predictions as computational examples.
What it assumes
Assumes familiarity with model predictions and molecular pools while defining exploration, exploitation, and epistemic uncertainty.
What you can do after
Apply it via Python acquisition functions to select promising or uncertain molecules for the next batch.
How it connects
Connects active learning, acquisition functions, uncertainty sampling, Monte Carlo dropout, and chemical space exploration.

Also explained in

Compare how 6 books explain exploration-exploitation trade-off →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.