liveBook · Manning

Uncertainty

As explained in Build AI Drug Discovery Pipelines

Uncertainty is a quantitative or qualitative representation of limited confidence in a model prediction.

Uncertainty represents limited confidence in a model prediction. In virtual screening, it helps identify molecules whose predictions may provide especially valuable information for improving the model. section 9.2.5

Common ways to estimate predictive uncertainty include:

  • Predictive variance, where probabilistic models such as Gaussian processes quantify uncertainty directly. section 9.2.5 section 9.2.2
  • Ensemble disagreement, where differing model predictions indicate less confidence. section 9.2.5
  • Monte Carlo dropout, which samples neural-network predictions with dropout enabled during inference. section 9.2.5
  • Distance to a classification decision boundary, where nearby points represent uncertain cases. section 9.2.5

Uncertainty-guided molecule selection

An active-learning workflow can prioritize unlabeled molecules with novel structures or predictions near the active–inactive boundary, then obtain expensive docking or experimental labels for those molecules. section 9.2.5

Listing 9.8 Greedy Acquisition and Uncertainty Sampling
def greedy_acquisition(model, X_pool, device, batch_size=1):
    predictions = model.predict(X_pool, device)
    best_indices = np.argsort(predictions)[:batch_size]
    return best_indices
 
def uncertainty_sampling(
    model, X_pool, device, batch_size=1, n_samples=100
):
    model.train()    #A
    all_predictions = np.zeros((n_samples, len(X_pool)))
    
    for i in range(n_samples):    #B
        inputs = torch.tensor(X_pool, dtype=torch.float32).to(device)
        
        with torch.no_grad():    #C
            outputs = model(inputs)            
        all_predictions[i, :] = outputs.cpu().numpy().flatten()
    
    uncertainties = np.std(all_predictions, axis=0)    #D
    best_indices = np.argsort(
        uncertainties
    )[::-1][:batch_size]    #E
    return best_indices
 
def select_next_molecule(
    model, X_pool, smiles_pool, device, best_score, batch_size=1, 
    acquisition_function="greedy", xi=0.01
):    #F
    if acquisition_function == "greedy":
        best_indices = greedy_acquisition(model, X_pool, device, batch_size)
    elif acquisition_function == "uncertainty":
        best_indices = uncertainty_sampling(model, X_pool, device)
    elif acquisition_function == "pi":
        best_indices = probability_of_improvement(
            model, X_pool, device, best_score, xi
        )
    elif acquisition_function == "ei":
        best_indices = expected_improvement(
            model, X_pool, device, best_score, xi
        )
    else:
        raise ValueError(f"Unknown acquisition function: {acquisition_function}")
 
    best_smiles = [smiles_pool[i] for i in best_indices]
    return best_indices, best_smiles

The listing estimates uncertainty by collecting repeated stochastic neural-network predictions, computing their standard deviation, and selecting the most uncertain pool molecules. section 9.2.5

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Defines several estimation strategies and demonstrates stochastic sampling, but defers formal Gaussian-process theory.
Examples
Uses predictive variance, ensemble disagreement, dropout sampling, decision boundaries, and repeated neural-network predictions over a molecule pool.
What it assumes
Assumes basic model predictions and unlabeled data, while introducing predictive variance, ensemble disagreement, dropout sampling, and decision boundaries.
What you can do after
Apply it via uncertainty sampling to select molecules for expensive docking or experimental labeling.
How it connects
Connects uncertainty with active learning, acquisition functions, Gaussian processes, ensemble methods, Monte Carlo dropout, and virtual screening.

Also explained in

Compare how 17 books explain uncertainty →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.