liveBook · Manning

Early recognition problem

As explained in Build AI Drug Discovery Pipelines

Early recognition is the ability of a ranking method to retrieve a large proportion of active compounds within a small initial fraction of a screened collection.

The early recognition problem asks whether a ranking method can place many active compounds near the top of a screened collection, so hit discovery requires screening only a small fraction of it. In virtual screening, a scoring function ranks compounds by expected activity; similarity searches, supervised models, neural networks, linear regressors, and molecular docking can provide such scores. section 8.1.3

Evaluate early recognition by following this workflow:

  • Score every compound and sort the collection from highest to lowest predicted activity. section 8.1.3
  • Choose an initial screened fraction, commonly 0.5%, 1%, 2%, or 5% of the collection. section 8.1.3
  • Count the active compounds within that top-ranked fraction and compare their proportion with random selection. section 8.1.3
  • Interpret an enrichment factor above 1 as better-than-random early recognition; larger values indicate greater concentration of actives near the top. section 8.1.3

A 1% screening fraction

For 1,000 compounds containing 100 known actives, random selection expects one active in the top 1% (10 compounds). Finding five actives there gives EF(1%) = 5, meaning the scoring function is five times as effective as random selection at that cutoff. section 8.1.3

EF(X%)=(Number of actives found in top X% of the ranked list)Total actives in dataset×(X%100)

Use the enrichment-factor equation to compare active recovery in the top X% with the active rate expected from random selection. section 8.1.3

Listing 8.4 Calculating Enrichment Factors
import math
 
def calculate_enrichment_factors(
    y_true, y_pred, activity_threshold=6.3,
    ef_percentages=[0.005, 0.01, 0.02, 0.05]
):
    y_true_binary = (y_true >= activity_threshold).astype(int)
    sorted_indices = np.argsort(y_pred)[::-1]
    n_actives = int(np.sum(y_true_binary))    #A
    n_total = len(y_true_binary)    #B
    if n_actives == 0 or n_total == 0:    #C
        raise ValueError(
            "Need at least one active and a non-empty dataset."
        )
    enrichment_factors = {}
    for percentage in ef_percentages:    #D
        n_compounds = max(1, math.ceil(    #E
            n_total * percentage))  #E
        top = sorted_indices[:n_compounds]
        n_actives_found = int(np.sum(y_true_binary[top]))
        ef_model = (    #F
            (n_actives_found / n_compounds)    #F
            / (n_actives / n_total)    #F
        )  #F
        ef_optimal = min(1 / percentage, n_total / n_actives)  #G
        ef_random = 1.0  #H
        enrichment_factors[f"EF{percentage*100:.1f}%"] = {
            "Model": ef_model,
            "Optimal": ef_optimal,
            "Random": ef_random
        }
    return enrichment_factors

The Python implementation converts activity values to binary labels, sorts predictions, evaluates selected percentages, and rejects empty or all-inactive datasets. section 8.1.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Defines the goal, derives enrichment-factor interpretation, demonstrates a numerical case, and provides an implementation while deferring model construction.
Examples
Uses a 1,000-compound collection with 100 actives, a five-active top-1% result, Python evaluation, and enrichment plots.
What it assumes
Assumes ranked compound scores and active labels, while defining the early-recognition goal and enrichment-factor interpretation explicitly.
What you can do after
Evaluate ranking quality with enrichment factors at specified screened fractions and compare performance with random selection.
How it connects
Connects virtual screening, scoring functions, supervised learning, neural networks, molecular docking, and enrichment factors.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.