liveBook · Manning

Concordance index

As explained in Build AI Drug Discovery Pipelines

The concordance index is a metric that measures how often a model correctly orders pairs of examples according to their true values, with tied predictions receiving half credit.

The concordance index (CI) measures ranking quality: it checks how often predicted values order pairs of examples in the same direction as their true values. In drug-target affinity prediction, this asks whether a model can identify which compounds bind more strongly, even when absolute errors are large. section 11.6.1

For every comparable pair with different true affinities, CI awards one point for a correctly ordered pair, zero for an incorrect order, and half a point for tied predictions: CI = (concordant pairs + 0.5 × tied pairs) / total comparable pairs. A score of 1.0 is perfect ranking, 0.5 is chance-level ranking, and scores above 0.8 are described as excellent. section 11.6.1

Use CI alongside regression metrics rather than treating it as a replacement:

  • MSE and RMSE measure prediction accuracy, whereas CI measures whether candidate ordering is useful for prioritizing experiments. section 11.6.1
  • CI relates to Spearman's rank correlation and Kendall's tau, but focuses directly on pairwise comparisons. section 11.6.1
  • Tied predictions receive 0.5 credit, so predicting the same value for every example produces chance-level CI rather than an artificially high score. section 11.6.1

Drug-target affinity evaluation

For DAVIS and KIBA affinity benchmarks, CI evaluates whether predicted affinities preserve the ordering of drug-target pairs. Reported DGraphDTA results show CI = 0.904 on both datasets, alongside MSE and Pearson correlation, supporting use in virtual-screening prioritization. section 11.1.1 section 11.6.2

def concordance_index(y_true, y_pred):
    concordant = 0
    total = 0
    for i in range(len(y_true) - 1):
        for j in range(i + 1, len(y_true)):
            if y_true[i] != y_true[j]:
                total += 1
                if (y_true[i] < y_true[j] and y_pred[i] < y_pred[j]) or \
                   (y_true[i] > y_true[j] and y_pred[i] > y_pred[j]):
                    concordant += 1
                elif y_pred[i] == y_pred[j]:
                    concordant += 0.5
    return concordant / total

Inspect a direct Python implementation that enumerates comparable pairs, checks ordering, and assigns half credit to tied predictions. section 11.6.1

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Defines pairwise scoring, tie handling, interpretation thresholds, and a Python implementation, while deferring direct ranking optimization difficulties.
Examples
Uses DAVIS and KIBA drug-target affinity benchmarks, GNN comparisons, DGraphDTA results, and a nested-loop Python implementation.
What it assumes
Assumes paired true and predicted values, while defining ranking quality, comparable pairs, ties, and interpretation from zero.
What you can do after
Evaluate model rankings with CI and apply the result to prioritize stronger-binding compounds for virtual screening.
How it connects
Connects ranking quality with MSE, RMSE, Pearson correlation, Spearman's rank correlation, Kendall's tau, and virtual screening.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.