liveBook · Manning

Uncertainty sampling

Uncertainty sampling is an active-learning strategy that selects unlabeled instances about which a model is most uncertain.

How 3 Manning books explain it

Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI

A method of selecting unlabeled items near a model’s decision boundary, where its predictions are uncertain.

How it is used
Prioritizes unlabeled items for human review in an iterative active learning workflow.
How deep it goes
Explains scoring methods, iterative selection, batch-size tradeoffs, and evaluation, with extensions to different models and tasks.
Examples
Uses self-driving object recognition and semantic segmentation to illustrate uncertainty scores and human review.
What you can do after
Apply uncertainty sampling by ranking predictions, requesting labels, retraining iteratively, and evaluating on a random held-out set.

Read how Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI explains uncertainty sampling →

Build AI Drug Discovery Pipelines

Uncertainty sampling is an active-learning acquisition strategy that selects examples for which the current model is most uncertain.

How it is used
Prioritizes molecules for expensive docking when virtual screening cannot evaluate an ultra-large compound library exhaustively.
How deep it goes
Explains uncertainty measures, iterative acquisition, diversity trade-offs, and a PyTorch implementation, while deferring model-specific theory.
Examples
Demonstrates prediction standard deviation, 4AG8 docking, PCA chemical-space plots, TensorBoard metrics, and top-100 molecule capture.
What you can do after
Apply it via a surrogate model to select informative molecules and monitor candidate recovery across active-learning iterations.

Read how Build AI Drug Discovery Pipelines explains uncertainty sampling →

Machine Learning Algorithms in Depth

Uncertainty sampling is an active learning strategy that selects instances for which the current model is most uncertain about the predicted label.

How it is used
Selects which unlabeled examples to query in active learning to reduce the number of training labels needed.
How deep it goes
Defines several uncertainty criteria, compares their multiclass limitations, and provides an active-learner implementation.
Examples
Compares 30 random labels with 30 entropy-selected queries on balanced synthetic binary classification, yielding 90% and 92.5% held-out accuracy.
What you can do after
Build it from scratch: rank unlabeled candidates by a selected uncertainty criterion and request labels for the chosen queries.

Read how Machine Learning Algorithms in Depth explains uncertainty sampling →

What to know first

Related concepts

Questions about uncertainty sampling

What is uncertainty sampling?

Uncertainty sampling is an active-learning strategy that selects unlabeled instances about which a model is most uncertain.

Which Manning books explain uncertainty sampling?

Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI, Build AI Drug Discovery Pipelines and Machine Learning Algorithms in Depth.

What should I know before learning uncertainty sampling?

Manning books that teach uncertainty sampling usually introduce labeler, neural network, accuracy and diversity-based sampling first.

What is the difference between uncertainty sampling and decision tree?

Uncertainty sampling is an active-learning strategy that selects unlabeled instances about which a model is most uncertain. A decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.

What kinds of uncertainty sampling are there?

Kinds of uncertainty sampling include ATLAS.

What is uncertainty sampling a kind of?

Uncertainty sampling is a kind of active learning. Active learning is a machine-learning approach in which a learner selects informative data examples for labeling or feedback to improve a model.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.