Uncertainty sampling
Uncertainty sampling is an active-learning strategy that selects unlabeled instances about which a model is most uncertain.
Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI
A method of selecting unlabeled items near a model’s decision boundary, where its predictions are uncertain.
- How it is used
- Prioritizes unlabeled items for human review in an iterative active learning workflow.
- How deep it goes
- Explains scoring methods, iterative selection, batch-size tradeoffs, and evaluation, with extensions to different models and tasks.
- Examples
- Uses self-driving object recognition and semantic segmentation to illustrate uncertainty scores and human review.
- What you can do after
- Apply uncertainty sampling by ranking predictions, requesting labels, retraining iteratively, and evaluating on a random held-out set.
Build AI Drug Discovery Pipelines
Uncertainty sampling is an active-learning acquisition strategy that selects examples for which the current model is most uncertain.
- How it is used
- Prioritizes molecules for expensive docking when virtual screening cannot evaluate an ultra-large compound library exhaustively.
- How deep it goes
- Explains uncertainty measures, iterative acquisition, diversity trade-offs, and a PyTorch implementation, while deferring model-specific theory.
- Examples
- Demonstrates prediction standard deviation, 4AG8 docking, PCA chemical-space plots, TensorBoard metrics, and top-100 molecule capture.
- What you can do after
- Apply it via a surrogate model to select informative molecules and monitor candidate recovery across active-learning iterations.
Read how Build AI Drug Discovery Pipelines explains uncertainty sampling →
Machine Learning Algorithms in Depth
Uncertainty sampling is an active learning strategy that selects instances for which the current model is most uncertain about the predicted label.
- How it is used
- Selects which unlabeled examples to query in active learning to reduce the number of training labels needed.
- How deep it goes
- Defines several uncertainty criteria, compares their multiclass limitations, and provides an active-learner implementation.
- Examples
- Compares 30 random labels with 30 entropy-selected queries on balanced synthetic binary classification, yielding 90% and 92.5% held-out accuracy.
- What you can do after
- Build it from scratch: rank unlabeled candidates by a selected uncertainty criterion and request labels for the chosen queries.
Read how Machine Learning Algorithms in Depth explains uncertainty sampling →
What to know first
- labelerA labeler is a person or process that assigns labels, ratings, judgments, or other structured annotations to data or model outputs.
- neural networkA neural network is a computational model composed of interconnected, parameterized units or transformations that learn to map inputs to outputs.
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- diversity-based samplingDiversity-based sampling selects examples to represent variation or cover underrepresented regions of a candidate set or feature space.
Related concepts
- ATLASincludes · An active transfer learning method for adaptive sampling that combines uncertainty and diversity considerations.
- decision treecontrasts with · A decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.
- expected error reduction samplingused by · An active learning strategy that selects items expected to produce the greatest reduction in model error when labeled.
- representative samplinguses · A sampling approach that seeks a sample reflecting relevant characteristics of the population.
- active learningkind of · Active learning is a machine-learning approach in which a learner selects informative data examples for labeling or feedback to improve a model.
- active learning strategykind of · A method for selecting items to label during active learning.
Questions about uncertainty sampling
What is uncertainty sampling?
Uncertainty sampling is an active-learning strategy that selects unlabeled instances about which a model is most uncertain.
Which Manning books explain uncertainty sampling?
Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI, Build AI Drug Discovery Pipelines and Machine Learning Algorithms in Depth.
What should I know before learning uncertainty sampling?
Manning books that teach uncertainty sampling usually introduce labeler, neural network, accuracy and diversity-based sampling first.
What is the difference between uncertainty sampling and decision tree?
Uncertainty sampling is an active-learning strategy that selects unlabeled instances about which a model is most uncertain. A decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.
What kinds of uncertainty sampling are there?
Kinds of uncertainty sampling include ATLAS.
What is uncertainty sampling a kind of?
Uncertainty sampling is a kind of active learning. Active learning is a machine-learning approach in which a learner selects informative data examples for labeling or feedback to improve a model.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


