concept human label in category machine learning
This is an excerpt from Manning's book Human-in-the-Loop Machine Learning MEAP V09.
Supervised learning models almost always get more accurate with more labelled data. Active Learning is the process of selecting which data needs to get a human label. Most research papers on Active Learning have focused on the number of training items. But the speed can be an even more important factor in many cases. Working in disaster response, I have often deployed Machine Learning models to filter and extract information from emerging disasters. Any delay in disaster response is potentially critical, so getting a usable model out quickly is more important than the number of labels that need to go into that model.
Figure 3.6: showing how Uncertainty Sampling as an Active Learning strategy, which over-samples unlabeled items that are closer to the decision boundary (and sometimes to each other), and are therefore more likely to get a human label that results in a change in that decision boundary.
