concept Active Learning in category machine learning
This is an excerpt from Manning's book Human-in-the-Loop Machine Learning MEAP V09.
Supervised learning models almost always get more accurate with more labelled data. Active Learning is the process of selecting which data needs to get a human label. Most research papers on Active Learning have focused on the number of training items. But the speed can be an even more important factor in many cases. Working in disaster response, I have often deployed Machine Learning models to filter and extract information from emerging disasters. Any delay in disaster response is potentially critical, so getting a usable model out quickly is more important than the number of labels that need to go into that model.
Just like there is no one algorithm, architecture, or set of parameters that will make one Machine Learning model more accurate in all cases, there is no one strategy for Active Learning that will be optimal across all use cases and data sets. But just like with Machine Learning models, there are some approaches that you should try first because they are more likely to work.
The last three chapters the examples and algorithms focused on document-level or image-level predictions. In this chapter, you will learn how the same principles of Uncertainty Sampling and Diversity Sampling can be straightforwardly applied to more complicated Computer Vision tasks like object detection and semantic segmentation (pixel labeling) and more complicated Natural Language Processing tasks like Named Entity Recognition and Natural Language Generation. The general principles are the same and in many cases there is no change needed at all. The biggest differences will be in how you decide to sample the items selected by Active Learning and that will depend on the real-world problem that you are trying to solve.
Figure 6.11: Three ways to encode predicted spans for Active Learning. Top, using one-hot encoding to encode each token as its own feature. Middle, using a non-contextual vector (embedding) like word2vec. Bottom, using a contextual embedding like BERT. You can also experiment with Average Pooling (Avepool) in place of or in addition to Maximum Pooling (Maxpool).
This is an excerpt from Manning's book Exploring Machine Learning Basics.
Supervised learning models almost always get more accurate with more labelled data. Active Learning is the process of selecting which data needs to get a human label. Most research papers on Active Learning have focused on the number of training items. But the speed can be an even more important factor in many cases. Working in disaster response, I have often deployed Machine Learning models to filter and extract information from emerging disasters. Any delay in disaster response is potentially critical, so getting a usable model out quickly is more important than the number of labels that need to go into that model.
Just like there is no one algorithm, architecture, or set of parameters that will make one Machine Learning model more accurate in all cases, there is no one strategy for Active Learning that will be optimal across all use cases and data sets. But just like with Machine Learning models, there are some approaches that you should try first because they are more likely to work.