5 Advanced active learning

 

This chapter covers

  • Combining uncertainty sampling and diversity sampling techniques
  • Using active transfer learning to sample the most uncertain and the most representative items
  • Implementing adaptive transfer learning within an active learning cycle

In chapters 3 and 4, you learned how to identify where your model is uncertain (what your model knows it doesn’t know) and what is missing from your model (what your model doesn’t know that it doesn’t know). In this chapter, you learn how to combine these techniques into a comprehensive active learning strategy. You also learn how to use transfer learning to adapt your models to predict which items to sample.

5.1 Combining uncertainty sampling and diversity sampling

This section explores ways to combine all the active learning techniques that you have learned up to this point so that you can use them effectively them for your particular use cases. You will also learn one new active learning strategy: expected error reduction, which combines principles of uncertainty sampling and diversity sampling. Recall from chapter 1 that an ideal strategy for active learning tries to sample items that are near the decision boundary but are distant from one another, as shown in figure 5.1.

5.1.1 Least confidence sampling with cluster-based sampling

5.1.2 Uncertainty sampling with model-based outliers

5.1.3 Uncertainty sampling with model-based outliers and clustering

5.1.4 Representative sampling cluster-based sampling

5.1.5 Sampling from the highest-entropy cluster

5.1.6 Other combinations of active learning strategies

5.1.7 Combining active learning scores

5.1.8 Expected error reduction sampling

5.2 Active transfer learning for uncertainty sampling

Summary