9 Advanced Data Annotation and Augmentation

 

This chapter covers:

  • Evaluating annotation quality for subjective tasks.
  • Optimizing annotation quality control with machine learning.
  • Treating model predictions as annotations.
  • Combining embeddings/contextual representations with annotations.
  • Using search and rule-based systems for data annotation.
  • Bootstrapping models with Lightly-supervised Machine Learning.
  • Expanding datasets with synthetic data, data creation, and data augmentation.
  • Incorporating annotation information into machine learning models.
 
 
 
 

9.1    Annotation Quality for Subjective Tasks

 

9.1.1   Requesting annotator expectations

 
 
 

9.1.2   Assessing viable labels for subjective tasks

 
 
 

9.1.3   Trusting an annotator to understand the diversity of possible responses

 
 

9.1.4   Bayesian Truth Serum for subjective judgments

 
 
 

9.1.5   Embedding simple tasks in more complicated ones

 
 

9.2    Machine Learning for annotation quality control

 
 
 

9.2.1   Calculating annotation confidence as an optimization task

 
 
 
 

9.2.2   Converging on label confidence when annotators disagree

 
 

9.2.3   Predicting whether a single annotation is correct or in agreement

 
 
 

9.2.4   Predicting whether a single annotation is in agreement

 
 

9.2.5   Predicting whether a single annotation is a bot

 
 
 

9.3    Model predictions as annotations

 
 
 
 

9.3.1   Trusting annotations from confident model predictions

 
 
 

9.3.2   Treating model predictions as a single annotator

 
 
 
 

9.3.3   Cross-validating to find mislabeled data

 
 
 
 

9.4    Embeddings/Contextual Representations

 
 
 
 

9.4.1   Transfer learning from an existing model

 
 
 

9.4.2   Representations from adjacent easy-to-annotate tasks 

 
 

9.9    Further Reading for Advanced Annotation 

 
sitemap

Unable to load book!

The book could not be loaded.

(try again in a couple of minutes)

manning.com homepage
test yourself with a liveTest