liveBook · Manning

Labeling function

As explained in Machine Learning System Design

A labeling function is a heuristic program or rule that assigns a possibly noisy label to an unlabeled data example.

A labeling function is a cheap, imperfect heuristic that labels otherwise unlabeled examples. Collections of these rules produce noisy training data for multisource weak supervision. section 13.4

Use labeling functions when:

  • A model issue needs a prompt workaround while retraining or a full release would take too long. section 13.4
  • Existing overrides can be collected into a noisy dataset rather than remaining untracked technical debt. section 13.4

From overrides to training data

Rule-based overrides can correct outputs for particular scenarios and later become labeling functions. Their combined, imperfect labels provide a foundation for model training, although the rules must be tracked and maintained. section 13.4

How Machine Learning System Design covers it

How deep it goes
Defines imperfect heuristic labeling and its training role, but defers detailed weak-supervision methods to external papers and tooling.
Examples
Uses collections of production overrides as noisy labels and points to Snorkel as a weak-supervision library.
What it assumes
Assumes unlabeled examples, heuristic or rule-based knowledge, and a downstream model-training process.
What you can do after
Apply via multisource weak supervision by converting tracked heuristics into a noisy training dataset.
How it connects
Connects overrides, fallbacks, multisource weak supervision, rule-based systems, and model training.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.