Epistemic uncertainty

As explained in Machine Learning System Design

Epistemic uncertainty is uncertainty caused by limited knowledge or insufficient coverage of the input space in the training data.

Epistemic uncertainty comes from limited knowledge: a model encounters an input in a region thinly covered by training data or outside its scope. Unlike aleatoric uncertainty, it can decrease when relevant training data is added. section 6.4

Use the distinction when deciding whether more data could improve a model:

• Treat sparse or out-of-scope inputs as signs of epistemic uncertainty. section 6.4
• Do not expect additional training data to remove aleatoric uncertainty caused by overlapping classes or additive noise. section 6.4
• After establishing a training pipeline, estimate whether collecting more data is economically worthwhile, especially when labeling is expensive. section 6.4

Figure 6.4 A schematic view of the main differences between aleatoric and epistemic uncertainties (source: “Explainable Uncertainty Quantifications for Deep Learning-Based Molecular Property Prediction” by ChuI Yang and YiPei Li, https://mng.bz/n0ZV)

Compare how limited training-data coverage differs from uncertainty inherent in the data itself. section 6.4

How Machine Learning System Design covers it

How deep it goes

Defines the cause and contrasts it with aleatoric uncertainty, then connects it to data-collection decisions without deriving uncertainty measures.

Examples

Uses thinly covered or out-of-scope inputs and a schematic comparison of epistemic and aleatoric uncertainty.

What you can do after

Evaluate whether collecting more training data is likely to improve performance and justify its economic cost.

How it connects

Connects epistemic uncertainty to aleatoric uncertainty, training data, and data sufficiency.

liveBookmanning.com

Read Machine Learning System Design on liveBook Get the book at manning.com

Related concepts

• aleatoric uncertaintycontrasts with · Aleatoric uncertainty is irreducible uncertainty arising from randomness, noise, or ambiguity inherent in the data-generating process.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.