liveBook · Manning

Training

As explained in Outlier Detection in Python

The process of fitting a model to data so that it learns parameters or patterns.

Training fits a model to data by identifying its predominant patterns and representing them. In outlier detection, this establishes a reference for what counts as normal; predicting is a distinct step that compares records against that reference. section 4.4 section 4.14

How fitting works depends on the model:

  • Some models summarize the training data with statistics, clusters, or frequent item sets. section 4.4
  • Distance- and density-based models such as KNN and Radius may retain the training points as their reference, making the choice of training data especially important. section 4.4
  • Training can use a sample to save time, but sample size trades off speed and accuracy. Test with manipulated records to estimate a suitable size; prediction usually still needs to cover every record. section 10.2.1

Setting a sensor baseline

Fit a model to collected sensor readings believed to be reasonably normal, then compare incoming readings with that baseline. Refitting may be needed as conditions change, since older or unrepresentative data can make the reference misleading. section 4.14 section 4.4

How Outlier Detection in Python covers it

How deep it goes
Defines fitting, contrasts model representations, and discusses data representativeness, refitting, and sample-size trade-offs.
Examples
Uses ongoing sensor readings and financial audits to illustrate comparisons with current or prior-period baselines.
What you can do after
Apply it by fitting a model to typical data, then comparing incoming records against the learned reference.
How it connects
Connects model fitting to outlier detection, prediction, training data, and distance-based methods such as KNN and LOF.

Also explained in

Compare how 24 books explain training →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Outlier Detection in Python. The section links point to the parts of the book it draws on.