A principal component is an orthogonal direction that captures variance in data. Components are selected successively: the first represents the greatest variance, while later components represent decreasing amounts of the remaining variance. PCA rotates the feature space into these components, often preserving the data’s structure with fewer dimensions. section 6.9
The transformation is useful for correlated features and outlier detection. An observation can be extreme along a component, or can appear unusual because its values do not follow the dominant pattern. In two dimensions, PCA rotates and stretches the data; higher-dimensional transformations work similarly but are harder to visualize. section 6.9
Use principal components in several complementary ways:
- Inspect explained variance to determine which components capture most of the dataset’s variation. section 6.9
- Apply univariate tests after transformation, because PCA components are independent by construction and can expose unusual combinations of feature values. section 6.9.1
- Select or weight early, middle, or late components according to whether extreme values or deviations from patterns matter most. section 6.9.1
- Treat detector improvements as dataset-dependent rather than guaranteed; PCA improved accuracy in the tested datasets, but this was not always expected to occur. section 10.1.1
Correlated two-dimensional data
Generate one random feature and a second feature formed by adding small noise to it, then append two outliers: one extreme point that follows the diagonal pattern and one point with typical individual values that violates the pattern. After fitting PCA, inspect the transformed coordinates; the points become conspicuous in component space. section 6.9
