A principal component is a new variable formed as a linear combination of the original variables. PCA orders these directions by captured variance: the first captures the most, while each later component is orthogonal to the preceding ones. This reduces dimensions while preserving important dataset structure. section 3.5
For implementation, PCA can select significant eigenvectors from an eigenvalue decomposition. Each eigenvector supplies a direction, while its eigenvalue represents the variance captured along that direction. The selected directions become the principal components used for further analysis. section 3.5.1
Estimating house value
Suppose house value depends on area, bedrooms, balconies, and distances to transport and schools. PCA can combine these variables into a new direction, such as a weighted expression of area, bedrooms, distance, and schools, chosen to capture maximum variance. A second component adds the greatest remaining variance while staying orthogonal to the first. section 3.5
Use principal components with these practical constraints:
- Prepare numeric inputs and normalize or standardize the dataset before applying PCA. section 3.7
- Expect reduced dimensionality and possible overfitting benefits, but accept information loss and weaker real-world interpretability. section 3.7
