Orthogonality describes directions whose inner product is zero; in Euclidean space, they meet at a 90° angle. In principal component analysis, each later component is orthogonal to the preceding components, preventing correlation between them while preserving useful variance. section 3.5
House-value dimensionality reduction
For a dataset containing area, bedrooms, balconies, and travel distances, PCA combines raw variables into principal components. The first captures maximum variance; the next captures additional variance while remaining orthogonal to the first. section 3.5
Figure 3.8 The first figure on the left is the first principal component. The second principal component can be at different angles with respect to the first principal component (middle). We should find the second principle that allows us to capture the maximum variance. To capture the maximum variance, the second principal component should be orthogonal to the first one, and thus the combined variance captured is maximized (right).
Compare candidate angles for a second principal component and see why the orthogonal choice maximizes combined captured variance. section 3.5
