An eigenvector is a nonzero vector whose direction is unchanged by a linear transformation; the transformation only scales it. In principal component analysis (PCA), eigenvectors of a covariance matrix identify directions through the data. The corresponding eigenvalues indicate how much variance those directions capture. section 3.5.1
(3.2)
A*v = λ*v
The equation A*v = λ*v expresses the relationship: applying square matrix A to eigenvector v has the same effect as multiplying v by eigenvalue λ. section 3.5.1
This makes eigenvectors useful for reducing dimensions. A principal component is a linear combination of the original variables, with weights supplied by an eigenvector. To choose components, normalize the variables, calculate their covariance matrix, find its eigenvectors and eigenvalues, then favor the eigenvectors associated with the largest eigenvalues. The chosen components retain the directions of greatest variance. section 3.5 section 3.5.1
Inspect directions in the Iris data
Start with flower measurements as input variables and keep species separate as the target. Inspect the data, standardize the measurements with StandardScaler, and calculate their covariance matrix with np.cov. Then use np.linalg.eig to obtain eigenvalues and eigenvectors. Compare this inspection with a fitted sklearn PCA model: its explained_variance_ratio_ and cumulative-variance plot help decide how many components to keep. A two-component result reduces four input variables to two while retaining all 150 observations. section 3.5.2
For a covariance matrix, the eigenvectors used as component directions are orthogonal: a subsequent component can capture variance along a direction distinct from the first. Eigenvalue decomposition provides the matrix-based explanation; the practical workflow can leave the calculation to NumPy or scikit-learn. section 3.5 section 3.5.1
