In principal component analysis (PCA), an eigenvalue tells you how much variance is captured along its corresponding eigenvector. The eigenvector supplies a direction; the eigenvalue helps decide whether that direction is worth retaining when reducing the number of variables. section 3.5.1
(3.2)
A*v = λ*v
The relation A*v = λ*v links a square matrix A, an eigenvector v, and its eigenvalue λ. For PCA, A is the covariance matrix computed from normalized data. section 3.5.1 section 3.5
Use eigenvalues to select principal components:
- Normalize the variables so they are comparable, then calculate their covariance matrix. section 3.5
- Calculate the covariance matrix’s eigenvalues and eigenvectors, keeping each value paired with its vector. section 3.5
- Sort the pairs by decreasing eigenvalue. Retain the eigenvectors associated with the largest values to capture the most variance. section 3.5
Iris flower measurements
With the Iris dataset, separate the flower measurements from the species label and standardize the measurements using StandardScaler. NumPy’s np.linalg.eig can calculate eigenvalues and eigenvectors from their covariance matrix. Alternatively, fit sklearn.decomposition.PCA to the standardized measurements and inspect explained_variance_ratio_. A bar plot or cumulative-variance plot helps assess how much variance the retained components capture; the example then reduces four measurement variables to two while keeping 150 rows. section 3.5.2 section 3.5
Eigenvalue decomposition applies to square matrices, such as the covariance matrix used here. Singular value decomposition also supports dimensionality reduction and can operate on rectangular matrices. section 3.5.1
