An eigenvector is a nonzero vector whose direction is preserved by a square matrix’s linear transformation. For a matrix A, vector v, and scalar eigenvalue λ, the defining relation is A v = λv. The transformation may stretch, shrink, or reverse v, but it leaves v parallel to its original direction. Eigenvectors expose the directions that a transformation treats in a particularly simple way, making them useful for finding broad structure in high-dimensional data. section 2.13
Figure 2.13 During rotation, points on the axis of rotation do not change position.
A rotation about the Z-axis gives direct geometric intuition: vectors on the rotation axis remain fixed, so the axis direction is an eigenvector. section 2.13
To find eigenvectors, rearrange A v = λv as (A − λI)v = 0. A nonzero solution exists only when det(A − λI) = 0; solving this condition gives eigenvalues, and substituting each eigenvalue back gives its eigenvector directions. The zero vector is excluded because it satisfies the homogeneous equation trivially and supplies no direction. section 2.13
Eigenvectors gain additional structure in important matrix classes:
- Eigenvectors associated with unequal eigenvalues are linearly independent. section 2.13.1
- For a symmetric matrix, eigenvectors associated with different eigenvalues are orthogonal. section 2.13.2
- For a rotation matrix, eigenvalue 1 has an eigenvector along the axis of rotation; vectors on that axis stay in place. section 2.14.4
If an n × n matrix has n linearly independent eigenvectors, place them as columns of S. The matrix can then be diagonalized as A = SΛS⁻¹, where Λ holds the eigenvalues on its diagonal. For symmetric A, the eigenvector columns are orthogonal, so S⁻¹ = Sᵀ and A = SΛSᵀ. Diagonalization changes coordinates to directions in which the transformation acts by simple scaling. section 2.15
Finding the axes of a rotated ellipse
For 5x² + 6xy + 5y² = 20, form the symmetric matrix A = [[5, 3], [3, 5]]. Its eigenvectors form an orthogonal rotation matrix. Rotating coordinates by that matrix diagonalizes the quadratic form, aligning coordinate axes with the ellipse’s axes; the major axis makes a 45° angle with the X-axis. section 2.17.1
In PCA, eigenvectors of a covariance matrix identify data directions. The eigenvector with the largest eigenvalue gives the direction of maximum variance, called the first principal axis. Remaining eigenvectors, ordered by decreasing eigenvalue, give orthogonal directions of successively lower variance. Projecting onto axes with larger eigenvalues can reduce dimensionality while retaining stronger patterns and discarding relatively noisy directions. section 4.7 section 5.10
Listing 2.13 Eigenvalues and vectors
from torch import linalg as LA
A = torch.tensor([[0.707, 0.707, 0],
[-0.707, 0.707, 0], [0, 0, 1]]) #1
l, e = LA.eig(A) #2
print("Eigen values are {}".format(l))
print("Eigen vectors are {}".format(e.T)) #3
PyTorch computes eigenvalues and eigenvectors of a matrix with torch.linalg.eig, returning eigenvalues and an eigenvector matrix. section 2.13.3
