Orthogonality describes perpendicular or mutually independent directions. For vectors, the condition is a zero dot product: uᵀv = 0. Geometrically, the vectors meet at 90 degrees; physically, one direction cannot influence the other. The dot product ||u||||v||cosθ also measures how effective one vector is along another: alignment maximizes influence, while orthogonality makes it zero. section 2.6
Orthogonality connects geometry to matrix computation and machine learning:
- Shows that eigenvectors of a symmetric matrix associated with different eigenvalues are mutually orthogonal, because (λ₁ − λ₂)v₁ᵀv₂ = 0. section 2.13.2
- Makes the eigenvector matrix of a symmetric matrix orthogonal, giving S⁻¹ = Sᵀ and the diagonalization A = SΛSᵀ. section 2.15
- Aligns coordinate axes with hyperellipse axes through an orthogonal eigenvector matrix, which acts as a rotation matrix. section 2.17.1
- Separates principal directions in PCA: eigenvectors of the covariance matrix are mutually orthogonal and ordered by decreasing variance. section 4.7
Rotation matrix in PyTorch
For a 30-degree rotation matrix R, verify that Rᵀ is its inverse and that RRᵀ equals the identity. Applying R to u = [4, 0] preserves its norm; applying a −30-degree rotation returns the original vector. section 2.14.3
Listing 2.16 Orthogonality of rotation matrices
R_30 = rotation_matrix_2d(30) #1
assert torch.allclose(
torch.linalg.inv(R_30),
R_30.T
) #2
assert torch.allclose(
torch.matmul(R_30, R_30.T),
torch.eye(2) #3
)
u = torch.tensor([[4],[0]], dtype=torch.float)
v = torch.matmul(R_30, u) #4
assert torch.linalg.norm(u) ==torch.linalg.norm(v) #5
R_neg30 = rotation_matrix_2d(-30)
w = torch.matmul(R_neg30, v) #6
assert torch.all(w == u)
assert torch.allclose(R_30, R_neg30.T)
assert torch.allclose(
torch.matmul(R_30, R_neg30),
torch.eye(2)) #7
Use the PyTorch listing to inspect computational tests for inverse-transpose equality, identity products, norm preservation, and inverse rotations. section 2.14.3
