liveBook · Manning

Principal vector

As explained in Math and Architectures of Deep Learning

An eigenvector in PCA that indicates a direction of variation in the data, paired with a principal value.

A principal vector is an eigenvector from PCA that indicates a direction of variation in data. Its paired principal value measures the variance along that direction. The largest value-vector pair identifies maximum variance; subsequent vectors are orthogonal directions with progressively smaller variance. section 14.4

Geometrically, principal vectors form axes for projecting data onto a lower-dimensional manifold. A small principal value signals little variation, allowing that direction to be discarded with relatively low reconstruction loss and possible noise reduction. section 14.4

Correlated 3D data

For points clustered near the X0 = X2 plane, the two largest principal vectors lie within that plane. The smallest-value vector is normal to it, so setting that component to zero produces a 2D representation. section 14.4

Figure 14.3 The original 3D data in figure 14.3a shows high correlation: points are clustered around the X0 = X2 plane. The first principal component corresponds to the direction of maximum variance. The last (third) principal reduced to a 2D latent vector.

(a) Original 3D data

(b) Lower-dimensional 2D representation obtained by setting the third principal value to zero

(c) The principal vectors of the original data. The third principal vector is normal to X0 = X2 plane; the other two are in-plane.

Compare the original 3D point cloud, its 2D projection, and the three principal-vector directions. section 14.4

How Math and Architectures of Deep Learning covers it

How deep it goes
Explains paired eigenvectors and variances geometrically, including orthogonality, projection, reconstruction loss, and noise removal.
Examples
Uses 3D points clustered around the X0 = X2 plane and removes the normal-to-plane component.
What you can do after
Evaluate PCA directions and decide which low-variance components can be removed with limited reconstruction loss.
How it connects
Connects PCA with eigenvectors, principal values, dimensionality reduction, latent representations, projection, and reconstruction.
Cover of Math and Architectures of Deep Learning
liveBookmanning.com
Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.