A dataset can have many measurements that partly repeat one another. When two features vary together, their covariance signals that some of their information may fit along a single direction. Principal component analysis (PCA) finds new, perpendicular axes along which the data varies most, then projects the data onto a chosen number of those axes. The first component captures the greatest variance; the next captures the greatest remaining variance. Keeping fewer components makes the data easier to work with, but discards variation. section 4.3.1
Turn a point cloud into a plot
A three-dimensional point cloud can be projected onto two components with `PCA(n_components=2).fit_transform(df)` from scikit-learn. Plotting the resulting x and y columns gives a view aligned with the cloud’s two strongest directions of variation—like choosing an angle for a photograph, rather than simply dropping one original coordinate. section 4.3.1
Listing 4.7 PCA magic
>>> import pandas as pd
>>> from sklearn.decomposition import PCA
>>> import seaborn
>>> from matplotlib import pyplot as plt
>>> DATA_DIR = ('https://gitlab.com/tangibleai/nlpia/'
... '-/raw/master/src/nlpia/data')
>>> df = pd.read_csv(DATA_DIR + '/pointcloud.csv.gz', index_col=0)
>>> pca = PCA(n_components=2) #1
>>> df2d = pd.DataFrame(pca.fit_transform(df), columns=list('xy'))
>>> df2d.plot(kind='scatter', x='x', y='y')
>>> plt.show()
The listing loads a point cloud, fits two components, and plots the projected coordinates. section 4.3.1
Singular value decomposition (SVD) is the matrix procedure behind this approach. A full decomposition can reconstruct the original matrix; a truncated decomposition keeps only selected leading dimensions. For sparse bag-of-words or TF–IDF document matrices, scikit-learn’s `TruncatedSVD` is often a better practical choice than `PCA`: PCA centers columns first, while centering sparse data can make it dense, slower, and more memory-intensive. Centering before `TruncatedSVD` can produce the same results as PCA, but loses that sparse-data advantage. section 4.3.2 section 4.4.2
Map word-vector relationships
PCA can also project 300-dimensional city word vectors into a two-dimensional bubble plot. Nearby points can suggest relationships learned from language, not physical proximity: cities similar in culture may cluster despite being far apart. Treat the plot as a view of the vectors, not an accurate geographic map; a different training corpus may produce different relationships. section 6.4.5
Listing 6.11 Bubble chart of US cities
>>> from sklearn.decomposition import PCA
>>> pca = PCA(n_components=2) #1
>>> us_300D = get_data('cities_us_wordvectors')
>>> us_2D = pca.fit_transform(us_300D.iloc[:, :300]) #2
The listing applies scikit-learn PCA to the first 300 columns of a city-vector dataset to produce 2D coordinates. section 6.4.5
