A dataset with many variables can be difficult to visualize and costly to analyze. Singular value decomposition (SVD) reduces dimensions by representing a data matrix through singular vectors and singular values. Like principal component analysis (PCA), it supports a transformation to fewer dimensions, but SVD can factorize a matrix with any number of rows and columns. section 3.0 section 3.6
(3.3)
A = U * S * VT
The factorization writes the original matrix A as U × S × Vᵀ. U and V contain orthonormal singular vectors; the diagonal matrix S holds the singular values. The vectors come from the square matrices AAᵀ and AᵀA, and the singular values are square roots of their eigenvalues. section 3.6
A practical Python workflow is:
- Inspect and clean the data. Convert categorical features to numeric values, separate features from any class label, and standardize the features. section 3.6.1 section 3.8
- Run `u, s, v = np.linalg.svd(X_features, full_matrices=True)`. Inspect the outputs: `s` contains singular values, while `u` and `v` contain singular vectors. section 3.6.1
- Calculate each displayed variance share as `s**2 / np.sum(s**2)`. Plot the shares to judge how many components to retain, then inspect a scatter plot using the selected components. section 3.6.1
Exploring mushroom classes
Encode the mushroom dataset’s categorical columns, standardize its feature columns, and run NumPy’s SVD. A bar plot compares the variance shares; a scatter plot places the first two plotted singular vectors on its axes and colors observations as edible or poisonous. Two axes make the data easier to inspect, but check the variance plot before deciding that two components are enough. section 3.6.1
Reduced dimensions can make visualization easier and lower training time; reducing correlated variables may also help model accuracy and overfitting. The trade-offs are possible information loss and components that are harder to relate to real-world attributes. SVD requires numeric, standardized input. For a telecommunications dataset combining demographics, subscriptions, usage, and payments, first check missing values, duplicates, and uninformative variables before reducing dimensions for later analysis. section 3.7 section 3.8
