Dimensionality reduction finds a representation with fewer dimensions while retaining as much useful information as possible. Think of a sofa’s shadow: it represents a 3D object in 2D, but what you can see depends on the view. Likewise, a useful representation depends on what you need to preserve. section 4.3
In text analysis, a bag-of-words or TF–IDF vector has a dimension for each term. Even a small dataset can produce thousands of dimensions, with related words occupying separate positions. Topic modeling instead seeks a smaller space whose dimensions capture shared semantic patterns. Its topic vectors can make document similarity and semantic search more meaningful than arithmetic on word-count vectors. section 4.1.2 section 4.3
Choose the reduction to fit the job:
- Use a tool such as principal component analysis (PCA) to simplify high-dimensional data for visualization; sparse, high-dimensional inputs can also make classifiers more prone to overfitting. section 4.3
- Reduce embedding dimensions before storing them with pgvector if they exceed its 2,000-dimension dense-vector limit. section 10.3.5
- For a different kind of reduction, stem related word forms into one vocabulary entry. This can broaden search recall, but may also return more irrelevant results. section 2.3.2
