A sparse vector stores many dimensions while leaving most entries at zero or empty. In language processing, each dimension can represent a vocabulary word, so a document usually activates only a small portion of the full vocabulary. section 1.5.3 section 3.2.3
For a corpus with K distinct words, each document can become a K-dimensional vector whose positions hold word counts or TF–IDF values. Sparsity makes these high-dimensional representations more manageable than their dimensionality alone suggests. section 3.2.3
Sparse vectors support practical comparisons but do not remove every high-dimensional challenge:
- They let NLP systems compare documents or phrases using numerical vector distances rather than exact character matches. section 1.5.3
- They reduce the impact of the curse of dimensionality because most TF values are zero. section 3.2.3
- They do not make large-scale search automatically easy; nearest-neighbor queries may still require approximate nearest neighbor algorithms. section 3.2.3
A two-dimensional illustration
A term-frequency example uses vectors [1, 0] and [2, 1]. In a real corpus, the same idea extends across every vocabulary term, with most document positions remaining zero. section 3.2.3
Listing 3.8 2D term frequency vector examples
>>> from matplotlib import pyplot as plt
>>> import seaborn as sns
>>> palette = sns.color_palette("muted") #1
>>> sns.set_theme() #2
>>> vecs = pd.DataFrame([[1, 0], [2, 1]], columns=['x', 'y'])
>>> vecs['color'] = palette[:2]
>>> vecs['label'] = [f'vec{i}' for i in range(1, len(vecs)+1)]
>>> fig, ax = plt.subplots()
>>> for i, row in vecs.iterrows():
... ax.quiver(0, 0, row['x'], row['y'], color=row['color'],
... angles='xy', scale_units='xy', scale=1)
... ax.annotate(row['label'], (row['x'], row['y']),
... color=row['color'], verticalalignment='top'
... )
>>> plt.xlim(-1, 3)
>>> plt.ylim(-1, 2)
>>> plt.xlabel('X (e.g. frequency of word "vector")')
>>> plt.ylabel('Y (e.g. frequency of word "space")')
>>> plt.show()
Inspect the runnable Python listing to see sparse-looking term-frequency vectors represented and plotted as coordinates. section 3.2.3
