A sparse representation stores only values that are present or nonzero, rather than filling a large structure with default zeros. In natural-language processing, a document may use only a few words from a vocabulary containing thousands or millions of tokens. A dense vector still allocates one position per vocabulary term; a sparse structure records only the active terms and their counts. section 2.5.3 section 3.2.1
Sparse structures are useful because they:
- Reduce RAM usage by omitting zero counts in document-term matrices. section 3.2.1
- Support mathematical operations that ordinary dictionaries do not, while retaining their compactness. section 2.5.3
- Scale count and TF–IDF vectors to large corpora without materializing every unused vocabulary position. section 3.2.1 section 3.6.2
Count vectors and search
CountVectorizer can fit a vocabulary and transform documents into a compressed sparse row matrix. A 16-document corpus with 240 vocabulary columns contains only 376 stored elements. For a larger TF–IDF corpus, 41,531 documents and 56,404 columns require 788,383 stored values rather than billions of dense entries. Query vectors can remain sparse while dot products rank documents by relevance.
Inspecting a sparse vector sometimes requires densifying it with .toarray(), which is reasonable for a tiny example or a small slice. It is dangerous for production-scale data: converting the 41,531 × 56,404 TF–IDF matrix to dense form would fill every gap with zeros and require approximately 2.3 billion floating-point values, or about 2.4 GB of RAM. section 3.2.1 section 3.6.2
