CountVectorizer turns documents into bag-of-words vectors: each vocabulary token receives a position, and each document records how often that token appears. The resulting matrix preserves token counts without depending on word position and uses sparse storage to avoid wasting memory on zeros. section 2.5.3
Use the transformer in three practical steps:
- Create and configure an empty CountVectorizer instance. section 3.2.1
- Fit it to the corpus so it learns the vocabulary and token document frequencies. section 3.2.1
- Transform the corpus into a sparse matrix of integer token counts; use max_df or min_df to filter overly common or rare tokens. section 3.2.1
Compute word-count vectors
from sklearn.feature_extraction.text import CountVectorizer corpus = [doc.text for doc in docs] vectorizer = CountVectorizer() count_vectors = vectorizer.fit(corpus).transform(corpus) print(count_vectors) print(count_vectors.toarray()) A sample corpus produces a 16-by-240 sparse matrix with 376 stored elements. Convert it to an array when inspecting individual rows, not when memory efficiency matters. section 3.2.1
Listing 3.3 Using sklearn to compute word count vectors
>>> import numpy as np
>>> from sklearn.feature_extraction.text import CountVectorizer
>>> np.set_printoptions(edgeitems=8) #1
>>> corpus = [doc.text for doc in docs]
>>> vectorizer = CountVectorizer() #2
>>> vectorizer = vectorizer.fit(corpus) #3
>>> count_vectors = vectorizer.transform(corpus) #4
>>> count_vectors
<16x240 sparse matrix of type '<class 'numpy.int64'>'
with 376 stored elements in Compressed Sparse Row format>
Open the runnable listing for the complete scikit-learn workflow, including imports, corpus preparation, fitting, transforming, and sparse-matrix inspection. section 3.2.1
