pandas is a Python library for manipulating labeled tabular data. Its DataFrame and Series structures turn token-count dictionaries into vectors, assigning vocabulary terms to columns and preserving each word’s position during arithmetic. section 3.2
Build a document-term matrix
Count tokens in short documents, construct a DataFrame, then replace missing values with zero and convert counts to integers: count_vectors = [Counter(tokens) for tokens in docs_tokens] tf = pd.DataFrame(count_vectors) tf = tf.T.sort_index().T tf = tf.fillna(0).astype(int) For 16 documents, this produces 246 vocabulary columns, with each cell recording a token count. section 3.2
Listing 3.1 Short documents about bias
>>> docs = [nlp(s) for s in bias_intro.split('\n')
... if s.strip()] #1
>>> counts = []
>>> for doc in docs:
... counts.append(Counter([
... t.text.lower() for t in doc])) #2
>>> df = pd.DataFrame(counts)
>>> df = df.fillna(0).astype(int) #3
>>> len(df)
16
>>> df.head()
algorithmic bias describes systematic ... between same service
0 1 1 1 1 ... 0 0 0
1 0 1 0 0 ... 0 0 0
2 1 1 0 0 ... 0 0 0
3 1 1 0 1 ... 0 0 0
4 0 1 0 0 ... 0 0 0
Inspect the complete workflow for converting bias-related documents into a labeled count matrix. section 3.2
