A pandas Series is a one-dimensional labeled vector: each value has a named index, such as a token, and the structure supports arithmetic without losing those labels. section 3.1 section 3.2
Use a Series to turn counted or scored dictionary data into an inspectable vector:
- Create one directly from a dictionary with pd.Series(dictionary). section 3.1
- Use a Series as a row in a DataFrame so vocabulary labels identify vector dimensions. section 3.2
Token counts as a Series
After tokenizing a sentence and counting tokens with Counter, convert the result with counts = pd.Series(dict(bag_of_words.most_common())). The resulting labels include punctuation and words, while values record their frequencies; counts.sum() equals the number of tokens.
