How can a chatbot tell whether two messages are related without a rule for every possible sentence? A vector space model turns language into lists of numbers, then uses relationships among those lists. The numbers might count words in a document or represent qualities of a word or message. Once language is represented as vectors, a system can compare messages, find patterns, and retrieve relevant knowledge. section 1.10 section 1.5.3
Start with a bag of words (BOW). Imagine sorting each token into a labeled bin and recording the count in every bin. The resulting document vector is usually sparse: most entries are zero, while entries for words present in the document hold small counts. All the vectors that could be formed from these bins make up a vector space. Counting discards word order, but gives a machine numbers it can work with. section 1.5.3
Figure 1.6 Token-sorting tray
The token-sorting tray makes the mapping from word bins to a BOW vector visible. section 1.5.3
Count a short message
In Python, `Counter("Guten Morgen Rosa".split())` gives each of the three whitespace-separated tokens a count of one. A second message such as `"Good morning Rosa!"` produces different keys—including `Rosa!` rather than `Rosa`. The experiment shows both how easy count vectors are to build and why token cleanup matters before comparing them. section 1.5.3
A separate dimension for each distinct token makes related messages in different words difficult to connect: “Good morning, Hobs” and “Guten Morgen, Hannes” need not look alike. A reduced set of continuous-valued dimensions can instead describe qualities such as subject matter, sentiment, or whether a message sounds like a question. Nearby vectors can then help a system recognize patterns and act on them. section 1.6
The choice of vector representation also determines what a later model consumes. Latent Dirichlet Allocation (LDiA) uses BOW vectors; latent semantic analysis (LSA) instead takes a TF–IDF matrix. When preparing BOWs for LDiA, calculate vocabulary and length statistics after tokenization and any stop-word filtering, so those statistics describe the vectors actually used. section 4.5.1
Explore continuous word vectors
A loaded Word2Vec model provides continuous word vectors through `gensim`’s `KeyedVectors`. Try `word_vectors.most_similar(positive=['cooking', 'potatoes'], topn=5)` to inspect neighbors, `word_vectors.similarity('princess', 'queen')` to compare two terms, or `word_vectors['phone']` to inspect the numbers themselves. These scores and neighbors are useful relationships to examine, not proof that a system understands every use of a word. section 6.3.4
