An embedding is a numerical vector used to represent meaning. A word embedding gives each word an ordered row of numbers: the positions stay consistent across words, even though individual positions have no simple labels. Nearby words in the resulting vector space tend to have similar meanings. That makes it possible to compare words by meaning rather than only by spelling or exact match. section 6.0 section 6.2.1
Not every text vector is an embedding. A bag-of-words vector records whether words occur, or how often; a TF–IDF vector weights their importance. Those representations are useful for retrieval, but they can miss relationships between different words. Embeddings aim to capture such relationships, so a search or question-answering system can match related wording without requiring the same tokens. section 1.1.1 section 4.9 section 6.2.1
Consider a job search. A query containing “Engineer” need not rely only on that exact word: a word-vector index can also retrieve nearby terms such as “Architect,” “Technician,” and “Programmer.” Neighbors are candidates to inspect, not proof that the jobs are interchangeable. The same approach supports semantic search for web pages, word finding, and rewording; embeddings can also represent longer passages. section 6.2.1 section 6.2
>>> from nessvec.indexers import Index #1
>>> index = Index(num_vecs=100_000) #2
>>> index.get_nearest("Engineer").round(2)
Engineer 0.00
engineer 0.23
Engineers 0.27
Engineering 0.30
Architect 0.35
engineers 0.36
Technician 0.36
Programmer 0.39
Consultant 0.39
Scientist 0.39
Inspect the nearest-neighbor results for “Engineer.” The returned terms and distances make semantic proximity concrete while leaving room to judge whether each match fits the search. section 6.2.1
Word2Vec learns embeddings from nearby words in a large body of text. Instead of supplying dictionary definitions, you tokenize and segment sentences, then use small neighborhoods of words as training context. This makes the method self-supervised, but its vocabulary still matters: a vector-based answer cannot retrieve a word for which no vector exists. section 6.2.4 section 6.2.3
Reasoning with vectors
An analogy can be posed as vector arithmetic: take the vector for “Louis Pasteur,” subtract “germs,” and add “physics” to seek a related name. This is a way to query relationships learned from text, not a guarantee that the result is correct. Similarly, a question-answering bot can compare semantic question vectors to find a FAQ whose wording differs from the user’s question. section 6.2.3 section 4.9
Keep the representation matched to the task. Bag-of-words vectors can work well for short greetings but discard word order that matters in complex statements. Embeddings add useful nuance; they do not make a chatbot generally intelligent or its answers inherently reliable. section 1.7 section 6.2.5
