liveBook · Manning

Analogy querying

As explained in Natural Language Processing in Action, Second Edition

Analogy reasoning is a technique that combines or compares vector representations arithmetically to model relationships among concepts.

Analogy querying answers relationship-based questions by combining word vectors arithmetically. Instead of searching for “Who is to nuclear physics what Louis Pasteur is to germs?”, subtract the known relationship and add the target domain. section 6.2.3 section 6.2

For example, calculate `wv['Louis_Pasteur'] - wv['germs'] + wv['physics']`. The result usually is not an existing vocabulary vector, so retrieve the nearest vector and read its associated word as the answer. section 6.2.3 section 6.3.1

A practical analogy-querying workflow is:

  • Express the question as a relationship: Portland Timbers + Seattle − Portland. section 6.3.1
  • Add and subtract the corresponding Word2Vec vectors in a continuous, often hundreds-dimensional space. section 6.3.1
  • Find the vocabulary vector closest to the resultant rather than expecting exact equality. section 6.3.1
  • Treat relationships as corpus-dependent: a model can expose cultural or economic similarities, but also reproduce training-data biases and stereotypes. section 6.4.5

Cities and teams

The expression `Portland Timbers + Seattle − Portland` should approach `Seattle Sounders`, because the model represents Portland Timbers and Seattle Sounders as similarly directed displacements from their cities. A related visualization adds state vectors to city vectors, then projects the resulting 300-dimensional representations into two dimensions with PCA. section 6.3.1 section 6.4.5

>>> answer_vector = wv['Louis_Pasteur'] - wv['germs'] + wv['physics']

Inspect the minimal Python expression that turns an analogy into vector arithmetic. section 6.2.3

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Explains vector subtraction, addition, nearest-vector retrieval, and corpus dependence, but does not develop embedding training mathematics.
Examples
Uses Louis Pasteur and germs, Portland Timbers and Seattle Sounders, standardized-test analogies, and city-plus-state vectors.
What it assumes
Assumes word vectors and basic vector arithmetic while introducing analogy questions through familiar applications and executable expressions.
What you can do after
Apply it via Word2Vec vector arithmetic and nearest-vector lookup to answer analogy questions when candidate words exist.
How it connects
Connects word embeddings with Word2Vec, semantic search, similarity querying, PCA, and corpus bias.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.