liveBook · Manning

Vector space

As explained in LLM Evaluation and Alignment, The Foundational Ideas

A vector space is a mathematical space whose elements are vectors that can be combined through vector addition and scalar multiplication.

A vector space represents language as points in a high-dimensional geometric space rather than as atomic strings. Words or passages with related meanings occupy nearby regions, allowing semantic metrics to recognize similarity beyond exact word overlap. section 3.2.3

Use the geometric model to interpret semantic comparison:

  • Treat lexical matching as checking whether words occupy identical coordinates. section 3.2.3
  • Treat semantic matching as measuring distance between vector representations. section 3.2.3
  • Expect contextual embeddings to assign different locations to the same word in different contexts. section 3.2.3
  • Compare orientation when vector magnitude reflects length or frequency more than semantic content. section 3.2.3

Meaning without shared words

“The attorney filed the brief” and “the lawyer submitted the document” can occupy nearby regions despite sharing no words, because their vector representations capture related meaning. section 3.2.3

Figure 3.6 Geometric representation of two semantic vectors. Note that while the vectors have different lengths (magnitudes), they point in roughly the same direction, indicating a shared semantic topic.

Use the geometric representation to see how vectors with different magnitudes can still point in similar directions and indicate a shared semantic topic. section 3.2.3

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Explains points, neighborhoods, contextual embeddings, distance, magnitude, and orientation without developing implementation details.
Examples
Uses synonym pairs, context-dependent uses of “run,” and semantically equivalent passages with no shared words.
What you can do after
Evaluate/understand semantic similarity by interpreting embedding proximity instead of relying only on exact word overlap.
How it connects
Connects vector spaces to BERTScore, contextual embeddings, cosine similarity, semantic similarity, and lexical metrics.

Also explained in

Compare how 6 books explain vector space →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.