liveBook · Manning

Semantic similarity

As explained in Retrieval Augmented Generation, The Foundational Ideas

Semantic similarity is the degree to which two linguistic items have similar meanings or conceptual content.

Semantic similarity measures how closely linguistic items share meaning or conceptual content. Modern systems learn this relationship from data rather than relying only on hand-engineered rules. section 2.3.3

Sentence-BERT learns similarity through several complementary signals:

  • Natural Language Inference pairs provide entailment relationships. section 2.3.3
  • Semantic Textual Similarity data supplies human-annotated similarity scores. section 2.3.3
  • Domain-specific fine-tuning can improve performance, but requires labeled pairs and a chosen loss. section 2.3.3

Indexing at production scale

Encoding a million documents is a batch job requiring a forward pass per text and GPU resources, unlike Word2Vec's arithmetic lookup. section 2.3.3

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains training signals, domain adaptation, and production limitations, but stops short of specifying model architecture or loss functions.
Examples
Contrasts million-document Sentence-BERT indexing with Word2Vec lookup and notes sentence-level context limits.
What you can do after
Apply via Sentence-BERT to perform similarity search while weighing domain, context-length, and indexing constraints.
How it connects
Connects semantic similarity with Sentence-BERT, NLI, STS, domain adaptation, retrieval, reranking, and RAG.

Also explained in

Compare how 8 books explain semantic similarity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.