Semantic similarity

As explained in Building Reliable AI Systems

A measure of how closely the meanings of two pieces of content correspond.

Semantic similarity estimates how closely two pieces of content correspond in meaning. In embedding-based retrieval, related documents and queries occupy nearby positions in a high-dimensional meaning space. section 4.1.1

Retrieval systems can use similarity thresholds to keep only passages directly relevant to a query, reducing distracting context. Similarity matching can also help compare generated claims with grounding data during hallucination checks. section 3.6.5 section 9.1.1

Filtering a roaming query

For “How do I set up international roaming?”, a similarity filter can retain the setup FAQ while excluding related rate, destination-coverage, and terms pages. section 3.6.5

Figure 4.3 Visualizing embeddings across a vector space. Each point represents a document that has been mapped to a specific location based on its semantic content.

See how documents with related meanings cluster as neighboring points in a projected vector space. section 4.1.1

How Building Reliable AI Systems covers it

How deep it goes

Explains spatial proximity and threshold filtering, but gives no similarity formula or scoring implementation.

Examples

Uses clustered policy documents, an international-roaming FAQ query, and claim comparisons against knowledge-base data.

What you can do after

Evaluate retrieved passages for relevance and compare generated claims with grounding data.

How it connects

Connects semantic similarity to embeddings, vector search, retrieval-augmented generation, and hallucination evaluation.

liveBookmanning.com

Read Building Reliable AI Systems on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  AI-Powered SearchSemantic textual similarity is the task or measure of determining how closely two text segments correspond in meaning.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionSemantic similarity is the degree to which two language items have similar meanings or contextual associations.
• 

  liveBookmanning.com
  Effective Conversational AISemantic similarity is the degree of shared meaning between two pieces of text, typically estimated from their representations.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningSimilarity of meaning between terms or documents, as opposed to exact string overlap.

Compare how 5 books explain semantic similarity →

Related concepts

• BERTScoremeasured by · BERTScore is an automatic text-generation evaluation metric that compares candidate and reference texts using contextual representations of their tokens.
• similarity searchused by · Similarity search is a retrieval technique that finds or ranks items by comparing their representations with a query according to a similarity or distance measure.
• embeddingmeasures · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.

This explainer was generated by Manning's knowledge graph from the text of Building Reliable AI Systems. The section links point to the parts of the book it draws on.