Semantic similarity

As explained in Math and Architectures of Deep Learning

Similarity of meaning between terms or documents, as opposed to exact string overlap.

Semantic similarity measures related meaning rather than exact term overlap. Two documents can share no explicit words yet remain similar when their terms repeatedly occur in related contexts. section 4.6.2

Latent semantic analysis addresses this blind spot by grouping co-occurring terms into topics. Geometrically, each topic is a linear subspace or direction in document-feature space, and topic-based similarity compares documents through these directions. section 4.6.2

The treatment connects semantic similarity to:

• Direct overlap: cosine similarity can miss secondary evidence carried by related terms. section 4.6.2
• Topic discrimination: useful topics show high variance across document vectors and distinguish documents in the corpus. section 4.6.2
• Dimensionality reduction: topics correspond to selected principal components after pruning components with insufficient variance. section 4.6.2

Gun and violence

Documents d5 and d6 have little or no direct overlap, yet documents containing gun can resemble documents containing violence because those terms co-occur elsewhere. Topic-based comparison captures this relationship. section 4.6.2

Figure 4.6 Document vectors from our toy dataset d₀, ⋯ d₆. Each word in the vocabulary corresponds to a separate dimension. Dots show projections of document feature vectors on the plane formed by the axes corresponding to the terms gun(s) and violence.

Inspect the document-vector projection to see how vocabulary dimensions and topic directions support similarity judgments. section 4.6.2

How Math and Architectures of Deep Learning covers it

How deep it goes

Explains semantic similarity through topics, subspaces, variance, and principal components.

Examples

Uses documents d5 and d6, the terms gun and violence, and a toy document-vector dataset.

What it assumes

Assumes document vectors and cosine similarity while explaining their limitation through indirect term relationships.

What you can do after

Evaluate document relatedness through topic-based representations rather than relying only on explicit word overlap.

How it connects

Connects semantic similarity to cosine similarity, latent semantic analysis, document vectors, and principal components.

liveBookmanning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  AI-Powered SearchSemantic textual similarity is the task or measure of determining how closely two text segments correspond in meaning.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionSemantic similarity is the degree to which two language items have similar meanings or contextual associations.
• 

  liveBookmanning.com
  Effective Conversational AISemantic similarity is the degree of shared meaning between two pieces of text, typically estimated from their representations.
• 

  liveBookmanning.com
  Building Reliable AI SystemsA measure of how closely the meanings of two pieces of content correspond.

Compare how 5 books explain semantic similarity →

Related concepts

• BERTScoremeasured by · BERTScore is an automatic text-generation evaluation metric that compares candidate and reference texts using contextual representations of their tokens.
• similarity searchused by · Similarity search is a retrieval technique that finds or ranks items by comparing their representations with a query according to a similarity or distance measure.
• embeddingmeasures · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.