liveBook · Manning

Similarity metric

As explained in Math and Architectures of Deep Learning

A measure of how alike two represented items are, especially when computed from their vector representations.

A similarity metric estimates how alike two represented items are. For vector representations, the dot product provides a first quantitative measure: similar vectors tend to produce larger values, while dissimilar or perpendicular vectors produce values near zero. Vectors pointing in opposite directions produce negative dot products. section 2.5.6

For vectors a and b, a·b = aᵀb = aₓbₓ + aᵧbᵧ = ||a|| ||b|| cos(θ). The result reflects how much one vector lies along the other: aligned directions increase the score, perpendicular directions remove directional agreement, and opposite directions reverse it. section 2.5.6

Document retrieval

Represent each document with a descriptor vector whose dimensions correspond to vocabulary terms and whose values record word frequencies. A query can then be compared with archive documents using vector dot products, allowing documents to be ranked by estimated similarity. Sparse storage keeps only the word–frequency pairs for terms that occur. section 4.1

A small comparison

The feature vectors for documents d5 and d6 have dot product 1 × 0 + 0 × 1 = 0, matching their lack of common words of interest. The vectors for d3 and d4 have a higher dot product, matching their greater overlap. section 2.5.6

How Math and Architectures of Deep Learning covers it

How deep it goes
Explains dot-product geometry, angle dependence, document-vector construction, sparse storage, and worked comparisons.
Examples
Uses document descriptors, query ranking, word-frequency vectors, and d5–d6 versus d3–d4 dot-product comparisons.
What it assumes
Assumes vector representations and word-frequency descriptors while developing geometric intuition for dot products.
What you can do after
Evaluate document relatedness and rank retrieval results by computing dot products between their feature vectors.
How it connects
Connects dot product, vector representation, feature space, document retrieval, word frequency, and sparse representation.

Also explained in

Compare how 10 books explain similarity metric →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.