liveBook · Manning

Cosine similarity

As explained in Build an Advanced RAG Application (From Scratch)

Cosine similarity measures the cosine of the angle between two vectors to quantify their directional relatedness.

Cosine similarity measures the angle between two non-zero vectors. In semantic search, it compares a query embedding with document embeddings so results can match meaning beyond exact words. section 3.4 section 3.5

Scores range from -1 to 1: 1 indicates the same direction, -1 the opposite direction, and 0 orthogonality or decorrelation. Higher similarity scores indicate more closely related embeddings; SciPy's cosine distance reverses this ordering because smaller distances indicate greater similarity. section 3.4

A practical semantic-search implementation follows this sequence:

  • Encode each document into a dense vector with an embedding model. section 3.4
  • Encode the incoming query into a vector in the same space. section 3.4
  • Compute cosine similarity or cosine distance between the query and every document embedding. section 3.4
  • Rank documents by highest similarity, or equivalently by lowest cosine distance. section 3.4

Toy semantic search

For the query “Machine learning,” six example sentences receive scores including 0.6723 for “Machine learning is based on algorithms,” 0.4380 for “Deep learning uses neural networks,” and 0.3609 for “Recurrent networks have connections.” The highest-scoring document is returned. section 3.4

Figure 3.4 A graphical demonstration of cosine similarity using two dimensions ("pet" and "furry"). Three words are drawn as vectors into the coordinate system. θ 1 denotes the angle between bird and dog, θ 2 depicts the angle between dog and cat, respectively. The smaller angle θ 2 illustrates a higher similarity of dogs with cats compared to birds.’

Visualizes cosine similarity as the angle between vectors: the smaller angle between dog and cat represents greater similarity than the larger angle between bird and dog. section 3.4

How Build an Advanced RAG Application (From Scratch) covers it

How deep it goes
Explains vector angles, score interpretation, cosine distance, and retrieval code, but does not derive the formula mathematically.
Examples
Uses pet-related vector diagrams, capital-country embeddings, and a six-sentence machine-learning retrieval example.
What it assumes
Assumes basic vector and embedding familiarity while defining cosine similarity and its role in semantic search.
What you can do after
Apply it via Sentence Transformers and SciPy to rank documents by semantic relatedness.
How it connects
Connects semantic search, embeddings, Sentence Transformers, TF-IDF, Euclidean distance, Bi-Encoder models, Cross-Encoder models, and FAISS.

Also explained in

Compare how 25 books explain cosine similarity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build an Advanced RAG Application (From Scratch). The section links point to the parts of the book it draws on.