liveBook · Manning

Cosine similarity

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Cosine similarity measures the orientational similarity of two non-zero vectors by calculating the cosine of the angle between them, with values from -1 to 1.

Cosine similarity measures the angle between two non-zero vectors while ignoring their lengths. Vectors pointing in the same direction score 1, perpendicular vectors score 0, and opposite vectors score -1. For text embeddings, a high score indicates that two texts point toward a similar semantic direction. section 3.2.2 section 3.2.3

A simple embedding-based evaluation workflow is:

  • Look up an embedding vector for each word in the candidate and reference text. section 3.2.2
  • Average the word vectors to create one sentence vector for each text. section 3.2.2
  • Compute cosine similarity between the two sentence vectors instead of requiring identical words. section 3.2.2 section 3.2.3
  • Interpret the score cautiously: averaging static vectors loses word order, syntax, and context, and the word bank receives the same vector in financial and river contexts. section 3.2.2

Paraphrases with different vocabulary

The attorney filed the brief and The lawyer submitted the document can receive high similarity because related words occupy nearby regions in embedding space. Likewise, The weather is freezing and It is cold today can be semantically close despite having no lexical overlap. section 3.2.2 section 3.2.3

Listing 3.3 Calculating cosine similarity .
python
def cosine_similarity(emb1, emb2):
    “””Compute cosine similarity between two embeddings”””
    return torch.nn.functiona.cosine_similarity(emb1, emb2).item()
 
passage1 = “The committee approved the proposal yesterday after extensive debate."
passage2 = "Yesterday, following lengthy discussions, the proposal received committee approval."
 
emb1 = get_embeddings(passage1)
emb2 = get_embeddings(passage2)
sim_1_2 = cosine_similarity(emb1, emb2)

Provides a PyTorch implementation that computes cosine similarity between embeddings of two passages. section 3.2.2

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines orientation-based scoring, demonstrates embedding construction, and stops short of solving contextual meaning and word order.
Examples
Uses attorney–lawyer paraphrases, weather–freezing paraphrases, and committee-approval sentences to illustrate semantic closeness.
What it assumes
Assumes basic vectors and embeddings while defining vector magnitude, orientation, and semantic comparison through examples.
What you can do after
Apply it via PyTorch to compare embedding vectors for semantically related passages.
How it connects
Connects embedding vectors with semantic similarity, semantic search, Euclidean distance, L2 norm, BERTScore, and lexical metrics.

Also explained in

Compare how 25 books explain cosine similarity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.