Cosine similarity measures the angle between two non-zero vectors while ignoring their lengths. Vectors pointing in the same direction score 1, perpendicular vectors score 0, and opposite vectors score -1. For text embeddings, a high score indicates that two texts point toward a similar semantic direction. section 3.2.2 section 3.2.3
A simple embedding-based evaluation workflow is:
- Look up an embedding vector for each word in the candidate and reference text. section 3.2.2
- Average the word vectors to create one sentence vector for each text. section 3.2.2
- Compute cosine similarity between the two sentence vectors instead of requiring identical words. section 3.2.2 section 3.2.3
- Interpret the score cautiously: averaging static vectors loses word order, syntax, and context, and the word bank receives the same vector in financial and river contexts. section 3.2.2
Paraphrases with different vocabulary
The attorney filed the brief and The lawyer submitted the document can receive high similarity because related words occupy nearby regions in embedding space. Likewise, The weather is freezing and It is cold today can be semantically close despite having no lexical overlap. section 3.2.2 section 3.2.3
Listing 3.3 Calculating cosine similarity .
python
def cosine_similarity(emb1, emb2):
“””Compute cosine similarity between two embeddings”””
return torch.nn.functiona.cosine_similarity(emb1, emb2).item()
passage1 = “The committee approved the proposal yesterday after extensive debate."
passage2 = "Yesterday, following lengthy discussions, the proposal received committee approval."
emb1 = get_embeddings(passage1)
emb2 = get_embeddings(passage2)
sim_1_2 = cosine_similarity(emb1, emb2)
Provides a PyTorch implementation that computes cosine similarity between embeddings of two passages. section 3.2.2
























