Cosine similarity compares the directions of two vectors by measuring the angle between them. In embedding-based search, vectors pointing in similar directions represent text with related meaning, even when the wording differs. section 4.1
For semantic retrieval, the metric has a practical distinction from dot product:
- Cosine similarity focuses on vector direction and is scale-invariant. section 4.1
- Dot product also factors in vector magnitude. section 4.1
Paraphrased search queries
“How do I get reimbursed for therapy?” and “What’s the process for submitting a mental health claim?” express similar intent despite sharing few words. Their embeddings can be close in vector space, supporting semantic retrieval. A health-care policy assistant also configures cosine as its Pinecone index metric. section 4.1 section 4.5.4
from pinecone import Pinecone, ServerlessSpec
import os
api_key = os.environ.get("PINECONE_API_KEY") #1
pc = Pinecone(api_key=api_key, environment="us-west1-gcp") #1
index_name = "healthcare-policies" #2
vector_dimension = embeddings.shape[1] #3
if index_name not in pc.list_indexes(): #4
pc.create_index( #4
name=index_name, #4
dimension=vector_dimension, #4
metric="cosine" #4
) #4
index = pc.Index(index_name) #5
The index setup specifies metric="cosine" alongside the embedding dimension. section 4.5.4
