Suppose a question must be matched to the most useful sentence in a collection. Represent each text as a vector, then compare how strongly the vectors point in the same direction. Cosine similarity makes that comparison without treating a longer vector as more similar simply because it has a greater magnitude. section 3.3.1 section 3.4
For two nonzero vectors A and B, cosine similarity is A·B / (||A|| × ||B||). Multiply corresponding values and add them to get the dot product. Each norm is the square root of the sum of that vector’s squared values. Dividing by both norms produces a score from −1 to 1: 1 means the same direction, 0 means orthogonal directions, and −1 means opposite directions. Unlike Euclidean distance, this comparison focuses on angle rather than the length of the difference between vectors. section 3.3 section 3.3.1
Listing 3.11 Computing cosine similarity in Python
>>> import math >>> def cosine_sim(vec1, vec2): ... dot_prod = 0 ... for x1, x2 in zip(vec1, vec2): ... dot_prod += x1 * x2 ... ... mag_1 = math.sqrt(sum([x1**2 for x1 in vec1])) ... mag_2 = math.sqrt(sum([x2**2 for x2 in vec2])) ... ... return dot_prod / (mag_1 * mag_2)
A short Python implementation exposes each step: pairwise multiplication, two vector norms, and division by their product. Neither input should have a zero norm. section 3.3.1
Find a candidate answer
Turn “What is algorithmic bias?” and a collection of sentences into count vectors, then compare the question vector with every sentence using scikit-learn’s cosine_similarity. One sentence scores about 0.433 and ranks highest, but its definition is not necessarily the best answer. Inspect the returned text; changing the vectorization pipeline, such as counting 2-grams, may improve the ranking. section 3.4
The same calculation works beyond token counts. gensim’s word_vectors.similarity('princess', 'queen') returns about 0.707 for word embeddings. For many sentence embeddings, first divide each vector by its norm; a dot product of the normalized matrix with its transpose then gives a square pairwise similarity matrix. section 6.3.4 section 6.4.6
Listing 6.18 Getting the similarity or affinity matrix
>>> np_array_sent_vecs_norm = np.array(sent_vecs) >>> similarity_matrix = np_array_sent_vecs_norm.dot( ... np_array_sent_vecs_norm.T) #1
The NumPy matrix multiplication computes every pairwise cosine similarity at once after the sentence vectors have been normalized. section 6.4.6
Spot related or repeated passages
A similarity matrix can supply edges for a graph of closely related sentences. It can also become a heatmap of transformer-generated passage embeddings: scores above roughly 0.85 are candidates for redundant wording. Inspect apparent matches before acting on them—a bright cluster in the heatmap came from similar machine-generated training-output lines, not necessarily repeated prose. section 6.4.6 section 11.2.3
