A search query and a document can each be represented as a vector whose positions correspond to the same features. Their inner product, or dot product, multiplies each pair of corresponding values and adds the results. The resulting score can help rank documents against a query. section 3.1.4 section 13.2.2
When term counts matter
For the query “apple juice,” use [1, 1]. A document with one occurrence of each term has vector [1, 1] and scores 1×1 + 1×1 = 2. A document with three occurrences of “apple” and four of “juice” has vector [3, 4] and scores 1×3 + 1×4 = 7. Cosine similarity instead gives the first document a perfect score because its term proportions exactly match the query. Thus, choosing the dot product changes what “better match” means: higher matching-term counts contribute more. section 3.1.4
Listing 3.5 Ranking text similarity based on term counts
def term_count(content, term): tokenized_content = tokenize(content) term_count = tokenized_content.count(term.lower()) return float(round(term_count, 4)) query = "the cat in the hat" terms = tokenize(query) query_vector = list(numpy.repeat(1, len(terms))) doc_vectors = [[term_count(doc, term) for term in terms] for doc in docs] doc_scores = [dot(v, query_vector) for v in doc_vectors] print_term_count_scores(terms, doc_vectors, doc_scores)
The Python example tokenizes a query, counts each query term in every document, and calls dot(...) to produce document scores. section 3.1.4
Raw counts are not always a desirable ranking signal: a document can score highly by repeating common query words while missing important terms. One illustrated query, “the cat in the hat,” even repeats “the” in its query vector. Replacing raw counts with document-length-normalized, square-root-dampened term frequencies changes the ranking in that example. The dot product combines the feature weights supplied to it; it does not decide which weighting scheme is appropriate. section 3.1.4
For dense embeddings, the same operation can compare vectors of semantic features rather than term counts. Cosine similarity equals the dot product divided by the two vector lengths. If both vectors are unit-normalized, those lengths are one, so a dot product returns the cosine score without calculating vector lengths at query time. Indexed document vectors can be normalized ahead of time; the query vector is normalized before scoring. section 13.2.2 section 2.3
Listing 13.7 Comparing all the phrases to each other
def normalize_embedding(embedding): #1
normalized = numpy.divide(embedding, numpy.linalg.norm(embedding))
return list(map(float, normalized))
normalized_embeddings = list(map(normalize_embedding, embeddings))
similarities = sentence_transformers.util.dot_score(normalized_embeddings,
normalized_embeddings)
print("The shape of the resulting similarities:", similarities.shape)
This implementation normalizes sentence embeddings and passes them to sentence_transformers.util.dot_score for pairwise comparison. section 13.4.3
Pairwise comparison has a scaling cost: comparing every phrase with every other phrase produces an n-by-n score matrix. Dot products also serve a different purpose in matrix factorization, where corresponding user and item latent features are multiplied and summed to predict a preference. In each setting, interpret the score in light of how its vectors were constructed. section 13.4.3 section 9.3.1
