A similarity metric measures how closely two patterns or signals resemble each other. For represented text, it compares feature, topic, document-context, or embedding vectors; higher similarity generally means the represented meanings are closer. section 4.6
The choice of metric changes what “close” means. Common options include Euclidean, squared Euclidean, cosine, Minkowski, Manhattan, Jaccard, Mahalanobis, and Levenshtein distance. Some applications convert distance to similarity, but the conversion must match the scale and interpretation required. section 4.6
Comparing words and search results
A gensim word-vector model can calculate cosine similarity between “princess” and “queen,” returning 0.70705315983704509. The same vector space can rank documents by comparing an embedded query with embedded documents. For large collections, approximate nearest-neighbor search avoids comparing the query with every stored vector. section 6.3.4 section 10.3.3
Listing 4.12 Pairwise distances available in sklearn
'cityblock', 'cosine', 'euclidean', 'l1', 'l2', 'manhattan', 'braycurtis', 'canberra', 'chebyshev', 'correlation', 'dice', 'hamming', 'jaccard', 'kulsinski', 'mahalanobis', 'matching', 'minkowski', 'rogerstanimoto', 'russellrao', 'seuclidean', 'sokalmichener', 'sokalsneath', 'sqeuclidean', 'yule'
Use the available scikit-learn pairwise metrics as a practical menu when selecting a distance or similarity basis for vector comparisons. section 4.6
