Sentence-BERT (SBERT) adapts BERT-style models to represent whole sentences as dense vectors. It pools token embeddings into one sentence vector and fine-tunes similarity with the STS-B benchmark, bringing semantically similar sentences closer in vector space. section 13.4.2 section 13.4.3
A typical SBERT semantic-search pipeline is:
- Load a pretrained SentenceTransformer model, such as roberta-base-nli-stsb-mean-tokens. section 13.4.3
- Encode phrases or documents into dense embeddings; the example produces 768-dimensional vectors. section 13.4.3
- Index embeddings with approximate nearest-neighbor search, then encode each incoming query with the same model. section 13.5.4
Semantic similarity and autocomplete
Encoding four phrases ranks “it's raining hard” against “it is wet outside” at 0.669, above unrelated phrase pairs. An ANN autocomplete function can then encode “mountain hike” or “dehyd” and return nearby concepts such as “mountain hiking,” “dehydrate,” and “dehydration.” A 0.75 similarity threshold filters suggestions, but should be tuned against real queries and measured for production throughput. section 13.5.4
Listing 13.6 Encoding phrases as dense vector embeddings
phrases = ["it's raining hard", "it is wet outside", #1
"cars drive fast", "motorcycles are loud"] #1
embeddings = transformer.encode(phrases, convert_to_tensor=True) #2
print("Number of embeddings:", len(embeddings))
print("Dimensions per embedding:", len(embeddings[0]))
print("The embedding feature values of \"it's raining hard\":")
print(embeddings[0])
Inspect the compact Python workflow for loading a model and encoding phrases as dense vectors. section 13.4.3
Listing 13.15 Encoding a query and returning the k-nearest-neighbor concepts
def embedding_search(index, query, phrases, k=20, #1
min_similarity=0.75): #1
matches = []
query_embedding = transformer.encode(query) #2
query_embedding = normalize_embedding(query_embedding)
ids, distances = index.knnQuery(query_embedding, k=k)
for i in range(len(ids)):
similarity = distances[i] * -1 #3
if similarity >= min_similarity: #4
matches.append((phrases[ids[i]], similarity))
if not len(matches):
matches.append((phrases[ids[1]], distances[1] * -1)) #5
return matches
def semantic_suggest(prefix, phrases):
matches = embedding_search(concepts_index, prefix, phrases)
print_labels(prefix, matches)
semantic_suggest("mountain hike", phrases)
semantic_suggest("dehyd", phrases)
Inspect the query function that encodes input, searches nearest neighbors, applies a similarity threshold, and returns suggestions. section 13.5.4
