liveBook · Manning

Sentence-transformer embedding

As explained in LLM Customization and Fine-Tuning

A transformer model that maps sentences to vector representations designed to capture semantic similarity.

A sentence-transformer maps sentences or text chunks to fixed-length dense vectors. Semantically similar text lands close in vector space, allowing paraphrases to match even when they share few words. section 4.5.3 section 4.3.5

For retrieval, encode the query into the same space and find nearby vectors. Cosine similarity is a standard measure; dot product suits encoders trained to produce unit-length vectors. section 4.5.3

Choose embedding size with retrieval quality and operating cost in mind:

  • 384 dimensions are faster to compute and compare and cheaper to store; all-MiniLM-L6-v2 is a practical default for many enterprise corpora under one million chunks. section 4.5.3
  • 768- or 1024-dimensional vectors can preserve more semantic nuance, with higher compute and storage costs. section 4.5.3

Selecting many-shot examples

Encode an example bank once, then compare each query with its vectors to select relevant demonstrations. Precompute and cache bank embeddings in production rather than re-encoding the full bank for every request. section 4.3.5

Listing 4.2 Many-shot prompt assembly (many_shot_demo.py)
# code/chapter04/many_shot_demo.py (key sections -- full source in code/chapter04/many_shot_demo.py)
 
import json
from pathlib import Path
from sentence_transformers import SentenceTransformer
import numpy as np
 
EMBEDDER = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
 
def load_example_bank(path: Path) -> list[dict]:
    """Load a JSONL example bank with `input`, `output`, and optional `category`."""
    with open(path) as f:
        return [json.loads(line) for line in f if line.strip()]
 
# Re-encodes the static bank each call for clarity; in production
# precompute and cache these vectors once (or use a vector DB).
def select_by_similarity(bank: list[dict], query: str, k: int) -> list[dict]:
    """Pick the k examples whose `input` is most similar to the query."""
    bank_embeddings = EMBEDDER.encode([ex["input"] for ex in bank],
                                      normalize_embeddings=True)
    query_embedding = EMBEDDER.encode([query],                      
                                      normalize_embeddings=True)[0]
    scores = bank_embeddings @ query_embedding
    # most similar last (recency rule)
    top_idx = np.argsort(-scores)[:k][::-1]  
    return [bank[i] for i in top_idx]
 
def assemble_prompt(bank: list[dict], query: str, k: int = 50) -> str:
    """Build a many-shot prompt: instruction, k selected examples, then the query."""
    selected = select_by_similarity(bank, query, k)
    parts = ["Classify the support ticket into one of twelve categories.\n"]
    for ex in selected:
        parts.append(f"Ticket: {ex['input']}\nCategory: {ex['output']}\n")
    parts.append(f"Ticket: {query}\nCategory:")
    return "\n".join(parts)

See the sentence-transformers implementation for encoding a JSONL example bank and selecting demonstrations by similarity. section 4.3.5

How LLM Customization and Fine-Tuning covers it

How deep it goes
Explains vector geometry, similarity metrics, and dimensionality tradeoffs, but does not detail model training.
Examples
Uses all-MiniLM-L6-v2 and all-mpnet-base-v2 embeddings, plus similarity-based selection from a many-shot example bank.
What you can do after
Apply sentence-transformers to encode text and rank semantically relevant chunks or demonstrations.
How it connects
Connects sentence-transformer embeddings to vector retrieval, RAG, vector databases, and many-shot prompting.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.