liveBook · Manning

Embedding

As explained in LLM Customization and Fine-Tuning

A numerical vector representation of an item that encodes features useful for computational comparison.

An embedding represents an item as a numerical vector. For text retrieval, a sentence-transformer maps each chunk to a fixed-length array of floats, placing semantically related chunks near one another in vector space. A query embedded in the same space can be compared with those chunks to find relevant results. section 4.5.3

Cosine similarity compares vector direction while ignoring magnitude. For normalized vectors, cosine similarity is their dot product. A simple retrieval implementation stores chunk embeddings as rows in a NumPy array, multiplies the array by the query vector, and sorts for the top results. section 4.5.3

Choose an embedding model with the retrieval workload in mind:

  • For many enterprise corpora under a million chunks, all-MiniLM-L6-v2 produces 384-dimensional embeddings and runs on CPU; smaller vectors are faster to compute and compare and cheaper to store. section 4.5.3
  • Larger vectors, such as the 768-dimensional output of all-mpnet-base-v2, can preserve more semantic nuance at greater compute and storage cost. section 4.5.3

Detecting drift in production text

Embed texts from a reference set and a production set, then average each set’s vectors to form two centroids. The drift score is one minus their cosine similarity: zero means the centroids have the same direction, while one means they are orthogonal. A sentence-transformer can replace TF-IDF vectorization without changing the centroid-and-cosine calculation. section 9.2.2

Embeddings also matter inside a model’s vocabulary. If a tokenizer fragments specialized terms and its embedding layer stays frozen, LoRA adapters cannot fix the vocabulary mismatch. In a medical summarization study, adding domain tokens and training their embeddings before fine-tuning improved results; another option is to start with a domain-pretrained model such as BioBERT. section 5.2.4

How LLM Customization and Fine-Tuning covers it

How deep it goes
Explains vector similarity, dimensionality tradeoffs, centroid drift detection, and frozen embedding-layer limits without deriving embedding training.
Examples
Uses NumPy top-k retrieval, reference and production text centroids, and medical vocabulary expansion as examples.
What you can do after
Evaluate embedding choices for retrieval and understand how embeddings support drift detection and vocabulary adaptation.
How it connects
Connects embeddings to cosine similarity, retrieval, drift detection, tokenization, and LoRA.

Also explained in

Compare how 72 books explain embedding →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.