liveBook · Manning

L2 norm

As explained in LLM Evaluation and Alignment, The Foundational Ideas

The L2 norm is the Euclidean length of a vector, calculated as the square root of the sum of the squares of its components.

The L2 norm, or Euclidean norm, measures a vector’s length by taking the square root of the sum of its squared components. In embedding-based evaluation, that magnitude may reflect sentence length or word frequency rather than meaning, so cosine similarity often focuses on direction instead. section 3.2.3

Why magnitude can mislead

The sentences “the weather is freezing” and “it is cold today” can express similar meaning despite differing vector magnitudes. Euclidean distance treats the straight-line gap as important, while cosine similarity compares their directions. section 3.2.3

Figure 3.6 Geometric representation of two semantic vectors. Note that while the vectors have different lengths (magnitudes), they point in roughly the same direction, indicating a shared semantic topic.

Compare vectors with different lengths but similar directions to see why semantic similarity can depend more on orientation than magnitude. section 3.2.3

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines the square-root-of-squared-components calculation and explains why magnitude can mislead in high-dimensional embeddings.
Examples
Uses weather-related sentences and geometric vectors with different lengths but similar directions.
What you can do after
Evaluate when vector magnitude may distort semantic comparisons and understand why cosine similarity can be preferred.
How it connects
Connects vector magnitude to embeddings, Euclidean distance, cosine similarity, semantic metrics, and the curse of dimensionality.

Also explained in

Compare how 7 books explain l2 norm →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.