An embedding is a learned representation that maps a discrete object, such as a word, to a continuous vector: an ordered list of numbers locating it in a high-dimensional space. Words used in similar contexts tend to occupy nearby regions, turning some semantic relationships into geometric relationships. section 3.2.1
Meaning beyond word overlap
“The attorney filed the brief” and “The lawyer submitted the document” use different words but convey similar meanings. Embeddings can place related words and sentences near one another, allowing semantic comparison where lexical matching misses the overlap. section 3.2.2 section 3.2.3
One approach to comparing sentences is to look up each word’s vector, average the vectors for each sentence, and calculate cosine similarity between the results. Cosine similarity compares vector directions rather than lengths: identical directions score 1, perpendicular directions 0, and opposite directions −1. section 3.2.2
Static word embeddings assign a word the same vector in every context, so they cannot distinguish a financial bank from a river bank. Averaging also discards word order and syntax. Contextual embeddings instead change with surrounding words: “run” can have different representations in “run a company” and “run a marathon.” section 3.2.2 section 3.2.3
Figure 3.2 Both max pooling and mean pooling are methods to aggregate information across many vectors. Max pooling selects (pools) the maximum values from input embeddings; this weighs towards the most prominent feature. Mean pooling takes the mean (average) of input embeddings to represent features equally.
Mean pooling averages vectors, while max pooling selects the maximum value in each position. These are different ways to aggregate information across embeddings. section 3.2.2
Embeddings support learned evaluation metrics, but the comparison method matters. BERTScore uses contextual embeddings to assess semantic similarity against a reference. COMET creates sentence embeddings for the source, candidate, and reference, then uses their interactions to predict quality. A similarity score should be interpreted in the context of the evaluation rather than treated as a universal measure of quality. section 3.2 section 3.2.3 section 3.3.3







































































