liveBook · Manning

Triplet margin loss

As explained in Build an Advanced RAG Application (From Scratch)

Triplet loss is an optimization objective that brings an anchor representation closer to a similar positive representation than to a dissimilar negative representation.

Triplet loss is an optimization objective for training sentence embeddings to represent semantic relationships. It improves a siamese network’s training rather than defining a separate network architecture. section 2.3.1

The objective organizes training around three inputs:

  • Anchor: the reference sentence being represented. section 2.3.1
  • Positive: a sentence similar to the anchor, whose embedding should move closer. section 2.3.1
  • Negative: a dissimilar sentence, whose embedding should move farther from the anchor. section 2.3.1

Shared subnetworks independently encode the sentences, while training minimizes anchor–positive distance and maximizes anchor–negative distance. SBERT uses this arrangement to produce embeddings suited to semantic textual similarity. section 2.3.1

Figure 2.6 A distinction between Bert (on the left) and Sentence-Bert (on the right) lies in their approach to semantic textual similarity. In BERT, the two sentences are initially concatenated and then fed into the network. Subsequently, the similarity is computed using the activation function. Conversely, in Sentence-Bert, the two sentences are independently input into Bert, and their cosine similarity is then calculated.

Compare BERT’s concatenated sentence processing with Sentence-BERT’s independent encoding and cosine-similarity calculation. section 2.3.1

How Build an Advanced RAG Application (From Scratch) covers it

How deep it goes
Defines the triplet inputs and distance objective, but does not provide an implementation or detailed loss derivation.
Examples
Uses triplets of an anchor sentence, a similar positive sentence, and a dissimilar negative sentence.
What you can do after
Evaluate how training separates similar and dissimilar sentence representations for semantic textual similarity.
How it connects
Connects siamese networks, sentence embeddings, semantic textual similarity, BERT, and Sentence-BERT.

Also explained in

Compare how 2 books explain triplet margin loss →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build an Advanced RAG Application (From Scratch). The section links point to the parts of the book it draws on.