liveBook · Manning

Cross-encoder

As explained in Build an Advanced RAG Application (From Scratch)

A cross-encoder jointly processes paired inputs to compute a direct relevance, similarity, or task-specific output score.

A cross-encoder scores the relationship between two inputs by processing them together. For semantic similarity, it concatenates a query and a document into one sequence, allowing the transformer to model fine-grained interactions between their tokens before producing a task score. This added interaction can improve comparison quality, but it requires more computation than encoding each input separately. section 4.0 section 4.3

Figure 4.3 An Overview of the Cross-Encoder Model to measure the similarity between two sentences.

Use this architecture view to trace the paired sequence from two sentences through BERT to a similarity classifier. section 4.3

For a BERT-style similarity model, place Sentence A and Sentence B in the same input sequence with a [SEP] token between them. BERT produces contextualized token embeddings across the combined sequence; a feedforward classification layer then converts the model output into a similarity score, typically from 0 to 1. Unlike a distance calculation over independently stored vectors, this score is produced from the pair itself. section 4.3

Choose between paired scoring and independent encoding based on the search system's constraints:

  • Use a cross-encoder when fine-grained query-document interaction is important and the extra computation is acceptable. section 4.0
  • Use a bi-encoder when embeddings must be precomputed, stored, and searched efficiently for fast retrieval. section 4.6
  • Do not expect a cross-encoder to yield standalone sentence embeddings for indexing; Sentence Transformers instead creates fixed-size sentence vectors for storage and comparison. section 4.2

Scoring a hotel request against a review

Pair the request “Hotel near the Louvre with great food nearby” with one hotel review. Feed the request and review together, separated by [SEP], through BERT and its classifier. The resulting score expresses the model's similarity judgment for that specific request-review pair. Repeating this operation for more reviews gives each candidate a direct paired score, at corresponding computational cost. section 4.3 section 4.0

Cross-encoders and bi-encoders therefore serve different parts of a semantic-search design. A bi-encoder supports vector-based retrieval through independently generated embeddings, distance metrics, and scalable indexes such as FAISS. A cross-encoder instead evaluates the interaction of a selected pair directly, trading efficiency for a more detailed comparison. section 4.0 section 4.6

How Build an Advanced RAG Application (From Scratch) covers it

How deep it goes
Explains concatenated BERT inputs, [SEP], contextual embeddings, and a classifier, but provides no runnable cross-encoder implementation.
Examples
Illustrates paired-sentence scoring and situates it in hotel-review search for a Louvre-area request.
What it assumes
Assumes familiarity with transformer-based embeddings and introduces the paired BERT input and classification-head pattern.
What you can do after
Evaluate/understand when joint query-document scoring offers finer relationships than independently encoded similarity search.
How it connects
Contrasts cross-encoders with bi-encoders, Sentence Transformers, BERT, cosine distance, Euclidean distance, and FAISS.

Also explained in

Compare how 4 books explain cross-encoder →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build an Advanced RAG Application (From Scratch). The section links point to the parts of the book it draws on.