A cross-encoder scores a query and document together. It accepts them as one input pair, processes the pair through all layers of a transformer, and outputs a single relevance score. This lets it consider interactions between their terms: a query about a work accommodation can rank a document about ADA highly even when the wording differs. section 4.3.3
That precision has a cost. A bi-encoder embeds queries and documents separately, so document vectors can be precomputed and compared efficiently. A cross-encoder must process each query–document pair jointly and is significantly slower. Use it to rerank top-k candidates from earlier retrieval stages, not to search the entire corpus. section 4.3.3 section 4.3.4
In an employee-policy retriever, the stages have distinct jobs:
- BM25 retrieves documents using keyword matches, including exact terms such as “resignation” and “California.” section 4.3.4
- E5 embeddings retrieve by semantic similarity, connecting wording such as “quitting” and “resignation.” section 4.3.4
- A cross-encoder scores the resulting query–document pairs so the most relevant policy information rises to the top. section 4.3.4
from rank_bm25 import BM25Okapi #1
from sentence_transformers import SentenceTransformer, CrossEncoder #1
from sklearn.metrics.pairwise import cosine_similarity #1
import numpy as np #1
documents = [ #2
"California employees must give a 30-day notice before resignation.",
"All new hires are eligible for health insurance after 90 days.",
"Termination for cause may lead to loss of severance benefits.",
"Employees may request remote work accommodations under ADA.",
"Maternity and adoption leave are fully paid for 12 weeks."
]
tokenized_corpus = [doc.lower().split() for doc in documents] #3
bm25 = BM25Okapi(tokenized_corpus) #3
dense_model = SentenceTransformer("intfloat/e5-base") #4
dense_embeddings = dense_model.encode(documents, normalize_embeddings=True) #5
cross_encoder = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2") #6
The Python setup initializes BM25, precomputes normalized E5 document embeddings, and loads `CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")` for the final scoring stage. section 4.3.4
Checking the final ranking
For “How much notice do I need before quitting in California?”, the sample retriever ranks the policy requiring 30-day notice before resignation first. Other sample queries put paid adoption leave and termination-for-cause policies first. These outputs illustrate reranking; the scores alone do not establish reliability on other queries. section 4.3.4
