liveBook · Manning

Hard negative example

As explained in Retrieval Augmented Generation, The Foundational Ideas

A hard negative is a candidate passage that appears plausible for a query but is labeled irrelevant during ranking training.

A hard negative is a candidate passage that looks plausible for a query but is labeled irrelevant during ranking training. Unlike an obviously unrelated passage, it challenges the retriever to learn a sharper relevance boundary. section 3.4 section 15.3.2

Hard negatives strengthen retrieval and ranking signals by:

  • Exposing the model to diverse question-document pairings within larger effective batches. section 3.4
  • Providing realistic distractors instead of only easy irrelevant passages. section 15.3.2
  • Supporting unified ranking and generation examples with explicit relevance targets. section 15.3.2

BM25 distractors in RankRAG

Combine domain question-answering data with passages retrieved by BM25. Mark the answer-bearing passage as relevant and use plausible retrieved alternatives as hard negatives. A rank_all example then asks the model to identify every relevant passage, making relevance supervision part of the same prompt-context-target structure used for generation. section 15.3.2

Listing 15.3 Formatting unified ranking-generation training data.
def format_rankrag_example(
    question: str,
    passages: list[dict],
    answer: str,
    task: str = "qa"
) -> dict:
    """Format a training example for unified rank+generate."""
    context = "\n".join(  #A
        f"Passage {i+1}: {p['text']}"
        for i, p in enumerate(passages)
    )
    if task == "qa":  #B
        prompt = (
            f"Answer the following question from context."
            f"\n{question}"
        )
        target = answer
    elif task == "ranking":  #C
        prompt = (
            f"For the question {question}, assess whether "
            f"the passage is relevant to the question."
        )
        target = "True" if passages[0].get("relevant") else "False"
    elif task == "rank_all":  #D
        prompt = (
            f"For the question {question}, find all passages "
            f"from the context that are relevant."
        )
        target = ", ".join(
            f"Passage {i+1}"
            for i, p in enumerate(passages) if p.get("relevant")
        )
    return {"prompt": prompt, "context": context, "target": target}
 
example = format_rankrag_example(
    question="What is the claims processing deadline?",
    passages=[
        {"text": "Claims must be filed within 30 days.",
         "relevant": True},
        {"text": "Our office hours are 9-5.", "relevant": False},
    ],
    answer="Claims must be filed within 30 days.",
    task="rank_all",  #E
)
print(example["prompt"])
print(example["target"])

Inspect the Python formatter that turns question, passage, answer, and relevance labels into QA, single-passage ranking, or rank_all training examples. section 15.3.2

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains their training role and construction through BM25 retrieval, but does not detail hard-negative mining algorithms.
Examples
Uses BM25-retrieved passages, question-answering pairs, and rank_all prompts with explicit relevant-passage targets.
What it assumes
Assumes familiarity with retrieval, relevance labels, question answering, and training examples.
What you can do after
Apply it via BM25 to create realistic ranking distractors for unified ranking-generation training data.
How it connects
Connects hard negatives to information retrieval, ranking, retriever training, question answering, and retrieval-augmented generation.

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.