RAG and fine-tuning are often treated as alternatives: retrieval supplies changing facts, while fine-tuning shapes task behavior. RankRAG combines them because judging passage relevance and extracting an answer depend on the same query–context understanding. section 15.3
A standard pipeline separates retrieval, reranking, and generation, allowing misalignment between relevance scores and passages that actually support an answer. RankRAG collapses reranking and generation into one instruction-tuned LLM. section 15.3.1
At inference time, RankRAG follows three steps:
- Retrieve a large candidate set with a bi-encoder such as DPR or a sparse model such as BM25. section 15.3.1
- Score each query–passage pair as True or False to identify relevant passages. section 15.3.1
- Generate the answer using only the highest-scoring passages. section 15.3.1
Training recipe
Stage I uses supervised fine-tuning on conversations, chain-of-thought reasoning, and long-form question answering. Stage II adds context-rich QA, retrieval-augmented QA with gold and hard-negative passages, MS MARCO context-ranking data, and mixed-set retrieval-augmented ranking. Roughly 50,000 ranking pairs alongside 128,000 total examples were sufficient to outperform dedicated ranking models trained on ten times more ranking data. section 15.3.1
