SimRAG adapts a general-purpose language model to specialized retrieval-augmented generation when labeled domain questions are unavailable. It generates training questions from unlabeled documents and keeps only question–answer pairs whose answers appear in retrieved passages. section 15.4.3 section 9.5
The self-improvement pipeline has two stages:
- Stage I teaches retrieval-oriented skills, including instruction following, context-grounded question answering, passage-based answer generation, and question generation from answer–passage pairs. section 15.4.3
- Stage II extracts candidate answer spans from domain documents and generates questions that those spans answer. section 15.4.3
- A round-trip consistency check retains a pair only when its answer appears in the top-k passages retrieved for the generated question. section 15.4.3
- The surviving synthetic data is mixed with Stage I data for continued fine-tuning, improving domain-specific use of retrieved context. section 15.4.3
Medical literature
A model can generate questions from an unlabeled medical corpus, retain only answers recoverable from retrieved papers, and fine-tune on those grounded examples. Across eleven datasets, reported gains range from 1.2% to 8.6%, averaging 8.01% over the Llama baseline in the medical domain. section 9.5
