A non-parametric approach separates a model’s reasoning engine from an external, updatable knowledge base. Updating stored knowledge can then avoid expensive model retraining. section 6.0
In retrieval-augmented generation, the retriever finds relevant documents and the reader uses them to generate an answer. Joint training lets reader performance directly teach the retriever which documents support correct answers. section 6.0
This approach is useful when knowledge changes or training data is scarce:
- Keep knowledge current through an external document index rather than repeatedly retraining model parameters. section 6.0
- Use aligned retrieval and reading to support few-shot performance with smaller models. section 6.0
- Adapt query behavior through query-side fine-tuning without re-indexing the entire corpus. section 6.0
Atlas
Atlas applies this design to few-shot retrieval-augmented language modeling, combining an external knowledge source with jointly trained retrieval and reading components. section 6.0
