Asynchronous index refresh rebuilds a retrieval index while model training continues. It becomes necessary when document-encoder updates change document embeddings, making the existing index stale. section 4.4
The architecture separates ongoing training from periodic index rebuilding. It requires dedicated infrastructure, synchronization to preserve consistency, substantial compute, and custom distributed-training code. section 4.4
The main design tradeoff is whether encoder updates justify refresh complexity:
- REALM updated the document encoder, query encoder, and generator together, forcing periodic refreshes of its document index. section 4.4
- RAG avoided this cost by freezing the document encoder and fine-tuning only what was necessary, making training more practical for small teams. section 4.4
A costly refresh cycle
REALM indexed 13 million Wikipedia passages and reportedly required 400 TPU hours just to refresh its document index during training, making the approach impractical for many teams. section 4.4
