liveBook · Manning

Asynchronous MIPS refresh

As explained in Retrieval Augmented Generation, The Foundational Ideas

Asynchronous MIPS refresh is an index-maintenance strategy that rebuilds document embeddings and search indexes periodically while training continues on a stale index.

When a retriever learns alongside a language model, its document embeddings change after gradient updates. A search index built from earlier embeddings quickly becomes stale, but rebuilding an index over millions of documents after every step is impractical. Asynchronous Maximum Inner Product Search (MIPS) refresh keeps training moving while the index catches up. section 3.2

Two jobs run in parallel:

  • The trainer continuously updates the model using a slightly stale search index. section 3.2
  • An index builder periodically recomputes all document embeddings and supplies a fresh index to the trainer. section 3.2

A 500-step refresh

The index builder might refresh the index every 500 training steps rather than after every update. Training proceeds in the meantime. This trades temporary index staleness for a tractable way to jointly pretrain the retriever and language model at scale; it does not make that pretraining inexpensive. section 3.2 section 3.5

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Describes parallel trainer and index-builder jobs and an example refresh interval, but does not specify index-builder implementation.
Examples
Uses a trainer working against a slightly stale index while a second job rebuilds it, potentially every 500 steps.
What you can do after
Evaluate/understand it as a tradeoff between stale retrieval results and the cost of rebuilding an index after every update.
How it connects
Connects MIPS indexing with document embeddings, retriever training, masked language modeling, and retrieval-augmented pretraining.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.