liveBook · Manning

Gleaning loop

As explained in Retrieval Augmented Generation, The Foundational Ideas

A gleaning loop is an iterative extraction procedure in which a model reviews its previous output and adds entities or relationships it missed.

A gleaning loop improves entity and relationship extraction during graph indexing. After an initial LLM pass, it shows the model its own output and requests additional items that were missed, repeating for a fixed number of iterations. section 12.2.1

Recovering overlooked incident details

From a passage about Anna Müller, AuthService, and INC-2347, the first pass may extract entities and relationships; subsequent passes ask the model to add omitted details, improving recall in dense, entity-heavy text. section 12.2.1

The procedure follows a compact indexing pattern:

  • Extract entities with types and descriptions, plus relationships with descriptions and numeric strengths. section 12.2.1
  • Show the model its prior output and prompt it to add missed entities and relationships. section 12.2.1
  • Bind results to source chunks so later graph nodes retain provenance. section 12.2.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains the self-review prompt, repeated passes, extraction schema, and provenance, but not detailed iteration-count selection.
Examples
Uses an incident passage involving Anna Müller, AuthService, and INC-2347 to illustrate recovered entities and relationships.
What it assumes
Assumes chunked corpus passages, an LLM extraction step, and entity-relationship output from an initial pass.
What you can do after
Apply iterative LLM extraction to recover details that a single pass overlooks in dense corpus passages.
How it connects
Connects entity extraction, relationship extraction, provenance, graph indexing, and community summarization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.