Information retrieval

As explained in Building Reliable AI Systems

Retrieval is the process of finding relevant information from an external collection to supply as model context.

Information retrieval finds relevant information in an external collection and supplies it as model context. In retrieval-augmented generation, the system embeds documents and a user query, then uses similarity search to surface nearby document vectors. If relevant context is missed, a capable model and accurate source documents may still produce vague or outdated answers. section 4.0 section 4.1.1

Insurance policy chatbot

A patient asks whether physical therapy after surgery is covered. The answer exists in the policy documents, but the chatbot fails to find it; the retrieval pipeline, particularly its embedding model, is identified as the problem. section 4.0

Figure 4.1 Document embedding in RAG systems

Trace how documents and a user query are embedded before similarity search retrieves candidate context. section 4.0

How Building Reliable AI Systems covers it

How deep it goes

Traces document and query embedding through vector storage and similarity search, without detailing search algorithms.

Examples

Uses a health-insurance chatbot that misses policy coverage information despite having access to the documents.

What you can do after

Diagnose how embedding quality can cause a retrieval pipeline to miss relevant context.

How it connects

Connects retrieval to retrieval-augmented generation, embeddings, vector databases, and similarity search.

liveBookmanning.com

Read Building Reliable AI Systems on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Introduction to Generative AI, Second EditionRetrieval is the process of finding relevant information from an external collection in response to a query or representation.
• 

  liveBookmanning.com
  Effective Conversational AIThe retrieval phase is the stage of a retrieval-augmented generation process that finds and obtains information relevant to an input.
• 

  liveBookmanning.com
  Grokking AI ApplicationsThe process of finding and obtaining relevant information from a data source for use in an AI application.
• 

  liveBookmanning.com
  MongoDB 8.0 in Action, Third EditionInformation retrieval is the process of finding relevant information from a collection of stored data in response to an information need.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionFull-text search retrieves documents by searching their complete textual contents rather than only structured fields.
• 

  liveBookmanning.com
  Microsoft Azure in ActionRetrieval is the process of selecting relevant stored information to provide as context for a language model.
• 

  liveBookmanning.com
  Elasticsearch in Action, Second EditionSearching textual content using tokenization, matching, relevance scoring, or related linguistic analysis.
• 

  liveBookmanning.com
  Data Storytelling with Altair and AIInformation retrieval is the process of searching for and returning relevant information from a collection or external knowledge source.
• 

  liveBookmanning.com
  Build a Website with ChatGPTSearch is the process of finding records whose data matches a user-supplied word or phrase.
• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningInformation retrieval is the field concerned with finding and ranking relevant information items in response to queries.

Compare how 11 books explain information retrieval →

Related concepts

• retrieval-augmented generationused by · Retrieval-augmented generation is a technique that retrieves relevant information from external sources and supplies it as context to a generative model when producing an output.
• hybrid searchused by · Hybrid search is an information-retrieval technique that combines multiple retrieval methods or relevance signals to find or rank relevant results.
• similarity searchincludes · Similarity search is a retrieval technique that finds or ranks items by comparing their representations with a query according to a similarity or distance measure.
• chunkingused by · Chunking is the process of dividing content or data into smaller units for processing, retrieval, or organization.
• agentic RAGused by · Agentic RAG is a retrieval-augmented generation approach in which an agent dynamically decides how and when to retrieve information, often iterating or using tools before generating a response.

This explainer was generated by Manning's knowledge graph from the text of Building Reliable AI Systems. The section links point to the parts of the book it draws on.