liveBook · Manning

Information retrieval

Information retrieval is the process of finding and selecting information relevant to a query or information need from a collection.

How 17 Manning books explain it

Introduction to Generative AI, Second Edition

Retrieval is the process of finding relevant information from an external collection in response to a query or representation.

How it is used
Uses retrieval to ground language-model responses in external information beyond static training data.
How deep it goes
Explains query, retrieval, summarization, and vector similarity at an overview level without implementation details.
Examples
Illustrates retrieval with a factual search query and vegan restaurant recommendations using previous preferences and Yelp reviews.
What you can do after
Evaluate and understand how retrieved sources can support an answer while recognizing that queries, summaries, and sources require checking.

Read how Introduction to Generative AI, Second Edition explains information retrieval →

Vibe Engineering

Retrieval is the process of selecting relevant information from an indexed or accessible knowledge source.

How it is used
Supplies coding agents with relevant repository, documentation, or tool context without loading everything at once.
How deep it goes
Explains embedding-based indexing and retrieval, contrasts live code search, and gives manual scoping and source-quality checks.
Examples
Uses an authentication handler and tests, file-picker documentation, and deferred tool discovery as retrieval scenarios.
What you can do after
Operate/configure retrieval by scoping context manually, curating sources, and checking critical tool discovery.

Read how Vibe Engineering explains information retrieval →

Effective Conversational AI

The retrieval phase is the stage of a retrieval-augmented generation process that finds and obtains information relevant to an input.

How it is used
Finds document passages for nuanced, long-tail chatbot questions that predefined intents may not answer well.
How deep it goes
Explains retrieval through document preparation, chunking, embeddings, and vector storage, while distinguishing retrieved passages from generated answers.
Examples
Contrasts a generic side-effects answer with passages addressing a vaccine, ibuprofen, pain, and blood-pressure medication.
What you can do after
Evaluate/understand when retrieved passages suffice and when RAG better reduces the work of assembling an answer.

Read how Effective Conversational AI explains information retrieval →

Retrieval Augmented Generation, The Foundational Ideas

Retrieval finds candidate documents or passages from a knowledge base in response to a query.

How it is used
Fetches candidate evidence for ranking and grounded answer generation, including information unavailable in a model’s training data.
How deep it goes
Explains offline indexing and online candidate retrieval, then surveys search techniques without specifying a complete implementation here.
Examples
Uses a warranty question, changing organizational knowledge, and a historical retrieval timeline to distinguish search needs and approaches.
What you can do after
Evaluate/understand whether missing content calls for indexing or retrieval changes and whether buried candidates call for ranking changes.

Read how Retrieval Augmented Generation, The Foundational Ideas explains information retrieval →

Grokking AI Applications

The process of finding and obtaining relevant information from a data source for use in an AI application.

How it is used
Supplies relevant document chunks to answer generation when a RAG user asks a question requiring stored knowledge.
How deep it goes
Describes selecting a few candidate chunks from a vector database and illustrates distance-based matching.
Examples
Uses a question matched against six pet-related chunks, with Euclidean distances showing which chunks are closest.
What you can do after
Understand how retrieved document chunks provide extra knowledge for an LLM’s generated answer.

Read how Grokking AI Applications explains information retrieval →

AI Recommender Systems

A candidate-generation technique that finds potentially relevant items or documents from a larger collection.

How it is used
Generates a broad candidate pool before filtering, scoring, and final ordering in recommender pipelines.
How deep it goes
Explains pipeline placement, interchangeable implementations, and recall-based evaluation; defers construction of the pool-size measurement.
Examples
Uses a popular-content carousel, co-purchased products, and a four-stage recommender function to illustrate candidate generation.
What you can do after
Build it from scratch as a pipeline stage, swap retrieval methods, and diagnose missed candidates with recall at multiple pool sizes.

Read how AI Recommender Systems explains information retrieval →

MongoDB 8.0 in Action, Third Edition

Information retrieval is the process of finding relevant information from a collection of stored data in response to an information need.

How it is used
Finds and ranks relevant documents for application search and retrieves external information to ground RAG responses.
How deep it goes
Explains search modes, inverted indexing, operational trade-offs, and a worked local query; stops short of implementing a search engine.
Examples
Demonstrates fuzzy inspection-record search with sector and city filters, relevance scores, and a three-result limit.
What you can do after
Apply it via Atlas Search by creating a local index and querying, filtering, and ranking collection results.

Read how MongoDB 8.0 in Action, Third Edition explains information retrieval →

Natural Language Processing in Action, Second Edition

Full-text search retrieves documents by searching their complete textual contents rather than only structured fields.

How it is used
Retrieves relevant text quickly and supplies grounded documents for search, question answering, and language-model workflows.
How deep it goes
Explains forward and inverted indexes, lookup scaling, TF-IDF, trigrams, and integration with semantic search, while deferring full implementation details.
Examples
Uses textbook indexes, cat-versus-cats matching, typo-tolerant trigrams, and large-corpus search as concrete examples.
What you can do after
Build a retrieval component with TF-IDF vectors, inverted indexing, character trigrams, and semantic-search or LLM integration.

Read how Natural Language Processing in Action, Second Edition explains information retrieval →

Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI

The task of finding and retrieving relevant information from a collection in response to an information need.

How it is used
Supports search and recommendation while guiding human annotation of ranked results.
How deep it goes
Explains ranked relevance, DCG weighting, annotator scoring, and model-assisted sampling without detailing retrieval algorithm implementation.
Examples
Uses online product search and basketball queries to illustrate retrieval workflows and sparse relevant results.
What you can do after
Evaluate ranked retrieval results and annotator rankings using relevance and discounted cumulative gain.

Read how Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI explains information retrieval →

Microsoft Azure in Action

Retrieval is the process of selecting relevant stored information to provide as context for a language model.

How it is used
Supplies relevant pieces of oversized books or conversations to a language model with a limited text window.
How deep it goes
Defines retrieval through an analogy but gives no search procedure or configuration steps.
Examples
Uses a fixed-size suitcase and an oversized book or conversation to illustrate selective fetching.
What you can do after
Evaluate/understand it as an option for giving a language model relevant context when the full input will not fit.

Read how Microsoft Azure in Action explains information retrieval →

Elasticsearch in Action, Second Edition

Searching textual content using tokenization, matching, relevance scoring, or related linguistic analysis.

How it is used
Supports application search across varied content providers, websites, mobile applications, and server-side services.
How deep it goes
Explains analysis, tokenization, normalization, and inverted-index matching, while deferring detailed query construction.
Examples
Uses enterprise content providers, application search, French analyzers, root words, synonyms, and spelling mistakes as examples.
What you can do after
Operate retrieval through Kibana or REST clients to execute queries and return matching data from Elasticsearch.

Read how Elasticsearch in Action, Second Edition explains information retrieval →

Data Storytelling with Altair and AI

Information retrieval is the process of searching for and returning relevant information from a collection or external knowledge source.

How it is used
Uses external product information to ground generated commentaries and annotations in data such as websites, reviews, or real-time information.
How deep it goes
Explains retrieval through loading, chunking, embedding, storage, prompting, querying, and returned sources, without developing retrieval algorithms internally.
Examples
Demonstrates retrieval over an HTML smartphone description using LangChain, Chroma, OpenAI embeddings, and a RetrievalQA chain.
What you can do after
Build it from scratch with LangChain and Chroma to answer product questions using retrieved webpage context.

Read how Data Storytelling with Altair and AI explains information retrieval →

Build a Website with ChatGPT

Search is the process of finding records whose data matches a user-supplied word or phrase.

How it is used
Supports interactive web pages by replacing displayed records with data matching user-entered text.
How deep it goes
Explains the text-box workflow and field selection, but defers the generated JavaScript implementation details.
Examples
Uses a course catalog, a Search text box, an article element, and JSON records as concrete artifacts.
What you can do after
Build it from scratch by prompting for HTML and JavaScript that search JSON records while typing.

Read how Build a Website with ChatGPT explains information retrieval →

Ensemble Methods for Machine Learning

Information retrieval is the field concerned with finding and ranking relevant information items in response to queries.

How it is used
Frames search as matching queries to documents and ranking results by relevance.
How deep it goes
Explains retrieval through matching, ranking, and three-class relevance modeling, then stops at model comparisons.
Examples
Uses web search, legal archives, journal databases, related articles, and query-document relevance classes as examples.
What you can do after
Apply information retrieval via LightGBM or XGBoost to classify query-document relevance and compare model performance.

Read how Ensemble Methods for Machine Learning explains information retrieval →

LLM Customization and Fine-Tuning

The process of finding the most relevant text chunks for a query based on embedding similarity.

How it is used
Finds text chunks relevant to a query by comparing their embeddings.
How deep it goes
Explains vector-space search, similarity metrics, top-k sorting, and embedding-size tradeoffs.
Examples
Uses revenue and earnings paragraphs as nearby matches, contrasted with an unrelated lunch paragraph.
What you can do after
Understand retrieval as finding the vector-space neighbors of an embedded query.

Read how LLM Customization and Fine-Tuning explains information retrieval →

Machines that Think

Retrieval is the computational process of selecting relevant information from an external collection in response to a query.

How it is used
Selects relevant passages from external knowledge bases for insertion into a model’s prompt during generation.
How deep it goes
Explains query embeddings, similarity ranking, and prompt insertion, then distinguishes retrieval from guaranteed grounding.
Examples
Uses legal archives, medical databases, corporate repositories, and query-to-document matching as examples.
What you can do after
Evaluate retrieval as a way to expose relevant evidence while recognizing that it cannot guarantee grounded generation.

Read how Machines that Think explains information retrieval →

Building Reliable AI Systems

Retrieval is the process of finding relevant information from an external collection to supply as model context.

How it is used
Supplies relevant external information as context for retrieval-augmented generation.
How deep it goes
Traces document and query embedding through vector storage and similarity search, without detailing search algorithms.
Examples
Uses a health-insurance chatbot that misses policy coverage information despite having access to the documents.
What you can do after
Diagnose how embedding quality can cause a retrieval pipeline to miss relevant context.

Read how Building Reliable AI Systems explains information retrieval →

Also covered in

What to know first

Related concepts

Questions about information retrieval

What is information retrieval?

Information retrieval is the process of finding and selecting information relevant to a query or information need from a collection.

Which Manning books explain information retrieval?

Introduction to Generative AI, Second Edition, Vibe Engineering, Effective Conversational AI, Retrieval Augmented Generation, The Foundational Ideas, Grokking AI Applications, AI Recommender Systems, MongoDB 8.0 in Action, Third Edition, Natural Language Processing in Action, Second Edition and 9 more.

What should I know before learning information retrieval?

Manning books that teach information retrieval usually introduce language model, retrieval-augmented generation, prompt and context window first.

What is the difference between information retrieval and context injection?

Information retrieval is the process of finding and selecting information relevant to a query or information need from a collection. Context injection is the technique of adding externally supplied information to a model’s input so it can inform the model’s generated response.

What is information retrieval part of?

Information retrieval is part of case-based reasoning. Case-based reasoning is an artificial-intelligence technique that solves new problems by retrieving, adapting, and revising solutions from similar past cases.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.