Information retrieval
Information retrieval is the process of locating and selecting relevant information from an available collection in response to a query or information need.
Introduction to Generative AI, Second Edition
Retrieval is the process of finding relevant information from an external collection in response to a query or representation.
- How it is used
- Supplies language models with external information before they answer questions, especially when current facts or supporting sources matter.
- How deep it goes
- Explains the query-search-summary sequence and semantic matching in vector databases at a high level, without implementation steps.
- Examples
- Uses a vegan-restaurant recommendation involving past preferences and Yelp reviews, alongside everyday searches for purchases and local activities.
- What you can do after
- Evaluate/understand retrieval-enabled answers by checking whether retrieved sources support the summary and whether those sources are credible.
Read how Introduction to Generative AI, Second Edition explains information retrieval →
Effective Conversational AI
The retrieval phase is the stage of a retrieval-augmented generation process that finds and obtains information relevant to an input.
- How it is used
- Finds document passages for nuanced, long-tail chatbot questions that predefined intents may not answer well.
- How deep it goes
- Explains retrieval through document preparation, chunking, embeddings, and vector storage, while distinguishing retrieved passages from generated answers.
- Examples
- Contrasts a generic side-effects answer with passages addressing a vaccine, ibuprofen, pain, and blood-pressure medication.
- What you can do after
- Evaluate/understand when retrieved passages suffice and when RAG better reduces the work of assembling an answer.
Read how Effective Conversational AI explains information retrieval →
Grokking AI Applications
The process of finding and obtaining relevant information from a data source for use in an AI application.
- How it is used
- Locates the few best knowledge chunks at runtime so an LLM can use them to answer a user question.
- How deep it goes
- Defines retrieval within RAG and vector-database search, stopping short of mathematical proofs and model-architecture internals.
- Examples
- Uses a user question, a vector database, and selected document chunks as the concrete retrieval scenario.
- What you can do after
- Apply it via a RAG flow by separating setup-time ingestion from per-question retrieval and generation.
Read how Grokking AI Applications explains information retrieval →
MongoDB 8.0 in Action, Third Edition
Information retrieval is the process of finding relevant information from a collection of stored data in response to an information need.
- How it is used
- Finds and ranks relevant documents for application search and retrieves external information to ground RAG responses.
- How deep it goes
- Explains search modes, inverted indexing, operational trade-offs, and a worked local query; stops short of implementing a search engine.
- Examples
- Demonstrates fuzzy inspection-record search with sector and city filters, relevance scores, and a three-result limit.
- What you can do after
- Apply it via Atlas Search by creating a local index and querying, filtering, and ranking collection results.
Read how MongoDB 8.0 in Action, Third Edition explains information retrieval →
Natural Language Processing in Action, Second Edition
Full-text search retrieves documents by searching their complete textual contents rather than only structured fields.
- How it is used
- Retrieves relevant text quickly and supplies grounded documents for search, question answering, and language-model workflows.
- How deep it goes
- Explains forward and inverted indexes, lookup scaling, TF-IDF, trigrams, and integration with semantic search, while deferring full implementation details.
- Examples
- Uses textbook indexes, cat-versus-cats matching, typo-tolerant trigrams, and large-corpus search as concrete examples.
- What you can do after
- Build a retrieval component with TF-IDF vectors, inverted indexing, character trigrams, and semantic-search or LLM integration.
Read how Natural Language Processing in Action, Second Edition explains information retrieval →
Microsoft Azure in Action
Retrieval is the process of selecting relevant stored information to provide as context for a language model.
- How it is used
- Supplies relevant pieces of oversized books or conversations to a language model with a limited text window.
- How deep it goes
- Defines retrieval through an analogy but gives no search procedure or configuration steps.
- Examples
- Uses a fixed-size suitcase and an oversized book or conversation to illustrate selective fetching.
- What you can do after
- Evaluate/understand it as an option for giving a language model relevant context when the full input will not fit.
Read how Microsoft Azure in Action explains information retrieval →
Elasticsearch in Action, Second Edition
Searching textual content using tokenization, matching, relevance scoring, or related linguistic analysis.
- How it is used
- Supports application search across varied content providers, websites, mobile applications, and server-side services.
- How deep it goes
- Explains analysis, tokenization, normalization, and inverted-index matching, while deferring detailed query construction.
- Examples
- Uses enterprise content providers, application search, French analyzers, root words, synonyms, and spelling mistakes as examples.
- What you can do after
- Operate retrieval through Kibana or REST clients to execute queries and return matching data from Elasticsearch.
Read how Elasticsearch in Action, Second Edition explains information retrieval →
Data Storytelling with Altair and AI
Information retrieval is the process of searching for and returning relevant information from a collection or external knowledge source.
- How it is used
- Uses external product information to ground generated commentaries and annotations in data such as websites, reviews, or real-time information.
- How deep it goes
- Explains retrieval through loading, chunking, embedding, storage, prompting, querying, and returned sources, without developing retrieval algorithms internally.
- Examples
- Demonstrates retrieval over an HTML smartphone description using LangChain, Chroma, OpenAI embeddings, and a RetrievalQA chain.
- What you can do after
- Build it from scratch with LangChain and Chroma to answer product questions using retrieved webpage context.
Read how Data Storytelling with Altair and AI explains information retrieval →
Build a Website with ChatGPT
Search is the process of finding records whose data matches a user-supplied word or phrase.
- How it is used
- Supports interactive web pages by replacing displayed records with data matching user-entered text.
- How deep it goes
- Explains the text-box workflow and field selection, but defers the generated JavaScript implementation details.
- Examples
- Uses a course catalog, a Search text box, an article element, and JSON records as concrete artifacts.
- What you can do after
- Build it from scratch by prompting for HTML and JavaScript that search JSON records while typing.
Read how Build a Website with ChatGPT explains information retrieval →
Ensemble Methods for Machine Learning
Information retrieval is the field concerned with finding and ranking relevant information items in response to queries.
- How it is used
- Frames search as matching queries to documents and ranking results by relevance.
- How deep it goes
- Explains retrieval through matching, ranking, and three-class relevance modeling, then stops at model comparisons.
- Examples
- Uses web search, legal archives, journal databases, related articles, and query-document relevance classes as examples.
- What you can do after
- Apply information retrieval via LightGBM or XGBoost to classify query-document relevance and compare model performance.
Read how Ensemble Methods for Machine Learning explains information retrieval →
Building Reliable AI Systems
Retrieval is the process of finding relevant information from an external collection to supply as model context.
- How it is used
- Supplies relevant external information as context for retrieval-augmented generation.
- How deep it goes
- Traces document and query embedding through vector storage and similarity search, without detailing search algorithms.
- Examples
- Uses a health-insurance chatbot that misses policy coverage information despite having access to the documents.
- What you can do after
- Diagnose how embedding quality can cause a retrieval pipeline to miss relevant context.
Read how Building Reliable AI Systems explains information retrieval →
Related concepts
- retrieval-augmented generationused by · Retrieval-augmented generation is a technique that retrieves relevant information from external sources and supplies it as context to a generative model when producing an output.
- hybrid searchused by · Hybrid search is an information-retrieval technique that combines multiple retrieval methods or relevance signals to find or rank relevant results.
- similarity searchincludes · Similarity search is a retrieval technique that finds or ranks items by comparing their representations with a query according to a similarity or distance measure.
- chunkingused by · Chunking is the process of dividing content or data into smaller units for processing, retrieval, or organization.
- agentic RAGused by · Agentic RAG is a retrieval-augmented generation approach in which an agent dynamically decides how and when to retrieve information, often iterating or using tools before generating a response.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.