A search result can do more than send a user to a document: it can supply the evidence for an answer. Retrieval-augmented generation (RAG) uses a search engine or vector database to find relevant information, then passes that information to a generative model as context. This helps with a language model’s two weaknesses as a factual source: it may confuse details from its training data, and that data may be out of date. Retrieved context can improve an answer, but it does not make the answer automatically reliable. section 15.2.1 section 15.2.2
A common vector-search pipeline works as follows:
- Split documents into sections, or chunks, and index an embedding for each section in a search engine or vector database. section 15.2.1
- When a user asks a question, create a retrieval query and its embedding. Rank matching sections by vector similarity, commonly cosine similarity or dot product. section 15.2.1
- Pass the ranked sections alongside the user’s prompt to the generative model. Ask it to answer from those results and cite the sources when the user needs to verify its claims. section 15.2.1 section 15.2.2
Answering from search results
For the question “What is a large language model?”, first retrieve relevant pages. A prompt can include their excerpts and URLs, then instruct the model to summarize them. The resulting answer draws details from those results and cites the original pages. Adding “Be concise” to the prompt produces a shorter cited answer; the retrieval step stays the same. section 15.2.2
Listing 15.3 Prompt to foundation model to summarize search results
Web search results:
[1] "A large language model, or LLM, is a deep learning algorithm that
can recognize, summarize, translate, predict and generate text and other
content based on knowledge gained from massive datasets."
URL: https://blogs.nvidia.com/blog/2023/
01/26/what-are-large-language-models-used-for/
[2] A large language model (LLM) is a language model consisting of a
neural network with many parameters (typically billions of weights or
more), trained on large quantities of unlabeled text using
self-supervised learning. LLMs emerged around 2018 and perform well at a
wide variety of tasks. This has shifted the focus of natural language
processing research away from the previous paradigm of training
specialized supervised models for specific tasks.
URL: https://en.wikipedia.org/wiki/Large_language_model
[3] "In this guide, we'll discuss everything you need to know about
Large Language Models (LLMs), including key terms, algorithms,
fine-tuning, and more. As ChatGPT has taken the internet by storm crossing
1 million users in its first 5 days, you may be wondering what machine
learning algorithm is running under the hood. While ChatGPT uses a
specific type of reinforcement learning called "Reinforcement Learning
from Human Feedback (RLHF)", at a high level it is an example of a
Large Language Model (LLM)."
URL: https://www.mlq.ai/what-is-a-large-language-model-llm/
Instructions: Using the provided web search results, write a comprehensive
reply to the given query. Make sure to cite results using
[[number](URL)] notation after the reference. If the provided search
results refer to multiple subjects with the same name, write separate
answers for each subject.
Query: What is a large language model?
The example prompt shows how numbered web results, excerpts, and URLs become context for a model asked to summarize search results. section 15.2.2
Retrieval quality matters because the model can only usefully answer from context that was found and supplied. Chunking is necessary in a common embedding-based pipeline but introduces trade-offs when dividing documents. Search results also require scrutiny: a model may misrepresent facts, reflect bias in its prompt, or be influenced by misleading pages. Keep source citations available for checking rather than treating a fluent response as proof. section 15.2.1 section 15.2 section 15.2.2
