A search for a broad topic can return several useful pages, each requiring a click and a read. Result summarization turns one or more retrieved results into a shorter, readable response. It can combine details across pages and cite the pages they came from, helping a user assess the information without opening every link. section 1.3.5 section 15.2
The technique sits between extractive question answering and more open-ended generation. Extraction returns relevant wording found in documents; summarization rewrites retrieved content. Because the rewritten response may not match any source verbatim, citations matter: users need a way to check what supports it. section 1.3.5 section 15.2.2
A retrieval-and-generation workflow has two main steps:
- Run the user's query and retrieve relevant results. Dense vector search is one option, not a requirement. section 15.2.2
- Put the query and retrieved results into a prompt that asks a foundation model to summarize them. Include source URLs when the response should carry citations. section 15.2.2
Summarizing results for a question
For “What is a large language model?”, a prompt supplies numbered web results and their URLs. The resulting response combines details from the results and links claims back to the articles. Adding “Be concise” produces a shorter cited answer. Instructions about tone or reading level can further shape the response, but they do not replace checking its claims against the retrieved pages. section 15.2.2
Listing 15.3 Prompt to foundation model to summarize search results
Web search results:
[1] "A large language model, or LLM, is a deep learning algorithm that
can recognize, summarize, translate, predict and generate text and other
content based on knowledge gained from massive datasets."
URL: https://blogs.nvidia.com/blog/2023/
01/26/what-are-large-language-models-used-for/
[2] A large language model (LLM) is a language model consisting of a
neural network with many parameters (typically billions of weights or
more), trained on large quantities of unlabeled text using
self-supervised learning. LLMs emerged around 2018 and perform well at a
wide variety of tasks. This has shifted the focus of natural language
processing research away from the previous paradigm of training
specialized supervised models for specific tasks.
URL: https://en.wikipedia.org/wiki/Large_language_model
[3] "In this guide, we'll discuss everything you need to know about
Large Language Models (LLMs), including key terms, algorithms,
fine-tuning, and more. As ChatGPT has taken the internet by storm crossing
1 million users in its first 5 days, you may be wondering what machine
learning algorithm is running under the hood. While ChatGPT uses a
specific type of reinforcement learning called "Reinforcement Learning
from Human Feedback (RLHF)", at a high level it is an example of a
Large Language Model (LLM)."
URL: https://www.mlq.ai/what-is-a-large-language-model-llm/
Instructions: Using the provided web search results, write a comprehensive
reply to the given query. Make sure to cite results using
[[number](URL)] notation after the reference. If the provided search
results refer to multiple subjects with the same name, write separate
answers for each subject.
Query: What is a large language model?
Inspect the prompt's numbered search results and URLs to see what evidence the model receives before writing its summary. section 15.2.2
A fluent summary is not automatically a trustworthy one. Foundation-model responses can reflect unreliable word choices or bias. Keep retrieved sources visible and make the response verifiable rather than asking the model to answer without search context. section 15.2.2
