liveBook · Manning

Context

As explained in LLMs in Production

Information supplied with a prompt that enables a language model to interpret a request and produce an appropriate response.

Context is information added to a prompt so a language model can interpret a request and respond appropriately. It supplies knowledge the model may not have, such as current company policies, rather than relying only on the user’s wording or training data. section 7.2.1

A useful prompt separates four responsibilities:

  • Input captures what the user writes. section 7.5
  • Instruction encodes task-specific directions in a template. section 7.5
  • Context adds retrieved information through retrieval-augmented generation (RAG) or database lookups. section 7.5
  • System instructions establish hidden, task-wide constraints. section 7.5

A policy question

For “How much does the company match for our 401k?”, a vector database can retrieve the closest policy documents and insert them as context. Combined with instructions to answer only from company policies and admit uncertainty, the model can respond with the relevant match rate and documentation link. section 7.2.1

How LLMs in Production covers it

How deep it goes
Explains context as one of four prompt parts and shows vector database retrieval, but does not detail retrieval implementation.
Examples
Uses a company-policy 401k question, retrieved vector database documents, and structured instructions as concrete examples.
What you can do after
Apply it through RAG or database lookups to add relevant information to prompts and improve response appropriateness.
How it connects
Connects context with prompts, instructions, system messages, RAG, database lookups, vector databases, and few-shot examples.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

Compare how 18 books explain context →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.