liveBook · Manning

Prompt

As explained in LLM Evaluation and Alignment, The Foundational Ideas

A system prompt is an instruction supplied to a language model that establishes standing behavior or constraints for an assistant.

A system prompt supplies standing instructions for an assistant. Like other prompts, it guides a model’s response without retraining the model. It differs from a user prompt, which supplies the specific query for an interaction. section 1.1.1 section 6.1.1

In retrieval-augmented generation (RAG), the assistant responds using three inputs: standing instructions, the user’s query, and in-context knowledge retrieved from external sources. Retrieved knowledge supplies information; it is distinct from the instructions and the query. section 6.1.1

Figure 6.4 The generalized RAG framework. The system prompt provides standing instructions. The user prompt provides the query. In-context knowledge, retrieved from external sources, populates the context window. The assistant response is produced from all three inputs combined.

The diagram shows how the system prompt, user prompt, and retrieved knowledge combine to produce a RAG response. section 6.1.1

The name and authority of standing instructions depend on the platform. Under OpenAI’s 2025 Model Spec, system rules are set by the platform and cannot be overridden by developers or users; developers customize behavior through a developer prompt within those limits. OpenAI formerly used the term system prompt for developer-set behavior, while Anthropic calls its API input a system prompt. section 8.4.2

Separating RAG inputs

For a question answered using retrieved documents, identify the standing assistant instructions as the system prompt, the question as the user prompt, and the documents as in-context knowledge. Evaluate whether the documents are relevant to the query and whether the answer is faithful to them as separate checks. section 6.1.1 section 6.1.2

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines the prompt’s role and describes its place in an instruction hierarchy, without detailing model implementation.
Examples
Uses a generalized RAG pipeline to distinguish standing instructions from a user query and retrieved knowledge.
What you can do after
Enables readers to evaluate and understand how standing instructions differ from queries and retrieved evidence in a RAG system.
How it connects
Connects system prompts to retrieval-augmented generation, in-context knowledge, user prompts, and instruction hierarchy.

Also explained in

Compare how 64 books explain prompt →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.