Chapter 1: Introduction to context engineering
The chapter introduction defines context engineering as supplying an LLM with the information needed for a task while respecting its context-window limit.
After this chapter you will be able to
- Context engineering purpose. Explain what context means for an LLM-based system and how context engineering aims to make each model call effective.
- Prompt and context engineering distinction. Distinguish refining the wording of a prompt from deciding what information the model sees when that prompt arrives, and explain why the broader view matters.
- Runtime context and storage analogy. Describe the CPU-and-RAM analogy and what it says about the information an LLM can use at runtime.
- Context limits and information tradeoffs. Distinguish the context window, token budget, and attention budget, and explain why effective context is the smallest relevant set that supports the task.
- Two dimensions of the stack. Explain how the stack separates horizontal functional layers from vertical cross-cutting concerns, and why it is a conceptual guide rather than a fixed architecture.
- Context management operations. Explain how context management shapes raw sources into an active context window before a model consumes them, using the four operations named here.
- Context orchestration role. Distinguish context orchestration from management by explaining that orchestration routes context through execution, including multi-step work and coordination.
- Production-wide concerns. Describe why evaluation, observability, governance, and operations cut across the functional layers, and state the distinct role of each.
- Next-token language generation. Explain how an LLM learns from text and produces a response to a prompt. Describe what tokens are and why fluent output does not imply human-like understanding.
- Inference-time generation controls. Describe how temperature, maximum output tokens, top-k, and top-p affect generation.
- Reasoning model tradeoffs. Explain how reasoning models handle multi-step problems and what costs accompany their intermediate reasoning tokens.
- Predefined workflow execution path. Describe how an AI workflow breaks a complex task into steps and who determines the order in which those steps run.
- Agent Autonomy and Execution. Distinguish an AI agent from a plain LLM and a predefined AI workflow. Describe the agent as an execution unit that pursues goals with some autonomy.
- Model and Harness Roles. Explain what the model contributes to an agent and what the surrounding harness does to let it operate beyond the model alone.
- Capabilities and Agent Loop. Trace how an agent’s actions and observations feed its next reasoning step, and describe how this supports adjustment during a multi-step task.
- Harness and Runtime Boundary. Distinguish the harness’s coordination of agent behavior from the runtime’s hosting and enforcement of execution constraints.
- Context and Harness Engineering. Distinguish the information-layer focus of context engineering from the software-infrastructure focus of harness engineering, while explaining why working agents need both.
- Agentic architecture and agents. Distinguish an AI agent from an agentic system, and explain why a system can be agentic with only one agent.
- Specialization benefits and costs. Explain why a system might use specialized agents and what costs their coordination introduces.
- Downstream impact of LLM limitations. Explain why limitations of LLMs matter for applications and systems built on them.
- Hallucinations and layered defenses. Explain why an LLM can present fabricated information convincingly and why systems use multiple defenses rather than relying on a single fix.
- Stale knowledge and context. Explain how a training cutoff and aging stored material cause stale answers, and distinguish the updates that retrieval and live tools can supply.
- Drift across system handoffs. Trace how a small distortion can grow as information passes between steps or agents, and describe the safeguards identified here.
- Sources of output variation. Explain why the same prompt can produce different outputs and why an LLM-based system should aim for acceptable behavior rather than exact repeatability.
- Token processing and latency. Explain why LLM responses can be slow and how the amount of context affects response time.
- Finite context window boundary. Describe the hard context-window limit and its consequence for choosing what material to supply.
- Lost in the middle effect. Explain how position within a long prompt can affect an included fact's influence on the output.
- Training Data Bias. Explain how LLMs can reproduce biases in their training data, and why mitigation does not guarantee neutral behavior.
- Jailbreaking and Prompt Injection. Distinguish deliberate jailbreaking from prompt injection, and explain why bringing outside material into context increases the safety risk.
- Privacy of Supplied Context. Explain why sensitive information must be screened before an LLM call, including information supplied to a hosted model.
- Task Relevant Evidence. Explain how supplying current, task-relevant information improves an agent’s accuracy compared with relying only on training data and the immediate prompt.
- Consistent Predictable Decisions. Distinguish consistency from predictability and explain how structured preservation of rules, constraints, preferences, and decisions supports both.
- Continuity Across Interactions. Describe how retaining relevant user information and previous decisions lets an agent continue a task across turns, sessions, and workflow steps.
- Visible Reasons And Auditability. Explain how explicitly managed facts, tool outputs, instructions, and decisions make an agent’s recommendations easier to explain and inspect.
- Selective Context Efficiency. Justify selecting and structuring information relevant to the current step instead of filling the context window with everything available.
- Reusable Context Pipelines. Explain why a reusable context pipeline allows an organization to adapt agents and iterate across tasks without changing the underlying model.
- System-owned context sources. Explain that the six context sources are categories managed by the surrounding system, not distinctions the LLM perceives. Describe why different systems use different subsets of those sources.
- Per-call context assembly. Trace how the context manager selects material for a call and how a turn's results feed the next turn.
- Orchestrator and shared state. Distinguish the orchestrator's choice of the next step from the context manager's preparation of its window, and explain how the state object keeps them aligned.
- Role of instructions in context. Explain that instructions are the part of context that sets high-level directives, constraints, and expectations for an LLM’s behavior, and name the three mechanisms described here.
- Skills and durable instruction artifacts. Distinguish agent skills as reusable, task-specific modules loaded when needed from instruction artifacts as agent-readable files containing stable project or environment guidance, usually sent with every LLM call.
- External knowledge as grounding. Explain why external knowledge is a context source: it gives an LLM dynamic access to information beyond its static training data and helps ground answers in accurate, domain-specific, up-to-date material.
- Retrieved context selection. Describe how only selected passages and useful citation or date metadata enter the context window, and why retrieval quality limits the answer: a missed passage cannot influence the model.
- Tool content in context. Explain how tools let an LLM interact with information and systems beyond its immediate context, and distinguish tool definitions, tool calls, and tool outputs. Describe what normally stays in working context versus what may be persisted as memory.
- Three tool use paradigms. Distinguish function calling, CLI tools, and MCP at the level described here, and state what they have in common.
- Memory and stateless LLM calls. Explain why an application needs memory to provide continuity across LLM calls.
- Current-session short-term memory. Describe what short-term memory retains, how long it lasts, and how it reaches a stateless model.
- Cross-session long-term memory. Describe what long-term memory preserves across sessions and when stored information enters the context window.
- Session State and Short-Term Memory. Distinguish state from short-term memory: state describes the current task situation, while short-term memory records what was said and done.
- Mutable State and Context Assembly. Explain how a transient state object changes as a task advances and how the context manager uses it when assembling the next context window.
- User prompt and context sources. Explain what a user prompt is and why other context sources are chosen in response to it.
- Programmatic context engineering. Explain why many examples involve developing software components: context engineering is mostly implemented through systems or applications that manage context sources or provide orchestration.
- Context visibility through assistant interfaces. Describe how to use prompts in an AI assistant interface to probe what context users can see, while recognizing that the answers may be approximate or sanitized.
- Application-layer LLM access. Explain why context engineering commonly requires programmatic access to LLMs and describe what the example function can configure.
- Limits of example outputs. Distinguish what the two model runs illustrate from what their responses and output values can establish.
- Ollama interaction options. Describe how the section has you install and interact with a local model through Ollama, including the programmatic example.
- Context visibility across interfaces. Distinguish what the assistant interface, cloud API, and local model reveal or control about context, and state what they have in common.
Section by section
Introduction
The chapter introduction defines context engineering as supplying an LLM with the information needed for a task while respecting its context-window limit.
- Context engineering purpose: Explain what context means for an LLM-based system and how context engineering aims to make each model call effective.
Why it matters: It frames context as something builders deliberately supply and constrain, rather than simply adding more text. - Generative AI and LLM roles: Distinguish generative AI from the LLMs at the core of text-based generative AI.
Why it matters: The distinction identifies the language model whose context the rest of the book addresses.
Key terms: context, inference time, context engineering, context window, tokens, generative AI, GenAI, large language models, LLMs
1.1 Defining context engineering
This section distinguishes prompt engineering from context engineering and introduces the limited context window that constrains what an LLM can see.
- Prompt and context engineering distinction: Distinguish refining the wording of a prompt from deciding what information the model sees when that prompt arrives, and explain why the broader view matters.
Why it matters: The distinction shifts attention from polishing one prompt to considering the full input the model receives. - Limited context window: Describe the context window as the limited amount of text an LLM can process at once, usually measured in tokens, and relate it to what context engineering places before the model.
Why it matters: The window's limit explains why deciding what the model sees is a meaningful task.
Key terms: prompt engineering, context engineering, prompt, context window, tokens
1.1.1 The context window
This section explains why LLM performance depends on the information placed in a limited context window and why that information must be chosen carefully.
- Context beyond prompt wording: Explain that context engineering concerns the information assembled for an LLM’s task, not just the wording of its prompt.
Why it matters: This shifts attention from polishing a prompt to choosing what information the model receives. - Runtime context and storage analogy: Describe the CPU-and-RAM analogy and what it says about the information an LLM can use at runtime.
Why it matters: The analogy clarifies why having data somewhere in a system does not mean the model can use it in a given call. - Context limits and information tradeoffs: Distinguish the context window, token budget, and attention budget, and explain why effective context is the smallest relevant set that supports the task.
Why it matters: Choosing context means balancing answer quality against distraction, response time, and token cost.
Key terms: context engineering, context window, tokens, persistent storage, tool calls, token budget, attention budget, reasoning quality per token
1.1.2 The context engineering stack
The section presents the context engineering stack as a conceptual map of how context is sourced, shaped, and routed, with production concerns applying across that work.
- Two dimensions of the stack: Explain how the stack separates horizontal functional layers from vertical cross-cutting concerns, and why it is a conceptual guide rather than a fixed architecture.
Why it matters: This map helps you place techniques in the stack without mistaking the diagram for a required system design. - Six foundational context sources: Describe the context sources layer as the raw material supplied to a model on each call, and name the six source categories used in the book.
Why it matters: The six categories identify what can contribute material before that material is shaped into active context. - Context management operations: Explain how context management shapes raw sources into an active context window before a model consumes them, using the four operations named here.
Why it matters: This is the layer that turns available material into the context a model actually receives. - Context orchestration role: Distinguish context orchestration from management by explaining that orchestration routes context through execution, including multi-step work and coordination.
Why it matters: The distinction separates preparing one call's context from coordinating context across a larger execution. - Production-wide concerns: Describe why evaluation, observability, governance, and operations cut across the functional layers, and state the distinct role of each.
Why it matters: A production context pipeline needs more than a path for assembling and routing information.
Key terms: context engineering stack, functional layers, cross-cutting concerns, context sources, instructions, external knowledge, tools, memory, state, context management
1.2 Evolution of LLM-based systems
1.2.1 LLMs, the foundational models
This section explains how LLMs generate text, distinguishes learned parameters from practitioner-chosen settings, and describes how reasoning models extend generation for multi-step tasks.
- Next-token language generation: Explain how an LLM learns from text and produces a response to a prompt. Describe what tokens are and why fluent output does not imply human-like understanding.
Why it matters: Next-token prediction is the foundation for the other model behaviors described in this section. - Learned parameters and model size: Distinguish learned model parameters from practitioner-chosen hyperparameters, and explain why parameter count alone does not establish model quality or computational cost.
Why it matters: It prevents model-size labels from being mistaken for a complete measure of capability or cost. - Inference-time generation controls: Describe how temperature, maximum output tokens, top-k, and top-p affect generation.
Why it matters: These settings affect the character and length of generated responses without changing learned model parameters. - Reasoning model tradeoffs: Explain how reasoning models handle multi-step problems and what costs accompany their intermediate reasoning tokens.
Why it matters: It explains both the potential benefit of reasoning models and the token and latency costs of using them.
Key terms: LLM, transformer architectures, token, prompt, next-token prediction, parameters, weights and biases, hyperparameters, parameter count, mixture of experts (MoE)
1.2.2 AI workflows
This section explains how a predefined AI workflow combines LLM calls with code and external data to handle tasks beyond a single LLM call.
- Standalone LLM call limits: Explain why a single LLM call cannot by itself handle tasks that require external information or several coordinated steps.
Why it matters: These limits explain why an application might put a workflow around an LLM. - Predefined workflow execution path: Describe how an AI workflow breaks a complex task into steps and who determines the order in which those steps run.
Why it matters: The fixed execution path is what distinguishes a workflow from letting the model decide what to do next. - Retrieval augmented workflow example: Trace the retrieval-augmented generation example from an external data source to the model prompt.
Why it matters: RAG gives a concrete example of how a workflow supplies external information to an LLM.
Key terms: standalone LLM call, single inference pass, AI workflow, predefined pipeline, control and data flow, retrieval-augmented generation, RAG, external data sources
1.2.3 AI agents
This section explains what makes an LLM-based system an AI agent, how its model, harness, and runtime differ, and how context supports iterative action.
- Agent Autonomy and Execution: Distinguish an AI agent from a plain LLM and a predefined AI workflow. Describe the agent as an execution unit that pursues goals with some autonomy.
Why it matters: This distinction clarifies what the book means when it calls a system an agent. - Model and Harness Roles: Explain what the model contributes to an agent and what the surrounding harness does to let it operate beyond the model alone.
Why it matters: The model–harness split shows why an LLM alone cannot provide the agent architecture described here. - Capabilities and Agent Loop: Trace how an agent’s actions and observations feed its next reasoning step, and describe how this supports adjustment during a multi-step task.
Why it matters: The feedback loop explains how an agent can respond to results and mistakes rather than merely run a fixed sequence. - Harness and Runtime Boundary: Distinguish the harness’s coordination of agent behavior from the runtime’s hosting and enforcement of execution constraints.
Why it matters: Separating coordination from execution constraints makes the agent’s operational and safety boundaries easier to explain. - Context and Harness Engineering: Distinguish the information-layer focus of context engineering from the software-infrastructure focus of harness engineering, while explaining why working agents need both.
Why it matters: The distinction keeps attention on both the information a model receives and the machinery that makes it usable in an agent.
Key terms: AI agents, autonomy, AI workflows, LLM, model, harness, context, execution state, agent loop, reasoning
1.2.4 Agentic systems
Defines agentic systems as goal-directed architectures that may contain one or more agents, then explains multi-agent coordination, its tradeoffs, and its context demands.
- Agentic architecture and agents: Distinguish an AI agent from an agentic system, and explain why a system can be agentic with only one agent.
Why it matters: This distinction keeps “agentic” from being mistaken for “multi-agent.” - Single and multi-agent coordination: Describe how work can be carried out within a single-agent system or distributed and coordinated among agents in a multi-agent system.
Why it matters: It clarifies what changes when work is distributed across agents without treating multi-agent design as a requirement. - Specialization benefits and costs: Explain why a system might use specialized agents and what costs their coordination introduces.
Why it matters: Multi-agent specialization is a design tradeoff, not an automatic improvement. - Context across collaborating agents: Describe the context decisions involved in coordinating an agentic system.
Why it matters: Coordinating agents also means managing the information on which their next actions depend.
Key terms: agentic system, AI agent, agentic AI, single-agent system, multi-agent system, orchestrator, system boundary, specialization, focused context, coordination
1.3 Limitations of LLMs
This introduction establishes that LLM limitations can become weaknesses in applications built on them.
- Downstream impact of LLM limitations: Explain why limitations of LLMs matter for applications and systems built on them.
Why it matters: An application can inherit weaknesses from the model it relies on.
Key terms: LLMs, limitations, technical barriers
1.3.1 Information integrity
This section explains four ways LLM systems can compromise information integrity and the kinds of safeguards that address each risk.
- Hallucinations and layered defenses: Explain why an LLM can present fabricated information convincingly and why systems use multiple defenses rather than relying on a single fix.
Why it matters: A convincing answer still needs evidence and checks before it can be trusted. - Stale knowledge and context: Explain how a training cutoff and aging stored material cause stale answers, and distinguish the updates that retrieval and live tools can supply.
Why it matters: Supplying context only helps with freshness when that context is itself current. - Domain expertise and customization: Distinguish context-based ways to improve domain-specific answers from fine-tuning, including what fine-tuning does not guarantee.
Why it matters: Domain fluency is not the same as having current, verifiable domain information. - Drift across system handoffs: Trace how a small distortion can grow as information passes between steps or agents, and describe the safeguards identified here.
Why it matters: A chain of individually plausible steps can produce a final claim that the original source does not support.
Key terms: hallucination, instructions, grounding, tools, memory, faithfulness metrics, stale data, training cutoff, retrieval, live data
1.3.2 Technical and architectural constraints
This section explains four technical constraints on LLM-based systems—variable outputs, latency, finite context windows, and uneven attention within long prompts—and why each matters for context engineering.
- Sources of output variation: Explain why the same prompt can produce different outputs and why an LLM-based system should aim for acceptable behavior rather than exact repeatability.
Why it matters: You cannot design evaluation or runtime checks around an assumption that the model will always give the exact same answer. - Token processing and latency: Explain why LLM responses can be slow and how the amount of context affects response time.
Why it matters: Context choices affect not only answer quality but also how long users wait. - Finite context window boundary: Describe the hard context-window limit and its consequence for choosing what material to supply.
Why it matters: Information cannot help a response if it never fits into the context the model receives. - Lost in the middle effect: Explain how position within a long prompt can affect an included fact's influence on the output.
Why it matters: Even information that fits in the window may have little effect if its position makes it easy for the model to neglect.
Key terms: nondeterminism, next token, temperature, exact repeatability, latency, token generation, context window, cache-augmented generation, retrieval, memory
1.3.3 Bias, safety, and privacy
This section explains how training-data bias, unsafe responses, prompt injection, and exposure of sensitive context limit LLM-based systems.
- Training Data Bias: Explain how LLMs can reproduce biases in their training data, and why mitigation does not guarantee neutral behavior.
Why it matters: Reducing bias is not the same as ensuring fair responses in every situation. - Jailbreaking and Prompt Injection: Distinguish deliberate jailbreaking from prompt injection, and explain why bringing outside material into context increases the safety risk.
Why it matters: Material supplied to help the model can also carry instructions it should not follow. - Privacy of Supplied Context: Explain why sensitive information must be screened before an LLM call, including information supplied to a hosted model.
Why it matters: Once sensitive data enters context, the model itself cannot keep that data from being used or disclosed.
Key terms: biases, training data, mitigation, jailbreaking, prompt injection, context window, guardrails, access control, context, hosted model
1.4 Benefits of context engineering
The section explains how explicitly managing an AI system’s context improves accuracy, consistency, continuity, explainability, efficiency, and scalability without eliminating every LLM limitation.
- Task Relevant Evidence: Explain how supplying current, task-relevant information improves an agent’s accuracy compared with relying only on training data and the immediate prompt.
Why it matters: Relevant evidence makes recommendations more dependable when details such as prices and availability can change. - Consistent Predictable Decisions: Distinguish consistency from predictability and explain how structured preservation of rules, constraints, preferences, and decisions supports both.
Why it matters: Stable criteria help prevent a later recommendation from contradicting an earlier choice. - Continuity Across Interactions: Describe how retaining relevant user information and previous decisions lets an agent continue a task across turns, sessions, and workflow steps.
Why it matters: A continuing task is more useful when the agent remembers what the user has already supplied. - Visible Reasons And Auditability: Explain how explicitly managed facts, tool outputs, instructions, and decisions make an agent’s recommendations easier to explain and inspect.
Why it matters: Visible context lets users understand a recommendation and developers examine how it was produced. - Selective Context Efficiency: Justify selecting and structuring information relevant to the current step instead of filling the context window with everything available.
Why it matters: Relevant context can keep an agent useful without making every step slower and more expensive. - Reusable Context Pipelines: Explain why a reusable context pipeline allows an organization to adapt agents and iterate across tasks without changing the underlying model.
Why it matters: Reusing the surrounding pipeline makes it easier to expand or revise an AI application.
Key terms: accuracy, parametric knowledge, model context, consistency, predictability, constraints, intermediate decisions, continuity, user preferences, task constraints
1.5 Taxonomy for context sources
This section explains how an LLM-based system selects and assembles six possible context sources for each call, coordinates that work across turns, and lets information move between source roles.
- System-owned context sources: Explain that the six context sources are categories managed by the surrounding system, not distinctions the LLM perceives. Describe why different systems use different subsets of those sources.
Why it matters: It keeps the source taxonomy from being mistaken for something built into the model. - Per-call context assembly: Trace how the context manager selects material for a call and how a turn's results feed the next turn.
Why it matters: It identifies where decisions about what the model sees on each call are made. - Orchestrator and shared state: Distinguish the orchestrator's choice of the next step from the context manager's preparation of its window, and explain how the state object keeps them aligned.
Why it matters: It separates deciding what happens next from deciding what information that step receives. - Changing roles of information: Explain how information can move between context-source roles over time, using an example from the section.
Why it matters: It shows why source categories are useful roles rather than permanent labels on information.
Key terms: context sources, LLM-based system, context window, context selection, context assembly, context curation, context manager, agent harness, context orchestrator, state object
1.5.1 Instructions
Defines instructions as behavior-governing context and distinguishes the system prompt, agent skills, and instruction artifacts as ways to deliver them.
- Role of instructions in context: Explain that instructions are the part of context that sets high-level directives, constraints, and expectations for an LLM’s behavior, and name the three mechanisms described here.
Why it matters: This distinguishes behavioral guidance from other kinds of information supplied to an LLM. - System prompt behavior settings: Describe the system prompt as a typically hidden message placed before user input that establishes a model’s role, tone, output conventions, and safety boundaries.
Why it matters: It explains where an application can establish baseline behavior for the model. - Skills and durable instruction artifacts: Distinguish agent skills as reusable, task-specific modules loaded when needed from instruction artifacts as agent-readable files containing stable project or environment guidance, usually sent with every LLM call.
Why it matters: The distinction helps explain why specialized task guidance and stable project rules enter an agent’s context differently.
Key terms: instructions, system prompt, agent skills, instruction artifacts, role, tone, output conventions, safety boundaries, AGENTS.md, CLAUDE.md
1.5.2 External knowledge
External knowledge supplies current, domain-specific context to an LLM, but the quality of its answers depends on which source passages retrieval selects.
- External knowledge as grounding: Explain why external knowledge is a context source: it gives an LLM dynamic access to information beyond its static training data and helps ground answers in accurate, domain-specific, up-to-date material.
Why it matters: It explains why a model may need information from outside its training data to answer a question well. - Retrieved context selection: Describe how only selected passages and useful citation or date metadata enter the context window, and why retrieval quality limits the answer: a missed passage cannot influence the model.
Why it matters: Even a capable model cannot use relevant evidence if retrieval leaves it out of the context.
Key terms: external knowledge, static data, RAG, ground, context window, passages, metadata, retrieval, rank, tokens
1.5.3 Tools
This section explains how tools extend an LLM’s capabilities, what tool-related content enters its context, and three common ways tools are exposed.
- Tool content in context: Explain how tools let an LLM interact with information and systems beyond its immediate context, and distinguish tool definitions, tool calls, and tool outputs. Describe what normally stays in working context versus what may be persisted as memory.
Why it matters: It helps you distinguish the guidance a model receives from the requests it makes and the results it gets back. - Three tool use paradigms: Distinguish function calling, CLI tools, and MCP at the level described here, and state what they have in common.
Why it matters: It gives you a basic way to recognize how an agent’s available tools are presented and invoked.
Key terms: tool definitions, tool call, tool outputs, current working context, memory, function calling, command-line interface (CLI), model context protocol (MCP), structured invocation, agent harness
1.5.4 Memory
This section explains how memory gives stateless LLM calls continuity, distinguishing model knowledge from the short- and long-term memory managed outside the model.
- Memory and stateless LLM calls: Explain why an application needs memory to provide continuity across LLM calls.
Why it matters: It explains why conversational continuity must be supplied by the application rather than assumed of the model. - Parametric and non-parametric memory: Distinguish knowledge embedded in model weights from runtime-updatable memory outside the model, and identify which form context engineering works with.
Why it matters: This distinction prevents confusing an application's stored information with what the model learned during training. - Current-session short-term memory: Describe what short-term memory retains, how long it lasts, and how it reaches a stateless model.
Why it matters: It shows how an agent can use something said earlier in the same session without the model retaining it internally. - Cross-session long-term memory: Describe what long-term memory preserves across sessions and when stored information enters the context window.
Why it matters: It explains how an agent can use a saved preference weeks later without loading every stored fact into every call.
Key terms: stateless, memory, context source, parametric, non-parametric, model’s weights, short-term memory, current session, curated view, session object
1.5.5 State
This section defines session state, distinguishes it from short-term memory, and explains how a changing state object informs the model’s next context window.
- Session State and Short-Term Memory: Distinguish state from short-term memory: state describes the current task situation, while short-term memory records what was said and done.
Why it matters: The distinction helps clarify whether information describes the current task or records earlier interaction. - Mutable State and Context Assembly: Explain how a transient state object changes as a task advances and how the context manager uses it when assembling the next context window.
Why it matters: It explains how an application can keep track of a changing task without putting its entire state into every model turn.
Key terms: state, short-term memory, session, state object, scratchpad, context orchestrator, context manager, context window
1.5.6 User prompt
Defines the user prompt as the end user's immediate task and explains how it sets the goal for other context sources.
- User prompt and context sources: Explain what a user prompt is and why other context sources are chosen in response to it.
Why it matters: The user's task gives the other context sources a purpose.
Key terms: user prompt, context source
1.6 Hands-on
This hands-on introduction explains why the book uses software examples and where readers can find their complete, runnable code.
- Programmatic context engineering: Explain why many examples involve developing software components: context engineering is mostly implemented through systems or applications that manage context sources or provide orchestration.
Why it matters: It explains why hands-on work in the book often involves code rather than just writing prompts. - Complete example code location: Describe why readers should use the companion GitHub repository for hands-on examples: it contains the complete, runnable implementations, while book listings may be trimmed.
Why it matters: Readers should use the repository when they want to reproduce an example rather than assume a shortened listing is complete.
Key terms: context engineering, context sources, orchestration capabilities, context engineering stack, companion GitHub repository, code listings, full, runnable implementation
1.6.1 Exploring context through an AI assistant interface
This exercise uses an AI assistant interface to explore which aspects of an LLM’s context are visible to users and which remain hidden.
- Context visibility through assistant interfaces: Describe how to use prompts in an AI assistant interface to probe what context users can see, while recognizing that the answers may be approximate or sanitized.
Why it matters: It helps you interpret what an assistant says about its context without treating that answer as a complete view.
Key terms: AI assistant, context, opaque, sanitized responses
1.6.2 Programmatic access to LLMs
This section shows why context engineering uses programmatic LLM access and how to interpret a simple script’s responses and output signals.
- Application-layer LLM access: Explain why context engineering commonly requires programmatic access to LLMs and describe what the example function can configure.
Why it matters: Application code gives you a way to configure and query models as part of a larger system. - Limits of example outputs: Distinguish what the two model runs illustrate from what their responses and output values can establish.
Why it matters: It prevents a model’s answer or a sample run from being mistaken for verified information about the service.
Key terms: application layer, APIs, SDKs, maximum output tokens, temperature, reasoning, basic model, reasoning model, self-report, observability signals
1.6.3 Installing and using a local LLM
Shows how to run an LLM locally with Ollama and uses that exercise to make context visible as inputs passed to a model on each call.
- Local model benefits and limits: Explain why someone might run an LLM on their own machine and how hardware limits which models they can use.
Why it matters: These tradeoffs help you choose a realistic local model and understand why you might use one. - Ollama interaction options: Describe how the section has you install and interact with a local model through Ollama, including the programmatic example.
Why it matters: The available interfaces let you try a local model directly or incorporate it into a program while observing its token use. - Context visibility across interfaces: Distinguish what the assistant interface, cloud API, and local model reveal or control about context, and state what they have in common.
Why it matters: This comparison connects the hands-on exercises to the book’s central idea that context is something a system supplies on each call.
Key terms: local LLM, quantized models, system memory, GPU memory, Ollama, CLI, GUI, local API server, token streaming, prompt and generation token counts
1.7 Summary
Where this is used later
- Understanding the framework landscape (chapter 10) · Assumes the idea of building context for LLM-based systems.
- Principles (chapter 7) · Context selection applies the principle of supplying enough relevant information without unnecessary material.
- Context management (chapter 7) · Distinguishes context management from concerns that apply across the stack.
- (chapter intro) (chapter 8) · Introduces the context engineering stack and its functional layers.
- Adoption models (chapter 9) · The context engineering stack supplies the context components whose control varies by posture.
- Context orchestration (chapter 7) · Develops the orchestration layer introduced here.
- Evaluation (chapter 8) · Develops systematic evaluation of system behavior.
- Observability (chapter 8) · Develops visibility into context during execution.
Concepts explained in this chapter
- context engineeringContext engineering is the discipline of selecting, organizing, and updating information supplied to a model in an appropriate form and at an appropriate time.
- AI agentAn AI agent is a system that pursues goals by interpreting tasks, planning actions, using resources, and adapting its behavior with limited human intervention.
- context windowA context window is the bounded amount of token context a language model can process for an inference request.
- context sourceA context source is a source of information supplied to an AI model for interpreting or executing a task.
- agent harnessA harness is a surrounding system of guides, tools, constraints, and feedback loops that shapes an AI agent's behavior within an environment.
- stale dataStale data is information that is outdated because it does not reflect events or changes after the data’s last update.
- working memoryShort-term memory is information about the current user-agent session, including prior turns and agent actions, that ends with the session.
- context assemblyContext selection is the choice of which information enters an AI system's active context.
- parameterA model parameter is a learned numerical value, such as a weight or bias, that determines how a model transforms its inputs.
- reasoning modelA language model optimized to solve demanding problems by generating intermediate reasoning steps before a final output.
- working contextModel context is the information available to a model during inference that influences its output.
- agent orchestrationAgentic orchestration coordinates an agent's actions, tools, and execution steps toward a task.
- context managementContext management is the shaping of context through selecting, writing, compressing, and isolating information before model consumption.
- prompt injectionPrompt injection is an attack that attempts to manipulate an AI system by introducing instructions into its input or context.
- memoryInformation retained across interactions or steps so an AI system can use it later.
- external memoryNon-parametric memory is information outside a model that can be saved, retrieved, and updated at runtime.
- stateInformation representing a system's current situation or progress.
- promptA prompt is an input supplied to a language model to elicit a response or guide its behavior.
- instructionA directive supplied to an AI system that specifies desired behavior, constraints, or task requirements.
- instruction fileA project-specific document or configuration that gives operational instructions to an AI agent.
- token limitA token budget is a limit on the tokens allocated to an AI system or request.
- ReActReAct is an agent reasoning pattern that alternates between internal reasoning and external actions such as tool invocation.
- sandboxingExecution of generated code or commands within a restricted environment that limits their access and effects.
- attention budgetAn attention budget is the finite capacity a model has to distribute attention across the information it receives.
- continuityContinuity is the preservation of relevant state and information across interaction turns, sessions, or workflow steps.
- groundingGrounding is the practice of supplying a model with evidence that supports or constrains its generated response.
- temperatureAn inference parameter that controls the randomness of token selection during generation.
- sessionAn ongoing interaction between a user and an assistant.
- parametric knowledgeParametric memory is knowledge embedded in a model’s parameters through training.
- tool useTool calling is a model capability for requesting that an external tool or operation be invoked.
- MCPThe Model Context Protocol is a protocol that enables external servers to expose tools, resources, and prompts to AI applications.
- command-line interfaceA command-line interface is a text-based interface through which users interact with a computer program by entering commands.
- tool descriptionA tool definition specifies a tool's name, purpose, and expected arguments so a system can determine how to invoke it.
- tool resultA tool output is information returned by a tool after it runs, such as a response, computed value, or command result.
- prompt engineeringPrompt engineering is the practice of designing and refining inputs to a language model to influence the quality and character of its output.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
