Chapter 12: The state of the art in context engineering
This chapter introduction previews a synthesis of context engineering’s mathematical foundations, research challenges, technical frontiers, applications, and future directions.
After this chapter you will be able to
- Context conditioning of model outputs. Explain that context engineering conditions a language model’s probability distribution to encourage useful, grounded, and policy-compliant outputs.
- Conditional output probabilities. Explain what Pθ(Y | C) represents and how the probability of an output sequence depends on each next token’s preceding tokens and supplied context.
- Parameters versus runtime context. Distinguish changing model parameters through model development from changing the context assembled for inference, and explain the intended effect on outputs.
- Six assembled context sources. Describe the six sources the assembly function A can use to produce context, and distinguish this structured runtime artifact from a static prompt string.
- Assembly beyond concatenation. Describe the decisions the assembly function makes beyond joining inputs, including source selection, token allocation, ordering, presentation, and conflicts.
- Task-specific context optimization. Explain how a family of context-management functions generates context for each task and is chosen to maximize expected task quality across tasks.
- Constrained context tradeoffs. Justify why maximizing context length is not the optimization goal, naming practical constraints and ways extra context can hurt.
- Answer-informative context selection. Explain why useful retrieval is judged by how much a passage reduces uncertainty about the desired answer, given the query, rather than by similarity to the query alone. Describe how practical systems approximate this objective when the answer is unavailable.
- Task-relevant compression efficiency. Explain why context compression aims for high task utility per token while removing redundancy. Describe what must be preserved to avoid changing the model’s decision or undermining its answer.
- Bayesian context inference. Explain why context is treated as an inferred variable and how prior beliefs and the current query contribute to a posterior belief over candidate contexts.
- Four context quality dimensions. Explain how relevance, completeness, consistency, and efficiency contribute to context quality and how teams can estimate them in practice.
- Application dependent quality weights. Explain why the quality dimensions receive different weights for different applications and why documenting those choices matters.
- Upstream context failure diagnosis. Explain how a context quality framework helps teams trace flawed answers to upstream pipeline components rather than attributing every failure to the model.
- Unified context pipeline. Trace how the support agent assembles instructions, policy passages, a tool schema, memory, ticket state, and the user request, then selects and compresses information to fit a token limit.
- Completeness failure in compression. Explain why a short, relevant-looking ticket summary can still lead to the wrong return decision, and identify where the quality breakdown occurs.
- Relevant context over volume. Explain why putting more information into the context window can make reasoning worse rather than better.
- Task-dependent context routing. Describe why context selection requires a policy for choosing among ways to supply information, rather than always using long context or RAG.
- Task-suited context representation. Explain why a context pipeline must choose how to represent and update task context, not just which passages or memories to include.
- Grounding against verified sources. Explain that context grounding keeps generated claims aligned with verified source material, and describe how the section illustrates that idea in code review or documentation.
- Sources of grounding failures. Trace how a generated answer can lose grounding at different points in a RAG pipeline, from finding and choosing evidence to using it in the final answer.
- Stale memory failure mode. Explain how accumulated information becomes stale and how that can affect an agent’s behavior when information conflicts.
- Lossless context separation. Describe how Lossless Context Management separates preserved history from the context sent to the model, and why that separation helps.
- Long-context privacy gap. Explain why longer, more personalized contexts can increase sensitive-information leakage even when private material is labeled and privacy instructions remain present.
- Persistent memory poisoning. Describe how an attacker can use a seemingly benign task to plant a payload in long-term memory and affect later agent sessions.
- Governed context artifacts. Describe how a filesystem abstraction could make context artifacts subject to consistent access and governance controls.
- Long-context capacity and degradation. Explain why accepting a longer prompt does not mean a model will reliably use the needed evidence, even when that evidence is present.
- Task-specific context thresholds. Describe how a team should decide when to compact or retrieve context before relying on a large window.
- Long-context KV cache bottleneck. Explain why longer context windows increase KV-cache memory demands during inference.
- Context construction efficiency alternative. Distinguish hardware-aware cache optimization from optimizing context construction, including the roles of the two agents in the example.
- Governed persistent memory. Explain what modern memory systems retain across interactions and why their operations are treated as an explicit lifecycle.
- Memory architecture tradeoff. Distinguish LCM's hierarchical-summary approach from file-system-style context infrastructure, and explain the failure-cost question used to choose between them.
- Structured multi-agent communication. Explain why communication is an architectural part of a collaborative agent system and why shared context is more reliable in structured artifacts than in unconstrained conversation.
- Inspectability of partner assessments. Describe how an agent may assess another agent from prior exchanges and why a production system should record the resulting judgment explicitly.
- Context as a measurement target. Explain why context observability measures more than final answers, and describe how ContextBench and AgentOps make context construction or agent failures inspectable.
- Traceable context construction. Describe how addressable context artifacts, metadata, event history, and preserved summary history let teams inspect context construction and trace a model’s claim to its sources.
- Shared cross-domain context requirements. Explain which context engineering capabilities the six representative domains share and why context capacity must be allocated according to the constraints that matter most in each domain.
- Domain-specific context priorities. Distinguish the priorities of healthcare and law, finance and scientific research, and education and enterprise knowledge management, including the context practices the section associates with each pair.
- Standard context interfaces. Explain how a standard context interface could make information usable across agents, tools, model providers, and frameworks. Describe the kinds of information the interface should carry, such as trust, permissions, freshness, and allowed use.
- Personal and organizational memory roles. Distinguish how personal memory supports long-term adaptation from how organizational memory preserves shared decisions and procedures.
- Memory research and rights. Describe the research directions named for memory and explain why the section calls for memory with rights.
- Governed enterprise agent ecosystems. Explain how governed agent ecosystems differ from isolated agents and why governance belongs in their architecture. Describe the defined identities, roles, permissions, tool scopes, escalation paths, and audit logs associated with enterprise agents.
- Agent Loop Control. Explain how loop engineering differs from direct prompting and describe what the control system does between iterations.
- Unattended Loop Safeguards. Explain the risks of unattended iterations and the safeguards the section calls for.
- Directed graph workflow structure. Describe an agentic system as a directed graph whose nodes do work and whose edges specify permitted transitions. Explain that nodes may be deterministic code, model calls, or complete agents.
- Predefined paths and cycles. Justify using graphs for production workflows with describable paths, including cycles for retries, requests for information, revisions, or human approval. Distinguish those workflows from exploratory work that needs more adaptability.
- Context routing through edges. Explain that each graph transition determines what context the next node receives, making context movement an explicit part of the system design.
- Context-aware process evaluation. Explain how context-aware evaluation extends answer-quality assessment to the quality of the context and processes behind an answer, and describe its move toward automated monitoring in production.
- Feedback-Driven Context Artifacts. Explain that retrieval queries, ranking rules, prompt templates, compression policies, and formatting conventions can be evaluated, revised, and versioned rather than remaining fixed developer choices. Describe how task traces and other feedback can guide improvements to the functions that assemble context.
- Gated Improvement Loop. Trace the four stages of the closed loop: recording assembled context and outcomes, scoring the outcome, proposing a change to one context function, and admitting it only if it improves a held-out evaluation set. Explain why incremental playbook updates avoid rewriting an entire instruction set after one failure.
- Recursive Improvement Boundaries. Distinguish recursive self-improvement from the basic loop: a meta-level agent revises the skills that construct future context artifacts, not model weights. Explain the risk of compounding drift and the safeguards of an externally frozen evaluation set, bounded changes, and human approval before production.
- Primary sources for direct evidence. Explain what makes a source primary and identify the original research and official documentation the section recommends monitoring.
- Secondary sources for practical interpretation. Describe how secondary sources interpret primary material and why practitioner writing and technical communities are useful to a context engineering radar.
- Tertiary indexes for discovery. Explain how tertiary sources support discovery and what cautions apply when using their lists.
- Reusable base knowledge layer. Explain why raw documents are compiled into a persistent Markdown wiki and how it differs from repeatedly retrieving disconnected files. Describe how the wiki becomes reusable context for agents.
- Source discipline and verification. Explain what makes the resulting base knowledge trustworthy enough to reuse and how to check a claim found through its graph.
What it builds on
- Defining context engineering (chapter 1) · Assumes the reader’s earlier introduction to context engineering.
- The context window (chapter 1) · The basic idea of information supplied to an LLM as context.
- Taxonomy for context sources (chapter 1) · The taxonomy of six context sources.
- Context management (chapter 7) · Context management as a concern in assembling model inputs.
- Retrieval models and vector databases (chapter 3) · Prior discussion of retrieval models provides context for judging retrieved passages.
- Context compression (chapter 7) · Prior discussion of context compression provides the operational background.
- Evaluation (chapter 8) · Evaluation methods used to estimate measurable properties.
- Governance (chapter 9) · Governance as an explicit consideration in AI systems.
Section by section
Introduction
This chapter introduction previews a synthesis of context engineering’s mathematical foundations, research challenges, technical frontiers, applications, and future directions.
12.1 Mathematical foundations
Introduces the shared mathematical idea behind context engineering: shaping a language model’s output probabilities through context.
- Context conditioning of model outputs: Explain that context engineering conditions a language model’s probability distribution to encourage useful, grounded, and policy-compliant outputs.
Why it matters: This gives the book’s varied context-engineering techniques a common purpose.
Key terms: context engineering, probability distribution, grounded, policy-compliant
12.1.1 Context formalization
Formalizes context engineering as assembling model-readable context that changes the probabilities of an LLM’s outputs without changing its parameters.
- Conditional output probabilities: Explain what Pθ(Y | C) represents and how the probability of an output sequence depends on each next token’s preceding tokens and supplied context.
Why it matters: This gives a precise way to describe how supplied context influences generation. - Parameters versus runtime context: Distinguish changing model parameters through model development from changing the context assembled for inference, and explain the intended effect on outputs.
Why it matters: It explains what application developers can improve without retraining a model. - Six assembled context sources: Describe the six sources the assembly function A can use to produce context, and distinguish this structured runtime artifact from a static prompt string.
Why it matters: The six sources show what can be deliberately assembled for a model at runtime. - Assembly beyond concatenation: Describe the decisions the assembly function makes beyond joining inputs, including source selection, token allocation, ordering, presentation, and conflicts.
Why it matters: These choices determine what information the model actually receives and how it encounters it. - Model-readable context projection: Explain why relevant context extends beyond text or chat history and how the final context relates to the wider situation.
Why it matters: It prevents treating chat history as the whole situation an application may need to represent.
Key terms: Pθ(Y | C), output sequence, conditional probabilities, token, parameters θ, context C, inference time, conditional distribution, assembly function A, instructions
12.1.2 Context optimization
This section frames context engineering as choosing a task-specific context pipeline to improve evaluated task quality under practical constraints.
- Task-specific context optimization: Explain how a family of context-management functions generates context for each task and is chosen to maximize expected task quality across tasks.
Why it matters: It identifies what the pipeline is optimizing: quality across tasks, not the amount of context supplied. - Task quality evaluation: Describe what the ideal output and reward function represent, and how teams approximate reward in practice.
Why it matters: The optimization objective is useful only if task quality can be assessed in practice. - Constrained context tradeoffs: Justify why maximizing context length is not the optimization goal, naming practical constraints and ways extra context can hurt.
Why it matters: It prevents treating a larger context window as a reason to include everything available.
Key terms: context management pipeline, family of functions, task instance, task distribution, task-specific context, expected task quality, ideal output, target behavior, reward, evaluation function
12.1.3 Context selection and compression
This section uses an information-theoretic view to explain how retrieval should select answer-relevant evidence and compression should preserve task utility per token.
- Answer-informative context selection: Explain why useful retrieval is judged by how much a passage reduces uncertainty about the desired answer, given the query, rather than by similarity to the query alone. Describe how practical systems approximate this objective when the answer is unavailable.
Why it matters: It helps distinguish evidence that advances an answer from passages that merely echo the question. - Task-relevant compression efficiency: Explain why context compression aims for high task utility per token while removing redundancy. Describe what must be preserved to avoid changing the model’s decision or undermining its answer.
Why it matters: A shorter context helps only if it retains the evidence and constraints the task depends on.
Key terms: mutual information, context selection, answer relevance, embedding similarity, information density, task utility per token, context compression, structured summaries
12.1.4 Context inference
The section presents context selection as inference under uncertainty, using prior beliefs, the current query, and observed outcomes to guide what information a system supplies.
- Bayesian context inference: Explain why context is treated as an inferred variable and how prior beliefs and the current query contribute to a posterior belief over candidate contexts.
Why it matters: It gives a way to reason about context choices when relevance is uncertain. - Outcome-informed context choices: Describe how practical signals and observed outcomes can change which contexts are favored, including the examples of a helpful document and an unhelpful memory.
Why it matters: Feedback from actual answers can make later context choices more useful. - Multi-agent context decisions: Explain how uncertainty can guide an orchestrator's decisions about distributing information and sharing state among agents.
Why it matters: An orchestrator must make context-sharing choices even when it is unsure which choice will work best.
Key terms: Bayesian inference, candidate contexts, posterior belief, prior, likelihood, history, retrieval scores, recency weights, user feedback, tool success
12.1.5 Context quality
The section defines measurable dimensions of context quality, explains why their weights depend on the application, and shows how the resulting framework helps diagnose upstream failures.
- Four context quality dimensions: Explain how relevance, completeness, consistency, and efficiency contribute to context quality and how teams can estimate them in practice.
Why it matters: These dimensions give teams more precise ways to assess context than judging an answer alone. - Application dependent quality weights: Explain why the quality dimensions receive different weights for different applications and why documenting those choices matters.
Why it matters: A quality score reflects decisions about what a particular system should prioritize. - Upstream context failure diagnosis: Explain how a context quality framework helps teams trace flawed answers to upstream pipeline components rather than attributing every failure to the model.
Why it matters: Tracing context failures to their source helps teams fix the responsible pipeline component.
Key terms: relevance, completeness, consistency, efficiency, token budget, weights, governance, weighting scheme, quality function, upstream
12.1.6 Putting the formulations together
This section uses a customer-support example to show how context assembly, selection, compression, and quality assessment work together to shape an LLM’s answer.
- Unified context pipeline: Trace how the support agent assembles instructions, policy passages, a tool schema, memory, ticket state, and the user request, then selects and compresses information to fit a token limit.
Why it matters: A context budget makes choosing and preserving the right information more important than simply including more text. - Completeness failure in compression: Explain why a short, relevant-looking ticket summary can still lead to the wrong return decision, and identify where the quality breakdown occurs.
Why it matters: Separating quality into terms helps locate a bad answer’s cause rather than mistaking a concise, relevant summary for an adequate one. - Formulations as diagnostic abstractions: Describe how the formulations help diagnose context failures without claiming that common frameworks compute them directly or that they form a complete theory.
Why it matters: These abstractions are useful for diagnosing failures even when their quantities are not calculated directly.
Key terms: assembly function 𝐴, 𝐿𝑚𝑎𝑥, mutual information criterion, Efficiency, quality function, Relevance, completeness, Compress, mathematical formulations, available metric
12.2 Research challenges
Introduces unresolved research challenges that limit the reliability of context-aware systems and points to a table and later subsections for details.
12.2.1 Context selection
This section explains why selecting useful context is difficult and why systems must choose both how to obtain context and how to represent it for a task.
- Relevant context over volume: Explain why putting more information into the context window can make reasoning worse rather than better.
Why it matters: Selecting relevant information matters more than simply supplying more of it. - Task-dependent context routing: Describe why context selection requires a policy for choosing among ways to supply information, rather than always using long context or RAG.
Why it matters: A routing policy lets the system choose an approach suited to the task instead of relying on one default. - Task-suited context representation: Explain why a context pipeline must choose how to represent and update task context, not just which passages or memories to include.
Why it matters: The form in which context is maintained is itself part of the selection decision.
Key terms: context selection, retrieval noise, context rot, context window, LaRA, long-context processing, RAG, routing problem, policy, meta context engineering
12.2.2 Context grounding
This section defines context grounding as aligning generated claims with verified sources and explains why failures in a RAG pipeline can make that alignment difficult.
- Grounding against verified sources: Explain that context grounding keeps generated claims aligned with verified source material, and describe how the section illustrates that idea in code review or documentation.
Why it matters: Grounding gives you a way to ask whether an AI-generated claim is supported by the material it should rely on. - Sources of grounding failures: Trace how a generated answer can lose grounding at different points in a RAG pipeline, from finding and choosing evidence to using it in the final answer.
Why it matters: An unsupported answer may reflect a problem anywhere along the evidence-to-answer path, not just a problem with retrieval.
Key terms: context grounding, verified source material, HalluJudge, codified context, RAG evaluation, retriever, reranker, generator, synthesis prompt
12.2.3 Stale memory
This section explains why outdated agent memory causes errors and presents three approaches to keeping usable context accurate over time.
- Stale memory failure mode: Explain how accumulated information becomes stale and how that can affect an agent’s behavior when information conflicts.
Why it matters: Recognizing stale memory helps explain why retaining more history does not necessarily make an agent more reliable. - Lossless context separation: Describe how Lossless Context Management separates preserved history from the context sent to the model, and why that separation helps.
Why it matters: Keeping the original record separate from the prompt makes compact context less dependent on an aging summary. - ACC compressed cognitive state: Describe how the Agent Cognitive Compressor limits the influence of stale context over long horizons.
Why it matters: ACC illustrates an approach to preventing a long history from carrying obsolete material into future actions. - ACE evolving playbooks: Explain how Agentic Context Engineering updates context and what its updates aim to preserve or discard.
Why it matters: ACE shows why maintaining useful memory involves revising strategies, not simply saving past context.
Key terms: stale memory, decision drift, Lossless Context Management, immutable store, active context, summary nodes, token budget, Agent Cognitive Compressor, ACC, artifact recall
12.2.4 Governance
This section explains why growing and persistent context creates privacy and security risks, and why governance needs architectural controls over context artifacts.
- Long-context privacy gap: Explain why longer, more personalized contexts can increase sensitive-information leakage even when private material is labeled and privacy instructions remain present.
Why it matters: Adding privacy instructions alone may not protect the larger stores of user history used for personalization. - Persistent memory poisoning: Describe how an attacker can use a seemingly benign task to plant a payload in long-term memory and affect later agent sessions.
Why it matters: A malicious memory entry can outlast the task that introduced it. - Architectural injection risks: Explain why agent security depends on where context enters a system and on choices about roles, communication, and memory.
Why it matters: Security review must consider context handoffs and agent architecture, not just the wording of individual prompts. - Governed context artifacts: Describe how a filesystem abstraction could make context artifacts subject to consistent access and governance controls.
Why it matters: Governance becomes easier to enforce when context is managed as identifiable artifacts rather than unstructured prompt text.
Key terms: context windows, sensitive information leakage, privacy, personalization, attention problem, memory poisoning, long-term memory, Zombie Agent, indirect prompt injection, AgentSentry
12.3 Technical frontiers
Introduces technical frontiers as practical innovations aimed at reducing the failures identified by research challenges, with details deferred to the table and following subsections.
12.3.1 Long-context models
Long-context models provide more input capacity, but reliable use of that capacity requires measuring degradation and managing context before accuracy falls.
- Long-context capacity and degradation: Explain why accepting a longer prompt does not mean a model will reliably use the needed evidence, even when that evidence is present.
Why it matters: A larger context window is useful capacity, not a guarantee of reliable understanding. - Task-specific context thresholds: Describe how a team should decide when to compact or retrieve context before relying on a large window.
Why it matters: Testing the actual model and workload gives a team a safer threshold than relying on the advertised window size.
Key terms: long-context models, long-context degradation, context rot, compaction, retrieval threshold, state recovery
12.3.2 Context caching
This section explains why KV caching strains GPU memory in long-context inference and contrasts cache-level optimization with improving the context supplied to the model.
- Long-context KV cache bottleneck: Explain why longer context windows increase KV-cache memory demands during inference.
Why it matters: This explains why supporting longer contexts is also an inference-memory problem. - TailorKV cache footprint approach: Describe how TailorKV addresses cache footprint and the limitation the section identifies for selective loading.
Why it matters: It illustrates both a hardware-aware response to cache pressure and a constraint on that response. - Context construction efficiency alternative: Distinguish hardware-aware cache optimization from optimizing context construction, including the roles of the two agents in the example.
Why it matters: It shows that reducing unnecessary context is another route to efficiency, not just making the cache cheaper to run.
Key terms: key-value (KV) cache, long-context inference, GPU memory, TailorKV, quantization, offloading, selective loading, meta context engineering, meta-level agent, base-level agent
12.3.3 Memory architectures
This section describes governed memory across interactions and compares two architectures for preserving and managing context.
- Governed persistent memory: Explain what modern memory systems retain across interactions and why their operations are treated as an explicit lifecycle.
Why it matters: Persistent memory needs management throughout its lifecycle, not just a way to save and retrieve information. - Memory architecture tradeoff: Distinguish LCM's hierarchical-summary approach from file-system-style context infrastructure, and explain the failure-cost question used to choose between them.
Why it matters: The comparison ties an architecture choice to the kind of context failure a team most needs to avoid.
Key terms: persistent knowledge, governed memory systems, lifecycle operations, LCM, hierarchical summary DAG, immutable store, file-system-style context infrastructure, governed namespace, lineage
12.3.4 Multi-agent context sharing
The section explains how agents can share context through structured artifacts and make evidence-backed judgments about other agents inspectable.
- Structured multi-agent communication: Explain why communication is an architectural part of a collaborative agent system and why shared context is more reliable in structured artifacts than in unconstrained conversation.
Why it matters: How agents communicate affects whether they can reliably use one another’s work. - Inspectability of partner assessments: Describe how an agent may assess another agent from prior exchanges and why a production system should record the resulting judgment explicitly.
Why it matters: An explicit, evidence-backed judgment lets an operator see why one agent changed how it works with another.
Key terms: multi-agent context sharing, structured artifacts, typed handoff packages, previous exchanges, explicit context artifact, supporting evidence, confidence level
12.3.5 Context observability
This section shifts observability from judging final answers to auditing the context that produced them and tracing claims back to their sources.
- Context as a measurement target: Explain why context observability measures more than final answers, and describe how ContextBench and AgentOps make context construction or agent failures inspectable.
Why it matters: Inspecting the context behind an answer helps teams investigate failures that the answer alone cannot explain. - Traceable context construction: Describe how addressable context artifacts, metadata, event history, and preserved summary history let teams inspect context construction and trace a model’s claim to its sources.
Why it matters: A traceable history lets developers investigate which sources and transformations influenced a model’s claim.
Key terms: context observability, ContextBench, AgentOps, auditable runtime process, file-system-style context infrastructure, addressable artifacts, metadata, event history, LCM, summary nodes
12.4 Domain-specific applications
This section explains the context engineering capabilities shared across high-stakes domains and how different domains prioritize competing demands on context.
- Shared cross-domain context requirements: Explain which context engineering capabilities the six representative domains share and why context capacity must be allocated according to the constraints that matter most in each domain.
Why it matters: The same foundations support different applications, but the available context must serve each domain’s most important constraints. - Domain-specific context priorities: Distinguish the priorities of healthcare and law, finance and scientific research, and education and enterprise knowledge management, including the context practices the section associates with each pair.
Why it matters: Choosing the wrong priority can leave an answer unsupported, outdated, numerically wrong, or based on unreliable memory.
Key terms: authoritative evidence retrieval, governance and access policies, auditable execution traces, context capacity, completeness, authoritative source, temporal freshness, numerical precision, deterministic tools, memory quality
12.5 Future directions
Introduces the expectation that context engineering may mature into an infrastructure layer for AI systems and points to a table of future directions.
12.5.1 Interoperable context
The section explains why interoperable context needs shared interfaces, presents a file-system-style abstraction as one possible approach, and uses OKF as an early example.
- Standard context interfaces: Explain how a standard context interface could make information usable across agents, tools, model providers, and frameworks. Describe the kinds of information the interface should carry, such as trust, permissions, freshness, and allowed use.
Why it matters: A shared description lets different parts of a system exchange context without losing the rules for using it. - File-system-style context resources: Describe the file-system-style abstraction proposed as one possible path to interoperable context. Explain what it treats as resources and what common capabilities those resources need.
Why it matters: This abstraction shows how otherwise different context sources might be accessed and governed consistently. - OKF knowledge representation: Describe how OKF represents curated knowledge and why it illustrates a move toward interoperable context. State the limits of the claim made for it in this section.
Why it matters: OKF gives a concrete example of knowledge that carries its own source and freshness information across different ways of using it.
Key terms: standardized context interfaces, formal context interface, standard context blocks, file-system-style context infrastructure, context resources, lineage, lifecycle control, open knowledge format (OKF), provenance, staleness date
12.5.2 Memory infrastructure
This section presents personal and organizational memory as emerging infrastructure and argues that it needs clear rights for managing stored information.
- Personal and organizational memory roles: Distinguish how personal memory supports long-term adaptation from how organizational memory preserves shared decisions and procedures.
Why it matters: The two kinds of memory serve different needs as memory becomes core infrastructure. - Memory research and rights: Describe the research directions named for memory and explain why the section calls for memory with rights.
Why it matters: Useful long-term memory also needs explicit ways to manage and remove what it retains.
Key terms: personal memory, organizational memory, memory infrastructure, dynamic extraction, graph organization, retrieval, forgetting, memory with rights
12.5.3 Governed agent ecosystems
This subsection presents governed agent ecosystems as an architectural direction for enterprise agents, with governance built in rather than added after deployment.
- Governed enterprise agent ecosystems: Explain how governed agent ecosystems differ from isolated agents and why governance belongs in their architecture. Describe the defined identities, roles, permissions, tool scopes, escalation paths, and audit logs associated with enterprise agents.
Why it matters: It frames privacy, security, and operational control as part of how enterprise agents are designed to work together.
Key terms: governed agent ecosystems, isolated agents, identities, roles, permissions, tool scopes
12.5.4 Loop engineering
Defines loop engineering as a recurring agent-control process and explains the infrastructure and safeguards needed to run it reliably.
- Agent Loop Control: Explain how loop engineering differs from direct prompting and describe what the control system does between iterations.
Why it matters: The loop changes who prepares each prompt and decides the next step. - Recurring Context Infrastructure: Describe why an agent loop needs more than a conversation history to support repeated and potentially concurrent work.
Why it matters: These supports let the loop carry work forward without relying solely on an ever-growing chat. - Unattended Loop Safeguards: Explain the risks of unattended iterations and the safeguards the section calls for.
Why it matters: Repeating an agent's work also repeats—and can compound—its mistakes and costs.
Key terms: loop engineering, agent harness, direct prompting, control system, context construction, durable state, task discovery, isolation, reusable skills, tool connectors
12.5.5 Graph engineering
This section explains how directed graphs make agent workflows and context routing explicit, and where predetermined paths help or hinder agents.
- Directed graph workflow structure: Describe an agentic system as a directed graph whose nodes do work and whose edges specify permitted transitions. Explain that nodes may be deterministic code, model calls, or complete agents.
Why it matters: This gives you a concrete way to describe which parts of an agentic workflow do work and which paths they may take. - Predefined paths and cycles: Justify using graphs for production workflows with describable paths, including cycles for retries, requests for information, revisions, or human approval. Distinguish those workflows from exploratory work that needs more adaptability.
Why it matters: The value of a graph depends on whether the task benefits more from known routes or from freedom to explore. - Context routing through edges: Explain that each graph transition determines what context the next node receives, making context movement an explicit part of the system design.
Why it matters: A graph describes not only where work goes next, but also what information accompanies it.
Key terms: directed graph, nodes, edges, deterministic code, model call, agent, production workflows, cycles, exploratory work, adaptivity
12.5.6 Context-aware evaluation
This section argues that evaluation should assess how an AI system uses context, not just the quality of its final answer, and increasingly support continuous monitoring.
- Context-aware process evaluation: Explain how context-aware evaluation extends answer-quality assessment to the quality of the context and processes behind an answer, and describe its move toward automated monitoring in production.
Why it matters: A good final answer alone does not show whether the system used context reliably and appropriately.
Key terms: context-aware evaluation, process-level evaluation, continuous quality monitoring
12.5.7 Self-improving context pipelines
The section explains how feedback can improve versioned context-building functions through an evaluated, gated loop, and how recursive improvement extends that loop while requiring safeguards against drift.
- Feedback-Driven Context Artifacts: Explain that retrieval queries, ranking rules, prompt templates, compression policies, and formatting conventions can be evaluated, revised, and versioned rather than remaining fixed developer choices. Describe how task traces and other feedback can guide improvements to the functions that assemble context.
Why it matters: It identifies what the pipeline improves and what evidence can justify an improvement. - Gated Improvement Loop: Trace the four stages of the closed loop: recording assembled context and outcomes, scoring the outcome, proposing a change to one context function, and admitting it only if it improves a held-out evaluation set. Explain why incremental playbook updates avoid rewriting an entire instruction set after one failure.
Why it matters: The gate distinguishes a measured improvement from a plausible-sounding revision prompted by one failure. - Recursive Improvement Boundaries: Distinguish recursive self-improvement from the basic loop: a meta-level agent revises the skills that construct future context artifacts, not model weights. Explain the risk of compounding drift and the safeguards of an externally frozen evaluation set, bounded changes, and human approval before production.
Why it matters: Improving the improvement process can amplify mistakes unless each revision remains testable and controlled.
Key terms: self-improving context pipeline, context functions, artifacts, feedback, trace store, evaluator, reflection step, gate, held-out evaluation set, ACE
12.6 Hands-on
This section introduces upcoming hands-on activities for turning context-engineering research and industry trends into an active knowledge-acquisition practice.
12.6.1 Building a context engineering radar
This section shows how to build a context engineering radar from primary, secondary, and tertiary sources and keep it current as the field changes.
- Primary sources for direct evidence: Explain what makes a source primary and identify the original research and official documentation the section recommends monitoring.
Why it matters: Direct sources let you check new claims against the original work or official release. - Secondary sources for practical interpretation: Describe how secondary sources interpret primary material and why practitioner writing and technical communities are useful to a context engineering radar.
Why it matters: Interpretation from practitioners can help you evaluate how a new idea works outside a paper or benchmark. - Tertiary indexes for discovery: Explain how tertiary sources support discovery and what cautions apply when using their lists.
Why it matters: Indexes are good starting points, but they should not be treated as proof that a technique is current or sound. - Maintained radar versus printed snapshots: Distinguish the book's representative source tables from the maintained radar in its companion repository.
Why it matters: Using the maintained radar avoids treating a dated reading list as permanently current.
Key terms: primary sources, arXiv, model vendors' documentation, framework documentation, secondary sources, technical communities, tertiary sources, curated GitHub lists, topic maps, technology radar
12.6.2 Building a base knowledge
This section shows how to turn selected research materials into a persistent, navigable base knowledge layer that agents can reuse while preserving a path back to sources.
- Reusable base knowledge layer: Explain why raw documents are compiled into a persistent Markdown wiki and how it differs from repeatedly retrieving disconnected files. Describe how the wiki becomes reusable context for agents.
Why it matters: The base knowledge layer turns a reading collection into context agents can return to across tasks. - Graphify knowledge graph workflow: Describe how Graphify processes project materials and how the resulting graph helps a reader navigate the literature.
Why it matters: The graph helps readers find related ideas without treating the graph itself as the source paper. - Source discipline and verification: Explain what makes the resulting base knowledge trustworthy enough to reuse and how to check a claim found through its graph.
Why it matters: A navigable knowledge base is useful only if its claims remain distinguishable from inference and checkable against sources.
Key terms: base knowledge layer, persistent Markdown wiki, reusable context, Graphify, entities and relationships, communities, interactive HTML graph, input discipline, attributed, unverified notes
12.7 Summary
Concepts explained in this chapter
- context engineeringContext engineering is the discipline of selecting, organizing, and updating information supplied to a model in an appropriate form and at an appropriate time.
- compactionA process that summarizes an agent conversation and discards lengthy details while preserving high-level task information.
- assembly functionAn assembly function combines instructions, retrieved passages, tool schemas, memory, state, and a request into a model’s context.
- working contextModel context is the information available to a model during inference that influences its output.
- AI governanceGovernance comprises the structures, processes, policies, and controls that direct how AI systems are developed and operated.
- stale dataStale data is information that is outdated because it does not reflect events or changes after the data’s last update.
- context assemblyContext selection is the choice of which information enters an AI system's active context.
- key-value cacheA key-value cache stores previously computed transformer attention keys and values for reuse during generation.
- prompt cachingPrompt caching reuses previously computed representations of prompt context, particularly shared prefixes, to avoid processing them again.
- groundingGrounding is the practice of supplying a model with evidence that supports or constrains its generated response.
- context windowA context window is the bounded amount of token context a language model can process for an inference request.
- LLM-as-a-judgeAn evaluator is a component that assesses outcomes against specified criteria or metrics.
- privacyPrivacy is the protection of people or their information from unauthorized collection, access, use, or disclosure.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
