liveBook · Manning

Context window

As explained in Agent Design Patterns

A context window is the maximum amount of input information a language model can consider in a single invocation.

A mortgage-advice agent may need user history, lending knowledge, financial calculations, and prior reasoning. Putting all of that into one model invocation can make the workflow unreliable: language-model performance deteriorates as context grows, even before a hard input limit is reached. The useful design boundary is therefore the effective context window—the amount of information a model can reasonably work with while maintaining quality. section 2.4 section 3.3.3

When a workflow approaches that boundary, choose how to reduce what any one model must handle:

  • Divide responsibilities among specialist agents. A supervisor can call mortgage analysts as needed rather than loading every task and its knowledge into one agent. section 2.4
  • Compact accumulated state. A compaction errand can filter, summarize, or distill conversation history just before it reaches the effective limit; the receiving agent must be designed to work from that summary. section 3.3.3
  • Decompose large inputs, process their parts, then recompose the results. Larger context windows alone do not eliminate retrieval errors or missed details in dense inputs. section 8.0

A growing conversation

Repeated user interactions can leave an agent with more relevant history than its next model call can handle well. A transcript-compaction errand compresses the conversation into a summary before the limit is reached. The next step should expect that summary rather than assume it still has the full transcript. section 3.3.3

Listing 3.5 Compaction_errand.py: Compaction errand to compress transcripts
class TranscriptCompactionErrand(Errand):
    def __init__(
        self, client: Client, compaction_prompt: str
    ):
        self.system_prompt = "You are a helpful assistant that compresses a transcript of an agent-user conversation into a single sentence. Output the compressed summary of the conversation in <summary> tags."
 
        self.compaction_prompt = "Compress the following transcript of an agent-user conversation into a single sentence:\n{transcript}"
 
        self.compaction_call = LMCall(
            client,
            [
                {
                    "role": "system",
                    "content": self.system_prompt,
                }
            ],
        )
        self.extraction_procedure = FunctionCall(
            self.extract_result
        )
 
    async def __call__(
        self, messages: list[dict[str, Any]]
    ) -> str:
        transcript: str = "\n----\n".join(
            [
                f"{message['role']}: {message['content']}"
                for message in messages
            ]
        )
        compaction_input: str = (
            self.compaction_prompt.format(
                transcript=transcript
            )
        )
        compaction_response: str = (
            await self.compaction_call(compaction_input)
        )
        return await self.extraction_procedure(
            compaction_response
        )
 
    def extract_result(self, response: str) -> Any:
        result_match = re.search(
            r"<summary>(.*)</summary>", response, re.DOTALL
        )
        if result_match:
            return result_match.group(1).strip()
        else:
            raise ValueError(
                "Compressed summary not found in the response"
            )

The transcript-compaction listing shows a Python-like Errand using an LMCall and a summary-extraction procedure. It illustrates the compaction component, not a complete threshold-detection policy. section 3.3.3

How Agent Design Patterns covers it

How deep it goes
Defines an effective token limit and explains compaction timing and receiver integration, but does not specify a numerical threshold.
Examples
Uses mortgage-advice analysts, a growing user–agent transcript, and large-input cases such as video and legal archives.
What it assumes
Assumes familiarity with agent workflows while defining errands as leaf routines and explaining the effective perception limit.
What you can do after
Diagnose/debug with it: Recognize context-growth failures and choose task decomposition or pre-limit transcript compaction.
How it connects
Connects agent teams, supervisor orchestration, compaction errands, conversation summarization, and decompose-process-recompose workflows.

Also explained in

Compare how 38 books explain context window →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Agent Design Patterns. The section links point to the parts of the book it draws on.