Chapter 4: The LLMAgent Class
This introduction connects the framework’s LLM and tool components to an agent and explains how to configure the chapter notebook’s LLM.
After this chapter you will be able to
- Agent Components and Capabilities. Explain how the backbone LLM and tools come together in LLMAgent, and identify the LLM capabilities its processing loop uses.
- Agent design requirements. Explain why LLMAgent needs an LLM, tools, and a way to execute user-submitted tasks.
- Task execution and results. Describe how run() relates Task inputs to TaskResult outputs, including the fields of each data structure.
- Awaitable run interface. Distinguish run() from an async coroutine and explain what its returned awaitable resolves to.
- Task and result attributes. Explain what the attributes of Task and TaskResult represent and how they associate a task with its result.
- Purpose based interaction templates. Explain how templates standardize interactions with the backbone LLM and how LLMAgentTemplates stores them.
- Agent initialization and registry. Describe how LLMAgent initializes its LLM, templates, and tools, including the registry and tools property.
- Tool addition and validation. Describe how add_tool() registers an additional tool and checks its name.
- Autonomous task execution interface. Explain what LLMAgent.run() takes and returns and its role in autonomous task execution.
- Discrete processing loop steps. Describe how the processing loop organizes task execution and the role of TaskHandler.
- Processing loop progression. Explain how the LLMAgent executes a task through sub-steps and decides when to stop.
- Sub-step data structures. Describe the roles of TaskStep, TaskStepResult, and NextStepDecision in representing and continuing task execution.
- TaskHandler role and awaitability. Explain TaskHandler’s role in LLMAgent and its relationship to run() and TaskResult.
- Task step association. Explain how a TaskStep is associated with its overarching Task.
- Next step decision representation. Describe how NextStepDecision is represented and where it is used.
- Task execution processing loop. Trace how run() and TaskHandler alternate between deciding and executing sub-steps until task execution stops.
- Structured next step decisions. Explain how get_next_step() uses the backbone LLM to choose another sub-step or a final result.
- Single turn substep chat. Describe how run_step() starts a sub-step chat and turns the backbone LLM's response into a TaskStepResult.
- Substep tool calling phase. Explain how tool calls are handled within a sub-step and what happens if the LLM requests a second round.
- Rollout history and formatting. Describe how TaskHandler builds the rollout and why its formatting matters for later sub-steps.
- Obfuscated Hailstone Tool. Explain why the example obscures the Hailstone tool’s name and description.
- Agent and Task Setup. Describe how the example sets up the LLMAgent and its Hailstone task.
- Asynchronous Task Result Retrieval. Describe how to wait for the Hailstone task and retrieve its final result, including what happens if result() is called too early.
- Hailstone Execution Rollout. Trace what the example’s rollout shows for the task starting at 4.
- Subagent composition and delegation. Explain how a coordinator and subagents built with LLMAgent can share work in a multi-agent system, and how subagents can be configured for different responsibilities.
What it builds on
- BaseTool: a blueprint for tools (chapter 2) · The framework’s tool blueprint.
- BaseLLM: a blueprint for LLMs (chapter 3) · The framework’s LLM blueprint.
- Setting the LLM (chapter 1) · Setting the LLM for the book’s examples.
- What is an LLM agent? (chapter 1) · The concept of an LLM agent and its backbone LLM.
- Defining BaseTool (chapter 2) · The base tool abstraction used by the agent’s tools.
- The AsyncBaseTool (chapter 2) · The asynchronous tool abstraction included in the Tool type.
- Defining BaseLLM (chapter 3) · The BaseLLM abstraction used by the agent.
- The processing loop (chapter 1) · The general processing loop introduced earlier.
Section by section
Introduction
This introduction connects the framework’s LLM and tool components to an agent and explains how to configure the chapter notebook’s LLM.
- Agent Components and Capabilities: Explain how the backbone LLM and tools come together in LLMAgent, and identify the LLM capabilities its processing loop uses.
Why it matters: This connects the components already built to an agent that can perform tasks. - Notebook LLM Configuration: Describe how the chapter notebook’s make_llm() selects an LLM and how a reader can configure the provider and model.
Why it matters: This lets you select an available LLM when following the chapter’s code.
Key terms: LLMAgent, backbone LLM, tools, processing loop, planning, tool-calling, make_llm(), OLLAMA_API_KEY, LLM_PROVIDER, OPENAI_API_KEY
4.1 LLMAgent: a simple LLM agent class
This section outlines the LLMAgent design, its task execution interface, and the data structures and attributes it uses.
- Agent design requirements: Explain why LLMAgent needs an LLM, tools, and a way to execute user-submitted tasks.
Why it matters: These requirements establish what the agent must bring together to perform a task. - Task execution and results: Describe how run() relates Task inputs to TaskResult outputs, including the fields of each data structure.
Why it matters: This defines the input and output contract for executing a task. - Awaitable run interface: Distinguish run() from an async coroutine and explain what its returned awaitable resolves to.
Why it matters: The distinction tells a caller how to obtain the task result without mistaking run() for a coroutine. - Supporting agent attributes: Describe the roles of tool_registry, templates, and add_tool() in LLMAgent.
Why it matters: These elements support tool access and interaction with the backbone LLM.
Key terms: LLMAgent, backbone LLM, tools, tool-call requests, run(), processing loop, Task, TaskResult, id_, instruction
4.1.1 Implementing Task and TaskResult
This section explains the attributes of Task and TaskResult and how they link a task to its result.
- Task and result attributes: Explain what the attributes of Task and TaskResult represent and how they associate a task with its result.
Why it matters: These attributes describe what the agent should do and connect the resulting output to the right task.
Key terms: Task, TaskResult, instruction, id_, task_id, content
4.1.2 Implementing LLMAgent: Part 1
This section implements the LLMAgent’s template storage, initialization, and tool registration, leaving its processing loop for later.
- Purpose based interaction templates: Explain how templates standardize interactions with the backbone LLM and how LLMAgentTemplates stores them.
Why it matters: Templates make recurring LLM interactions consistent while allowing their content to vary. - Agent initialization and registry: Describe how LLMAgent initializes its LLM, templates, and tools, including the registry and tools property.
Why it matters: This setup gives the agent its LLM and templates while making its tools accessible and efficiently searchable. - Tool addition and validation: Describe how add_tool() registers an additional tool and checks its name.
Why it matters: Adding tools safely lets an agent’s available capabilities grow without ambiguous names. - Logging and error support: Describe the purposes of the framework’s logger and errors modules.
Why it matters: These supporting modules help developers diagnose problems in the framework.
Key terms: templates, placeholder variables, LLMAgentTemplates, system message, __init__(), llm, tool_registry, tools, @property, add_tool()
4.2 Implementing a processing loop for LLMAgent
This section introduces the role of LLMAgent.run() and the discrete-step processing loop it will use to execute tasks.
- Autonomous task execution interface: Explain what LLMAgent.run() takes and returns and its role in autonomous task execution.
Why it matters: The run method is the entry point for an agent to carry out a task and return its result. - Discrete processing loop steps: Describe how the processing loop organizes task execution and the role of TaskHandler.
Why it matters: This division establishes the basic structure for implementing the agent's processing loop.
Key terms: LLMAgent, run(), Task, TaskResult, processing loop, TaskHandler, step execution
4.2.1 Designing a processing loop
This section designs the agent’s processing loop, its supporting data structures and task handler, and describes the ReAct loop as an alternative pattern.
- Processing loop progression: Explain how the LLMAgent executes a task through sub-steps and decides when to stop.
Why it matters: This loop turns the LLM’s planning and tool-calling abilities into a sequence of task-directed actions. - Sub-step data structures: Describe the roles of TaskStep, TaskStepResult, and NextStepDecision in representing and continuing task execution.
Why it matters: These structures represent each sub-step and the decision to continue or finish the task. - ReAct sub-step phases: Trace the Thought, Action, and Observation phases of a ReAct sub-step.
Why it matters: ReAct illustrates another way to structure a processing loop around reasoning, action, and feedback. - TaskHandler role and awaitability: Explain TaskHandler’s role in LLMAgent and its relationship to run() and TaskResult.
Why it matters: TaskHandler connects the agent’s awaitable run interface to management of the processing loop.
Key terms: processing loop, sub-step, planning, tool-calling, stopping condition, TaskStep, TaskStepResult, NextStepDecision, task_id, kind
4.2.2 Implementing TaskStep, TaskStepResult, and NextStepDecision
This section describes how task steps are associated with tasks and how next-step decisions are represented as structured output.
- Task step association: Explain how a TaskStep is associated with its overarching Task.
Why it matters: The association makes clear which task a step belongs to. - Next step decision representation: Describe how NextStepDecision is represented and where it is used.
Why it matters: This identifies the structured representation used inside the next-step method.
Key terms: TaskStep, TaskStepResult, task_id, Task, NextStepDecision, pydantic.BaseModel, structured output, get_next_step(), TaskHandler
4.2.3 Implementing LLMAgent: Part 2
The section implements LLMAgent task execution through a TaskHandler, structured next-step decisions, single-turn sub-step chats, and an asynchronous processing loop.
- Task execution processing loop: Trace how run() and TaskHandler alternate between deciding and executing sub-steps until task execution stops.
Why it matters: This loop is the structure that turns a submitted task into a sequence of agent actions and a result. - Structured next step decisions: Explain how get_next_step() uses the backbone LLM to choose another sub-step or a final result.
Why it matters: The structured decision connects the task history to either continued execution or completion. - Single turn substep chat: Describe how run_step() starts a sub-step chat and turns the backbone LLM's response into a TaskStepResult.
Why it matters: The role and context of this chat explain how the agent carries out an individual instruction. - Substep tool calling phase: Explain how tool calls are handled within a sub-step and what happens if the LLM requests a second round.
Why it matters: This limit defines what a single sub-step can accomplish and how further requests are carried forward. - Rollout history and formatting: Describe how TaskHandler builds the rollout and why its formatting matters for later sub-steps.
Why it matters: The rollout preserves useful context across otherwise separate sub-step chats. - Awaitable background task management: Explain how run() supports background execution and awaiting results, and describe the background task access safeguards.
Why it matters: This design lets callers manage multiple running tasks without blocking on each one.
Key terms: run(), TaskHandler, get_next_step(), run_step(), TaskStepResult, TaskResult, NextStepDecision, structured output, TaskStep, backbone LLM
4.3 Putting it all together: Hailstone LLM agent
This section constructs and runs a Hailstone LLMAgent, retrieves its result, and inspects its execution rollout.
- Obfuscated Hailstone Tool: Explain why the example obscures the Hailstone tool’s name and description.
Why it matters: The example is designed to demonstrate tool use rather than let the model answer from memory. - Agent and Task Setup: Describe how the example sets up the LLMAgent and its Hailstone task.
Why it matters: This setup connects the model, tool, and instructions needed to run the example. - Asynchronous Task Result Retrieval: Describe how to wait for the Hailstone task and retrieve its final result, including what happens if result() is called too early.
Why it matters: Waiting correctly lets you retrieve the result without an invalid-state error. - Hailstone Execution Rollout: Trace what the example’s rollout shows for the task starting at 4.
Why it matters: The rollout lets you check how the agent reached its answer, not just the answer itself. - Console Execution Logging: Describe how console logging can help with LLMAgent execution.
Why it matters: Console logging gives you another way to observe execution while debugging.
Key terms: parametric knowledge, Hailstone tool, tool-calling, make_llm(), backbone LLM, LLMAgent, Task, instruction string, TaskHandler, asyncio.Future
4.4 Extending to MAS: LLMAgent
The section explains how LLMAgent instances can participate in multi-agent systems as coordinated subagents or remotely accessible agents.
- Subagent composition and delegation: Explain how a coordinator and subagents built with LLMAgent can share work in a multi-agent system, and how subagents can be configured for different responsibilities.
Why it matters: This shows how the agent built in this chapter can become a component of a larger system. - Remote agent participation: Describe how an LLMAgent can participate in an external multi-agent system as a remote agent.
Why it matters: This identifies a way for agents outside the framework to use an LLMAgent remotely.
Key terms: subagents, coordinator, delegation pattern, backbone LLM, prompt templates, processing loop, remote A2A agent, A2A protocol, server
4.5 Summary
Where this is used later
- Implementing LLMAgentBuilder (chapter 5) · LLMAgent construction and configuration
- (chapter intro) (chapter 9) · LLMAgent provides the framework’s agent class.
- Implementing Skill (chapter 6) · The earlier use of templates for LLMAgent.
- Updating the LLMAgent and TaskHandler classes (chapter 7) · TaskHandler and task-step execution
- Wrapping LLMAgent in an A2A AgentExecutor (chapter 10) · The LLMAgent processing loop.
- Pattern 1: Tool-Based HITL (chapter 8) · The agent's processing loop and its use of tools.
- Pattern 3: Supervised-Execution HITL (chapter 8) · The processing loop and its next-step and step-execution operations.
- Context management (chapter 9) · The agent’s task-execution loop and its use of prior steps
Concepts explained in this chapter
- LLMAgentLLMAgent is the framework class that implements an LLM agent and its task-execution orchestration.
- TaskResultTaskResult is a framework data structure containing a task identifier and the result of executing that task.
- TaskHandlerTaskHandler is a framework helper class that orchestrates task execution across the processing loop and resolves to a TaskResult.
- TaskStepTaskStep is a book-specific data structure representing one execution step associated with an overarching task.
- run()run() is an LLMAgent method that accepts a user task, initiates its processing loop, and returns an awaitable that resolves to a TaskResult.
- LLM-powered agentAn LLM agent is a software system that orchestrates a large language model and external capabilities to turn the model's intentions into actions.
- TaskStepResultTaskStepResult is a framework data structure containing the result of executing one agent sub-step.
- trajectoryA trajectory, or rollout, is a record of an agent's task execution, including its actions and their results.
- processing loopA processing loop is a repeated execution structure in which an LLM agent plans, invokes tools, evaluates results, and continues through successive steps until a stopping condition is met.
- NextStepDecisionNextStepDecision is a framework data structure whose kind and content represent either the instruction for the next sub-step or the overall task result.
- get_next_step()get_next_step() is an asynchronous TaskHandler method that determines the next TaskStep or returns the final TaskResult.
- run_step()run_step() is an asynchronous TaskHandler method that executes one TaskStep and returns its TaskStepResult.
- taskA task is a framework data structure containing the identifier and instruction for a user-submitted task.
- templateA template is a modifiable text structure containing placeholders that can be filled with values before an LLM interaction.
- tool registryA tool registry is a collection that organizes tools for lookup and dispatch.
- LLMAgentTemplatesLLMAgentTemplates is a framework type that defines the prompt templates used for standardized interactions with an agent's backbone LLM.
- tool useToolCall is a data structure representing a requested tool invocation and its arguments.
- ChatMessageChatMessage is a structured representation of a message in an LLM conversation, including its role and content.
- concurrencyConcurrency is the coordination of multiple tasks whose execution can overlap in time.
- stopping criterionA stopping condition is a rule that determines when iterative task execution must terminate.
- structured_output()structured_output() is a BaseLLM method that prompts an LLM to return a response conforming to a specified structure or custom type.
- A2A agentAn A2A agent is an agent accessible to other agents through the Agent2Agent protocol without exposing its internal processes.
- subagentA subagent is an agent dispatched by another agent to work on a subtask contributing to an overall task.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
