liveBook · Manning

Structured_output()

As explained in Build a Multi-Agent System (from Scratch)

structured_output() is a BaseLLM method that prompts an LLM to return a response conforming to a specified structure or custom type.

When an agent needs a decision it can act on, a free-form response may not provide the required structure. structured_output() is an asynchronous BaseLLM method that requests a response in a specified format and returns an instance of a custom Pydantic model. section 3.1.2 section 4.2.3

The abstract method accepts a prompt, mdl, and additional keyword arguments. mdl is a type bound to pydantic.BaseModel; the return type matches that model. A BaseLLM subclass supplies the implementation. section 3.1.2

Listing 3.5 Implementing BaseLLM: structured_output()
# llm_agents_from_scratch/base/llm.py
from abc import ABC, abstractmethod
from typing import Any, Sequence, TypeVar
from pydantic import BaseModel
from llm_agents_from_scratch.base.tool import AsyncBaseTool, BaseTool
   from llm_agents_from_scratch.data_structures import (
    ChatMessage,
    CompleteResult,
    ToolCallResult,
)
 
StructuredOutputType = TypeVar("StructuredOutputType", bound=BaseModel) #A
 
class BaseLLM(ABC):
    """Base LLM Class."""
    … #B
 
    @abstractmethod
    async def structured_output(
        self,
        prompt: str,
        mdl: type[StructuredOutputType], #C
        **kwargs: Any,
    ) -> StructuredOutputType: #D
        """Structured output interface for returning ~pydantic.BaseModels.

The interface declares the model-bound type variable and asynchronous method signature. section 3.1.2

An OllamaLLM implementation translates the interface into Ollama’s structured output workflow:

  • Convert the prompt into an Ollama user message. section 3.2.1
  • Generate a JSON Schema from mdl using model_json_schema(), and supply it as the format for Ollama’s chat interface. section 3.2.1
  • Extract the JSON payload from the chat response, then validate it to create the model instance. section 3.2.1
Figure 3.14 Integrating with Ollama’s structured output interface, which is facilitated by their chat interface. A structured_output() invocation of OllamaLLM invokes the ollama.chat() but not before creating the Ollama message data object. The returned result is an ollama.ChatResponse from which we extract the JSON payload corresponding to mdl that we then use to validate and create the final structured output object.

The integration diagram traces the prompt and schema through Ollama chat to a validated model instance. section 3.2.1

Request a structured joke

Define Joke as a BaseModel with a subject limited to math, physics, or biology, and a joke string. Then call `joke = await llm.structured_output(prompt="Tell me a joke.", mdl=Joke)`. The result is a Joke instance with those fields. section 3.1.2 section 3.2.1

In a task workflow, TaskHandler.get_next_step() uses the history of completed sub-steps to prompt for a NextStepDecision. The decision determines whether to return another TaskStep or a final TaskResult. For the first step, the handler instead constructs a TaskStep directly from the task instruction. section 4.2.3

How Build a Multi-Agent System (from Scratch) covers it

How deep it goes
Defines the abstract method and traces an Ollama implementation from schema generation through response validation.
Examples
Demonstrates a typed joke response and an agent decision between another task step and a final result.
What it assumes
Assumes familiarity with JSON Schema and Pydantic models when implementing the Ollama integration.
What you can do after
Apply it via OllamaLLM to request and receive an instance of a specified Pydantic model.
How it connects
Connects structured output to Pydantic models, JSON Schema, LLM chat, and agent task planning.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a Multi-Agent System (from Scratch). The section links point to the parts of the book it draws on.