liveBook · Manning

BaseLLM

As explained in Build a Multi-Agent System (from Scratch)

BaseLLM is the example framework's abstract class defining an interface for large language models.

BaseLLM is an abstract, async-first interface for an agent’s backbone language model. It gives integrations with different providers and inference frameworks a common API, rather than exposing each provider’s inconsistencies to framework users. section 3.1 section 3.1.2

BaseLLM declares four asynchronous methods, all implemented by subclasses:

  • complete(prompt) takes a prompt string and returns a CompleteResult containing the generated response. section 3.1 section 3.1.2
  • chat(input, chat_messages, tools) supports conversational interactions and returns a pair of ChatMessage objects. History and tools are optional. section 3.1 section 3.1.2
  • continue_chat_with_tool_results(tool_call_results, chat_history, tools) returns tool results to the LLM. It produces a list of TOOL-role messages and the LLM’s response message. section 3.1.2
  • structured_output(prompt, mdl) requests an output matching a Pydantic BaseModel subclass and returns an instance of that model. section 3.1.2
Listing 3.3 Declaring BaseLLM: chat() and complete()
# llm_agents_from_scratch/base/llm.py
from abc import ABC, abstractmethod
from llm_agents_from_scratch.base.tool import AsyncBaseTool, BaseTool
from llm_agents_from_scratch.data_structures import (
    ChatMessage,
    CompleteResult,
    ToolCallResult,
)
from typing import Any, Sequence, TypeAlias
 
Tool: TypeAlias = BaseTool | AsyncBaseTool #A
 
class BaseLLM(ABC):
    """Base LLM Class."""
 
    @abstractmethod
    async def complete(
        self,
        prompt: str, #B
        **kwargs: Any
    ) -> CompleteResult: #C
        """Text Complete."""
 
    @abstractmethod
    async def chat(
        self,
        input: str, #D
        chat_messages: Sequence[ChatMessage] | None = None, #E
        tools: Sequence[Tool] | None = None, #F
        **kwargs: Any,
    ) -> tuple[ChatMessage, ChatMessage]: #G
        """Chat interface."""

The declarations show the asynchronous completion and chat signatures, including their framework data types and optional tool inputs. section 3.1.2

In a tool-calling chat, an ASSISTANT-role ChatMessage can carry ToolCall objects. Extract those calls and pass them to the selected tool’s __call__() method. Then submit the resulting ToolCallResult objects through continue_chat_with_tool_results() so the LLM can synthesize a response. section 3.1.2

Implementing an integration

OllamaLLM subclasses BaseLLM to connect the common interface to Ollama. It enables agents to use Ollama-supported open-source models, including models from the Llama and Qwen families. section 3.0 section 3.4

How Build a Multi-Agent System (from Scratch) covers it

How deep it goes
Defines four abstract method signatures and traces how chat returns tool results to the LLM.
Examples
Uses an OllamaLLM subclass and a Pydantic joke model to illustrate integration and structured output.
What you can do after
Build an LLM integration by subclassing BaseLLM and implementing its abstract methods.
How it connects
Connects text completion, chat, tool calling, structured output, and asynchronous programming.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a Multi-Agent System (from Scratch). The section links point to the parts of the book it draws on.