BaseLLM is an abstract, async-first interface for an agent’s backbone language model. It gives integrations with different providers and inference frameworks a common API, rather than exposing each provider’s inconsistencies to framework users. section 3.1 section 3.1.2
BaseLLM declares four asynchronous methods, all implemented by subclasses:
- complete(prompt) takes a prompt string and returns a CompleteResult containing the generated response. section 3.1 section 3.1.2
- chat(input, chat_messages, tools) supports conversational interactions and returns a pair of ChatMessage objects. History and tools are optional. section 3.1 section 3.1.2
- continue_chat_with_tool_results(tool_call_results, chat_history, tools) returns tool results to the LLM. It produces a list of TOOL-role messages and the LLM’s response message. section 3.1.2
- structured_output(prompt, mdl) requests an output matching a Pydantic BaseModel subclass and returns an instance of that model. section 3.1.2
Listing 3.3 Declaring BaseLLM: chat() and complete()
# llm_agents_from_scratch/base/llm.py
from abc import ABC, abstractmethod
from llm_agents_from_scratch.base.tool import AsyncBaseTool, BaseTool
from llm_agents_from_scratch.data_structures import (
ChatMessage,
CompleteResult,
ToolCallResult,
)
from typing import Any, Sequence, TypeAlias
Tool: TypeAlias = BaseTool | AsyncBaseTool #A
class BaseLLM(ABC):
"""Base LLM Class."""
@abstractmethod
async def complete(
self,
prompt: str, #B
**kwargs: Any
) -> CompleteResult: #C
"""Text Complete."""
@abstractmethod
async def chat(
self,
input: str, #D
chat_messages: Sequence[ChatMessage] | None = None, #E
tools: Sequence[Tool] | None = None, #F
**kwargs: Any,
) -> tuple[ChatMessage, ChatMessage]: #G
"""Chat interface."""The declarations show the asynchronous completion and chat signatures, including their framework data types and optional tool inputs. section 3.1.2
In a tool-calling chat, an ASSISTANT-role ChatMessage can carry ToolCall objects. Extract those calls and pass them to the selected tool’s __call__() method. Then submit the resulting ToolCallResult objects through continue_chat_with_tool_results() so the LLM can synthesize a response. section 3.1.2
Implementing an integration
OllamaLLM subclasses BaseLLM to connect the common interface to Ollama. It enables agents to use Ollama-supported open-source models, including models from the Llama and Qwen families. section 3.0 section 3.4
