liveBook · Manning

OllamaLLM

As explained in Build a Multi-Agent System (from Scratch)

OllamaLLM is an LLM interface implementation that connects an agent framework to locally running models supported by Ollama.

OllamaLLM connects an agent framework to open-source models supported by Ollama. It subclasses BaseLLM and communicates with an Ollama service, often running locally, through the Python library’s AsyncClient. section 3.2

Initialization takes a model name, an optional service host, and a think flag for models that support thinking mode. If host is omitted, the client uses Ollama’s default service address. For a local model, pull it before running the examples, for example with ollama pull qwen3:14b. section 3.2.1

Listing 3.6 Implementing OllamaLLM: __init__()
   # llm_agents_from_scratch/llms/ollama/llm.py
from typing import Any
from ollama import AsyncClient #A
from llm_agents_from_scratch.base.llm import BaseLLM
 
 
class OllamaLLM(BaseLLM):
    """Ollama LLM class."""
 
    def __init__(
        self,
        model: str, #B
        host: str | None = None,
        *,
        think: bool = False,
        **kwargs: Any,
    ) -> None:
        """Create an OllamaLLM instance.""" #C
        super().__init__(**kwargs)
        self.model = model
        self.think = think
        self._client = AsyncClient(host=host) #D

The constructor stores the model and thinking setting and creates the asynchronous Ollama client. section 3.2.1

The interface converts between framework and Ollama data types for four workflows:

  • complete() sends a prompt to the client’s generate() method and converts the response to a CompleteResult. section 3.2.1
  • structured_output() sends a prompt through the chat interface, extracts the JSON payload, and validates it against the requested model type. section 3.2.1
  • chat() converts the input, history, and tools to Ollama types, calls the chat interface, and returns the user and assistant ChatMessage objects. section 3.2.1
  • continue_chat_with_tool_results() converts tool results and history to messages, calls the chat interface, and returns the tool messages and assistant response. section 3.2.1

Complete a tool call

Wrap a hailstone step function in SimpleFunctionTool and pass it to chat() with a request for the next step from 3. The response requests a tool call with x=3. Invoke the tool with that call, then pass its ToolCallResult and the user and assistant messages to continue_chat_with_tool_results(). The resulting assistant response is 10. section 3.2.2

How Build a Multi-Agent System (from Scratch) covers it

How deep it goes
Implements the constructor and asynchronous interaction methods, including conversions between framework and Ollama data types.
Examples
Uses joke completion and structured output alongside a hailstone tool-call cycle that returns 10.
What it assumes
Assumes an existing BaseLLM interface and a tool implementation for the complete tool-calling example.
What you can do after
Build OllamaLLM from scratch using the Ollama Python library and complete a tool-calling workflow.
How it connects
Connects LLM interfaces with text completion, chat, structured output, and tool calling.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a Multi-Agent System (from Scratch). The section links point to the parts of the book it draws on.