liveBook · Manning

Chapter 3: Working with LLMs

What you will learn in chapter 3 of Build a Multi-Agent System (from Scratch)

This introduction previews adding an LLM abstraction, an Ollama integration, and a complete tool-calling demonstration to the framework.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction previews adding an LLM abstraction, an Ollama integration, and a complete tool-calling demonstration to the framework.

3.1 BaseLLM: a blueprint for LLMs

This section outlines the async BaseLLM interface, its interaction modes, and the data structures that standardize LLM access.

Key terms: BaseLLM, abstract class, async-first, complete(), CompleteResult, prompt, chat(), ChatMessage, ChatRole, chat history

3.1.1 Implementing CompleteResult, ChatMessage, and ChatRole

This section introduces the data structures for completion results and chat messages, including message roles and tool-related messages.

Key terms: CompleteResult, prompt, response, ChatMessage, ChatRole, SYSTEM, USER, ASSISTANT, TOOL, ToolCall

3.1.2 Defining BaseLLM

This section defines the BaseLLM interface and explains how it supports tool-result continuation and structured LLM output.

Key terms: BaseLLM, complete(), chat(), continue_chat_with_tool_results(), structured_output(), ChatMessage, ASSISTANT, ToolCall, ToolCallResult, TOOL

3.2 OllamaLLM: a subclass of BaseLLM

This section introduces OllamaLLM as a BaseLLM subclass that connects the framework to supported models through an Ollama service.

Key terms: OllamaLLM, BaseLLM, Ollama service, model, _client, AsyncClient, __init__()

3.2.1 Implementing OllamaLLM

This section implements OllamaLLM configuration and its four BaseLLM interaction methods using Ollama’s client interfaces and data types.

Key terms: model, host, think, _client, OLLAMA_MODEL, OLLAMA_API_KEY, ollama_settings(), ollama pull, complete(), ollama.AsyncClient

3.2.2 Hailstone tool call with OllamaLLM

This section demonstrates a complete Hailstone tool-calling process with OllamaLLM, from requesting a tool call to returning its result for an LLM response.

Key terms: OllamaLLM, hailstone_tool, chat(), ToolCallResult, continue_chat_with_tool_results(), ChatMessage

3.3 Extending to MAS: LLMs

This section explains how backbone LLM choices affect multi-agent systems and how the existing LLM integrations can be reused across agents.

Key terms: backbone LLM, coordinator LLM agent, subordinate LLM agents, LLM agent, MAS, OpenAILLM, AnthropicLLM, BaseLLM, OllamaLLM

3.4 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 2: Working with Tools Chapter 4: The LLMAgent Class →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.