liveBook · Manning

Ollama.chat()

As explained in Build a Multi-Agent System (from Scratch)

ollama.chat() is an Ollama client method that sends messages and optional tools to a model and returns its chat response.

Ollama’s chat interface supports both message-based conversations and structured output. Framework integrations call it after translating their own messages and tools into Ollama’s data types, then convert the response back. section 3.2.1

An integration prepares and handles a chat request by:

  • Converting chat history and the current user input into Ollama messages; converting optional tools into Ollama tools. section 3.2.1
  • Calling ollama.chat() and converting its response into a framework ChatMessage. section 3.2.1
  • For structured output, passing a JSON Schema generated from a Pydantic model and validating the returned JSON against that model. section 3.2.1

Structured joke

A prompt such as “Tell me a joke” can request output matching a Joke model, producing a validated structured object rather than plain text. section 3.2.1

Listing 3.9 Implementing OllamaLLM: chat()
# llm_agents_from_scratch/llms/ollama/llm.py
… #A
from llm_agents_from_scratch.llms.utils import ( #B
    chat_message_to_ollama_message,
    ollama_message_to_chat_message,
    tool_to_ollama_tool,
)
 
class OllamaLLM(BaseLLM):
    """Ollama LLM class."""
    … #C
 
    async def chat(
        self,
        input: str,
        chat_history: list[ChatMessage] | None = None,
        tools: list[BaseTool | AsyncBaseTool] | None = None,
        **kwargs: Any,
    ) -> tuple[ChatMessage, ChatMessage]:
        … #D
        # prepare messages #E
        o_messages = (
            [chat_message_to_ollama_message(cm) for cm in chat_history]
            if chat_history
            else []
        )
        user_message = ChatMessage(role="user", content=input)
        o_messages.append(
            chat_message_to_ollama_message(
                user_message,
            ),
        )
 
        # prepare tools #F
        o_tools = (
            [tool_to_ollama_tool(t) for t in tools]
            if tools else None
        )
 
        result = await self._client.chat( #G
            model=self.model,
            messages=o_messages,
            tools=o_tools,
        )
 
        return (
            user_message,
            ollama_message_to_chat_message(result.message) #H
        )

Inspect the chat() implementation listing to follow the message and tool conversion flow. section 3.2.1

How Build a Multi-Agent System (from Scratch) covers it

How deep it goes
Details message and tool conversion plus response mapping, but omits the conversion utilities’ implementations.
Examples
Uses a joke prompt and Joke model to demonstrate structured output.
What it assumes
Assumes framework message and tool types and, for structured output, a Pydantic model schema.
What you can do after
Apply it via Ollama’s chat interface to request structured output or exchange messages with history and optional tools.
How it connects
Connects chat messages, tools, structured output, JSON Schema, and Pydantic models.

Also explained in

Compare how 2 books explain ollama.chat() →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a Multi-Agent System (from Scratch). The section links point to the parts of the book it draws on.