Chapter 13: Tool use and function calling
The introduction explains why tools expand language-model capabilities, notes that tool use can be trained, and distinguishes tool use from function calling and code execution.
After this chapter you will be able to
- Capabilities gained through tools. Explain how tools let language models address tasks that their weights alone cannot handle, including current-information queries and actions on files.
- Tool terminology distinctions. Distinguish general tool use from function calling and code execution using the definitions given here.
- External tool invocation. Explain that an AI model uses an external tool by outputting special tokens that trigger a tool endpoint, and name the kinds of tools described here.
- Reasons for external tools. Explain how retrieval-like tools and code execution address different limitations of a language model answering from its trained weights alone.
- Tool-use evaluation dimensions. Describe the distinct checks used to evaluate tool use, including whether success is consistent across trials.
- Interleaved tool-call flow. Trace how a model generates a tool call, receives tool output inserted into the sequence, and continues generating a response.
- Function-calling training requirements. Explain what function-calling training data adds and what behavior training seeks from the model.
- Interleaved reasoning and tool loop. Trace how a model reasons about the information it needs, calls a tool, reasons over the result, and may repeat until it can answer confidently.
- Standardized external access. Explain that MCP gives applications a predictable way to pass context—data and actions—to language models, rather than requiring different tool-call formats for each external system.
- MCP server primitives. Distinguish the three core primitives an MCP server offers: resources, prompts, and tools.
- Server client host roles. Describe how MCP servers, clients, and hosts fit together, and why this arrangement helps developers reuse integrations.
- Tool-call data representation. Describe how tool calls can be represented in training data and converted into model tokens, including how calls divide an assistant turn.
- Tool-output loss masking. Explain why tool-output tokens are masked from the model's training loss.
- Tool-use post-training progression. Explain where tool-use training data commonly comes from and what SFT, preference optimization, and RL each contribute.
What it builds on
- Chat templates and the structure of instructions (chapter 4) · Chat templates and the structure of instruction messages
- Implementation details (chapter 4) · Implementation details of post-training
- Understanding reasoning training methods (chapter 7) · Reasoning-model training methods
- (chapter intro) (chapter 4) · Instruction fine-tuning
- Direct preference optimization (chapter 8) · Direct preference optimization
- The role of reinforcement learning in RLHF (chapter 6) · The role of reinforcement learning in post-training
Section by section
Introduction
The introduction explains why tools expand language-model capabilities, notes that tool use can be trained, and distinguishes tool use from function calling and code execution.
- Capabilities gained through tools: Explain how tools let language models address tasks that their weights alone cannot handle, including current-information queries and actions on files.
Why it matters: Tools address limitations that generating text from model weights cannot solve. - Tool use as trained skill: State that language models need to be trained to use tools and that RLHF and other methods discussed in the book can refine this skill.
Why it matters: Providing a tool and training a model to use it are different considerations. - Tool terminology distinctions: Distinguish general tool use from function calling and code execution using the definitions given here.
Why it matters: These terms identify related but different ways a model interacts with external capabilities.
Key terms: tools, knowledge cutoff, model weights, tool use, RLHF, function calling, code execution, orchestrator, schema
13.1 Tool-use overview
This section explains why language models use external tools, sketches how tool use developed, and identifies ways to evaluate it.
- External tool invocation: Explain that an AI model uses an external tool by outputting special tokens that trigger a tool endpoint, and name the kinds of tools described here.
Why it matters: This is the basic distinction between generating an answer alone and invoking an external capability. - Reasons for external tools: Explain how retrieval-like tools and code execution address different limitations of a language model answering from its trained weights alone.
Why it matters: The benefit of a tool depends on which limitation of answering unaided it addresses. - Tool-use development and reach: Describe the section's broad progression from early systems and external-capability integrations to models using many APIs and tool-enabled applications.
Why it matters: Tool use is an established and increasingly broad part of language-model applications, not a single recent feature. - Tool-use evaluation dimensions: Describe the distinct checks used to evaluate tool use, including whether success is consistent across trials.
Why it matters: A correctly formatted tool call alone does not establish that an agent completes tasks reliably.
Key terms: external tools, special tokens, endpoint, up-to-date information, code execution, probabilistic, generative nature, tool use, retrieval-augmented generation, APIs, Model Context Protocol (MCP)
13.2 Interweaving tool calls in generation
The section explains how tool calls and tool outputs are interleaved with model generation, and what training must teach a model to do in that token flow.
- Interleaved tool-call flow: Trace how a model generates a tool call, receives tool output inserted into the sequence, and continues generating a response.
Why it matters: Tool use is not just text describing an action: execution results enter the sequence the model uses to continue. - Function-calling training requirements: Explain what function-calling training data adds and what behavior training seeks from the model.
Why it matters: A useful tool call depends on knowing the available tools and reliably using the results, not just producing a plausible-looking call. - Code execution as tool use: Describe why code execution is a useful form of tool use and how its code and output appear in the sequence.
Why it matters: Code execution provides a concrete example of how a model can use an external result for a more precise answer.
Key terms: tool call, tool outputs, autoregressively generated tokens, orchestration loop, function calling, system prompt, tools, arguments, tool use, code execution
13.3 Multistep tool reasoning
This section explains how multistep tool reasoning alternates reasoning with tool actions to gather information and reach an answer.
- Interleaved reasoning and tool loop: Trace how a model reasons about the information it needs, calls a tool, reasons over the result, and may repeat until it can answer confidently.
Why it matters: The answer can develop through several rounds of reasoning and external information, rather than a single tool call. - Complementary reasoning and actions: Explain how reasoning traces and actions help each other in the interleaved approach illustrated by ReAct.
Why it matters: This explains why combining reasoning with tool actions is more useful than treating either as an isolated step.
Key terms: multistep tool reasoning, interleaved thinking, think-tool-think-tool pattern, ReAct, reasoning traces, task-specific actions
13.4 Model Context Protocol
This section explains how MCP standardizes the connection between language models and external data and tools, and identifies the roles of its main components.
- Standardized external access: Explain that MCP gives applications a predictable way to pass context—data and actions—to language models, rather than requiring different tool-call formats for each external system.
Why it matters: A common format reduces the need for a separate integration format for every external system. - MCP server primitives: Distinguish the three core primitives an MCP server offers: resources, prompts, and tools.
Why it matters: These categories clarify what kind of context or action a server makes available. - Server client host roles: Describe how MCP servers, clients, and hosts fit together, and why this arrangement helps developers reuse integrations.
Why it matters: The separation of roles makes tool-use integrations easier to reuse across models and applications.
Key terms: Model Context Protocol, MCP, JSON schema, JSON-RPC 2.0, resources, prompts, tools, MCP servers, MCP clients, hosts
13.5 Implementation details
This section explains implementation choices for tool-use models and how tool-use behavior is developed through post-training.
- Tool-call data representation: Describe how tool calls can be represented in training data and converted into model tokens, including how calls divide an assistant turn.
Why it matters: The model trains on tokens, so the representation of a tool call affects what it sees and produces. - Tool-output loss masking: Explain why tool-output tokens are masked from the model's training loss.
Why it matters: Masking keeps the training objective focused on tokens the model is responsible for generating. - Tool-use runtime constraints: Describe two practical constraints on tool-use systems: producing valid structured calls and managing the context used by tool results.
Why it matters: A usable tool call needs valid arguments, and its result must leave enough context for the model to continue. - Reasoning continuity across calls: Describe the implementation choice of whether reasoning tokens persist across tool calls and conversation turns.
Why it matters: Whether reasoning tokens persist affects continuity across calls and the cost of serving the model. - Tool-use post-training progression: Explain where tool-use training data commonly comes from and what SFT, preference optimization, and RL each contribute.
Why it matters: The objectives target different parts of tool use, from producing calls to choosing them and completing multistep tasks.
Key terms: JSON, Python code, multiturn formatting, chat templates, token streams, masking tool outputs, training loss, prompt masking, schema conformance, constrained decoding
13.6 Summary
Concepts explained in this chapter
- tool usageTool use is a model capability in which generated interactions invoke external tools or services to perform tasks.
- constrained decodingConstrained decoding is a generation technique that restricts the tokens or structures a model may produce to satisfy specified validity constraints.
- fine-tuningFine-tuning is further training of a pretrained model on a task-specific or behavior-specific objective.
- preference tuningPreference optimization is model training that adjusts behavior toward preferred outputs using preference-based objectives or rewards.
- reinforcement learningReinforcement learning is a machine learning paradigm in which an agent learns behavior by optimizing rewards obtained through interaction with an environment.
- trajectoryAn episode is a finite sequence of interaction steps experienced by an agent during one trial.
- chat templateA chat template is a formatting specification that converts structured conversation messages into the token sequence expected by a language model.
- special tokenA special token is a reserved token that marks structural or control information in a model's input or output sequence.
- thinking tokenA thinking token is a generated language-model token belonging to an intermediate reasoning sequence produced before a final response.
- code executionCode execution is the process of running source code to obtain its computational behavior or result.
- promptA prompt is an input text or instruction provided to a language model to elicit a response.
- chain-of-thought promptingA reasoning trace is a sequence of generated textual intermediate steps used while solving a problem before reaching an answer.
- post-trainingPost-training is the set of optimization and adaptation procedures applied to a pretrained model to improve its usefulness for downstream tasks or alignment objectives.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.