ToolFormer trains a language model to decide when to call external tools by emitting tagged API calls such as <API></API>. At inference, an interpreter executes the call and resumes generation, letting the model delegate tasks such as current-information lookup or arithmetic. section 7.4.1
Tool use extends an LLM beyond tasks it should perform unaided:
- A calculator handles basic arithmetic instead of consuming model computation. section 7.4.1
- Search, question answering, translation, calendars, weather, and internet access provide external information or capabilities. section 7.4.1
- The original approach fine-tuned GPT-J for five tools, but adding every new tool through fine-tuning is operationally inconvenient. section 7.4.1
Calling math tools with Guidance
A Guidance program defines add, subtract, multiply, and divide functions, provides worked examples such as “1 + 2 = add(1, 2) = 3,” and generates further tool calls. Guidance stops generation when it recognizes a call, runs the function, and resumes generation. section 7.4.1
Listing 7.4 Giving tools to our LLM models with Guidance
import guidance
from guidance import models, gen
falcon = models.Transformers("tiiuae/falcon-rw-1b") #1
@guidance
def add(lm, input1, input2):
lm += f" = {int(input1) + int(input2)}"
return lm
@guidance
def subtract(lm, input1, input2):
lm += f" = {int(input1) - int(input2)}"
return lm
@guidance
def multiply(lm, input1, input2):
lm += f" = {float(input1) * float(input2)}"
return lm
@guidance
def divide(lm, input1, input2):
lm += f" = {float(input1) / float(input2)}"
return lm
lm = (
falcon
+ """\
1 + 2 = add(1, 2) = 3
4 - 5 = subtract(4, 5) = -1
5 * 6 = multiply(5, 6) = 30
7 / 8 = divide(7, 8) = 0.875
Generate more examples of add, subtract, multiply, and divide
"""
)
lm += gen(max_tokens=15, tools=[add, subtract, multiply, divide])
print(lm)
The listing shows a compact implementation using a Falcon model, Guidance-decorated Python functions, tool selection, and resumed generation. section 7.4.1
