A language model cannot know an event that happened after its training date just because the prompt asks about it. A tool gives the model a way to use an external capability, such as internet search, a calculator, a clock, or a weather service. Retrieval-augmented generation can also supply fresh context, but then someone must keep its information current. section 7.4 section 7.4.1
A tool call is not merely a model describing what it would do. In the Toolformer approach, a model learns to produce tagged API calls; an interpreter runs them during generation. Guidance offers a prompt-based alternative: it stops when it recognizes a tool call, runs the tool, and resumes generation. This avoids fine-tuning the model whenever a new tool is added. section 7.4.1
Listing 7.4 Giving tools to our LLM models with Guidance
import guidance
from guidance import models, gen
falcon = models.Transformers("tiiuae/falcon-rw-1b") #1
@guidance
def add(lm, input1, input2):
lm += f" = {int(input1) + int(input2)}"
return lm
@guidance
def subtract(lm, input1, input2):
lm += f" = {int(input1) - int(input2)}"
return lm
@guidance
def multiply(lm, input1, input2):
lm += f" = {float(input1) * float(input2)}"
return lm
@guidance
def divide(lm, input1, input2):
lm += f" = {float(input1) / float(input2)}"
return lm
lm = (
falcon
+ """\
1 + 2 = add(1, 2) = 3
4 - 5 = subtract(4, 5) = -1
5 * 6 = multiply(5, 6) = 30
7 / 8 = divide(7, 8) = 0.875
Generate more examples of add, subtract, multiply, and divide
"""
)
lm += gen(max_tokens=15, tools=[add, subtract, multiply, divide])
print(lm)
Four small arithmetic functions show how Guidance supplies callable tools during generation rather than asking the model to calculate every answer itself. section 7.4.1
Tools also enable agent behavior: a model can select an action, pass it an input, and use the result in its next step. LangChain examples combine search, math, and Python execution tools with an agent. Search tools handle retrieval and return results; execution tools demand more caution. In one Python-agent trace, an attempted import fails, and the output warns that a Python REPL can execute arbitrary code. section 8.3
Listing 8.9 Agents and agent behavior
from langchain.llms import LlamaCpp
from langchain.chains.conversation.memory import (
ConversationBufferWindowMemory,
)
from langchain.agents import load_tools, initialize_agent, Tool
from langchain_experimental.tools import PythonREPLTool
from langchain.tools import DuckDuckGoSearchRun, YouTubeSearchTool
llm = LlamaCpp(
model_path="./models/mistral-7b-instruct-v0.1.Q4_0.gguf",
n_gpu_layers=0, #1
n_batch=512,
n_ctx=32768, #2
verbose=False,
)
search = DuckDuckGoSearchRun() #3
duckduckgo_tool = Tool(
name="DuckDuckGo Search",
func=search.run,
description="Useful for when an internet search is needed",
)
youtube_tool = YouTubeSearchTool()
coding_tool = PythonREPLTool()
tools = load_tools(["llm-math"], llm=llm)
tools += [duckduckgo_tool, youtube_tool, coding_tool]
memory = ConversationBufferWindowMemory( #4
memory_key="chat_history",
k=5,
return_messages=True,
output_key="output",
)
agent = initialize_agent( #5
tools=tools,
llm=llm,
agent="chat-conversational-react-description",
verbose=True,
memory=memory,
handle_parsing_errors=True,
)
B_INST, E_INST = "[INST]", "[/INST]" #6
B_SYS, E_SYS = "<<SYS>>\n", "\n<</SYS>>\n\n"
sys_msg = ( #7
"<s>"
+ B_SYS
+ """Assistant is a expert JSON builder designed to assist with a wide \
range of tasks.
Assistant is able to respond to the User and use tools using JSON strings \
that contain "action" and "action_input" parameters.
All of Assistant's communication is performed using this JSON format.
Assistant can also use tools by responding to the user with tool use \
instructions in the same "action" and "action_input" JSON format. Tools \
available to Assistant are:
- "Calculator": Useful for when you need to answer questions about math.
- To use the calculator tool, Assistant should write like so:
```json
{{"action": "Calculator",
"action_input": "sqrt(4)"}}
```
- "DuckDuckGo Search": Useful for when an internet search is needed.
- To use the duckduckgo search tool, Assistant should write like so:
```json
{{"action": "DuckDuckGo Search",
"action_input": "When was the Jonas Brothers' first concert"}}
Here are some previous conversations between the Assistant and User:
User: Hey how are you today?
Assistant: ```json
{{"action": "Final Answer",
"action_input": "I'm good thanks, how are you?"}}
```
User: I'm great, what is the square root of 4?
Assistant: ```json
{{"action": "Calculator",
"action_input": "sqrt(4)"}}
```
User: 2.0
Assistant: ```json
{{"action": "Final Answer",
"action_input": "It looks like the answer is 2!"}}
```
User: Thanks, when was the Jonas Brothers' first concert?
Assistant: ```json
{{"action": "DuckDuckGo Search",
"action_input": "When was the Jonas Brothers' first concert"}}
```
User: 12.0
Assistant: ```json
{{"action": "Final Answer",
"action_input": "They had their first concert in 2005!"}}
```
User: Thanks could you tell me what 4 to the power of 2 is?
Assistant: ```json
{{"action": "Calculator",
"action_input": "4**2"}}
```
User: 16.0
Assistant: ```json
{{"action": "Final Answer",
"action_input": "It looks like the answer is 16!"}}
```
Here is the latest conversation between Assistant and User."""
+ E_SYS
)
new_prompt = agent.agent.create_prompt(system_message=sys_msg, tools=tools) #8
agent.agent.llm_chain.prompt = new_prompt
instruction = ( #9
B_INST
+ " Respond to the following in JSON with 'action' and 'action_input' "
"values "
+ E_INST
)
human_msg = instruction + "\nUser: {input}"
agent.agent.llm_chain.prompt.messages[2].prompt.template = human_msg
agent.run( #10
"Tell me how old Justin Beiber was when the Patriots last won the "
"Superbowl."
)
The agent setup shows how separately defined search, math, and coding tools can be made available together. section 8.3
Use a tool when the answer needs current information, dependable calculation, retrieval, or an action the model cannot perform by generating text alone. Supplying a tool does not guarantee a sound result: the agent still has to choose an appropriate action and handle what comes back. section 7.4 section 8.3
