A chatbot can answer a question about your codebase. An AI agent can take the next steps: edit code, submit a pull request, or review one. The distinction is the surrounding application, not a fundamentally different language model. An LLM agent uses memory and tools to work through a multistep task rather than stop at a response. section 8.3
Three components make that possible:
- An LLM generates the next response or action. section 8.3
- Memory reintroduces relevant earlier steps. Passing the entire history is simple but quickly meets context limits; summarization adds latency and loses detail; structured storage aims to retrieve more useful information. section 8.3
- Tools let the agent do things the model cannot do alone, such as search the web or execute code. That extra reach also creates risk. section 8.3 section 6.3.6
From question to action
A search-and-calculation agent can look up information before answering a multistep question. A coding agent goes further by editing a project. In a LangChain Python-agent example, an attempted import fails during a neural-network task: tool access makes an attempt executable, not automatically correct. section 8.3
Listing 8.9 Agents and agent behavior
from langchain.llms import LlamaCpp
from langchain.chains.conversation.memory import (
ConversationBufferWindowMemory,
)
from langchain.agents import load_tools, initialize_agent, Tool
from langchain_experimental.tools import PythonREPLTool
from langchain.tools import DuckDuckGoSearchRun, YouTubeSearchTool
llm = LlamaCpp(
model_path="./models/mistral-7b-instruct-v0.1.Q4_0.gguf",
n_gpu_layers=0, #1
n_batch=512,
n_ctx=32768, #2
verbose=False,
)
search = DuckDuckGoSearchRun() #3
duckduckgo_tool = Tool(
name="DuckDuckGo Search",
func=search.run,
description="Useful for when an internet search is needed",
)
youtube_tool = YouTubeSearchTool()
coding_tool = PythonREPLTool()
tools = load_tools(["llm-math"], llm=llm)
tools += [duckduckgo_tool, youtube_tool, coding_tool]
memory = ConversationBufferWindowMemory( #4
memory_key="chat_history",
k=5,
return_messages=True,
output_key="output",
)
agent = initialize_agent( #5
tools=tools,
llm=llm,
agent="chat-conversational-react-description",
verbose=True,
memory=memory,
handle_parsing_errors=True,
)
B_INST, E_INST = "[INST]", "[/INST]" #6
B_SYS, E_SYS = "<<SYS>>\n", "\n<</SYS>>\n\n"
sys_msg = ( #7
"<s>"
+ B_SYS
+ """Assistant is a expert JSON builder designed to assist with a wide \
range of tasks.
Assistant is able to respond to the User and use tools using JSON strings \
that contain "action" and "action_input" parameters.
All of Assistant's communication is performed using this JSON format.
Assistant can also use tools by responding to the user with tool use \
instructions in the same "action" and "action_input" JSON format. Tools \
available to Assistant are:
- "Calculator": Useful for when you need to answer questions about math.
- To use the calculator tool, Assistant should write like so:
```json
{{"action": "Calculator",
"action_input": "sqrt(4)"}}
```
- "DuckDuckGo Search": Useful for when an internet search is needed.
- To use the duckduckgo search tool, Assistant should write like so:
```json
{{"action": "DuckDuckGo Search",
"action_input": "When was the Jonas Brothers' first concert"}}
Here are some previous conversations between the Assistant and User:
User: Hey how are you today?
Assistant: ```json
{{"action": "Final Answer",
"action_input": "I'm good thanks, how are you?"}}
```
User: I'm great, what is the square root of 4?
Assistant: ```json
{{"action": "Calculator",
"action_input": "sqrt(4)"}}
```
User: 2.0
Assistant: ```json
{{"action": "Final Answer",
"action_input": "It looks like the answer is 2!"}}
```
User: Thanks, when was the Jonas Brothers' first concert?
Assistant: ```json
{{"action": "DuckDuckGo Search",
"action_input": "When was the Jonas Brothers' first concert"}}
```
User: 12.0
Assistant: ```json
{{"action": "Final Answer",
"action_input": "They had their first concert in 2005!"}}
```
User: Thanks could you tell me what 4 to the power of 2 is?
Assistant: ```json
{{"action": "Calculator",
"action_input": "4**2"}}
```
User: 16.0
Assistant: ```json
{{"action": "Final Answer",
"action_input": "It looks like the answer is 16!"}}
```
Here is the latest conversation between Assistant and User."""
+ E_SYS
)
new_prompt = agent.agent.create_prompt(system_message=sys_msg, tools=tools) #8
agent.agent.llm_chain.prompt = new_prompt
instruction = ( #9
B_INST
+ " Respond to the following in JSON with 'action' and 'action_input' "
"values "
+ E_INST
)
human_msg = instruction + "\nUser: {input}"
agent.agent.llm_chain.prompt.messages[2].prompt.template = human_msg
agent.run( #10
"Tell me how old Justin Beiber was when the Patriots last won the "
"Superbowl."
)
The LangChain listing wires a local LLM to search, calculation, and Python tools alongside conversation memory, making the application surrounding the model visible. section 8.3
Give an agent only the execution access its job requires, and isolate it accordingly. An email-sending agent, for example, could encounter an attacker’s prompt injection in an incoming message and send malicious content to contacts. Authentication, logging, and other ordinary application safeguards still apply. For lower-stakes work, generating email replies for a person to choose is a narrower step than sending them autonomously. section 6.3.6 section 12.2.7
Start with a bounded task, not a promise to replace a profession. Agents can be difficult to make reliable because LLM behavior is unpredictable; specialized tasks such as checking extracted résumé information offer a more concrete target. section 8.4 section 12.2.7
