A chatbot may need to draw on an LLM without sending users to a separate chat interface. LangChain is a library for connecting an LLM to an application's prompts and conversation logic. Unlike an intent-based chatbot framework such as Rasa, it abstracts the model API so developers can experiment with different models and approaches. section 12.5.1 section 12.5.2
An LLM can adapt its replies to varied questions, but fluent replies are not necessarily safe or correct. A math tutor that speaks with children, for example, must distinguish students from parents and seek parental permission before continuing. section 12.5.2
Start with a prompted model
Configure a hosted model through LangChain, then put the user's message into a prompt template and pass both to an LLMChain: from langchain.prompts import PromptTemplate from langchain.chains import LLMChain prompt = PromptTemplate( input_variables=["message"], template="Human says: {message}\nChatbot responds:" ) chain = LLMChain(llm=llm, prompt=prompt) reply = chain.predict(message="Hi Bot! My name is Maria.") Here, llm must already be configured. The example configuration uses a Replicate API token and a hosted Llama 2 chat model. This first chain accepts a message but does not, by itself, give the bot the preceding conversation. section 12.5.2
>>> from langchain.llms import Replicate
>>> os.environ["REPLICATE_API_TOKEN"] = '<your_API_key_here>'
>>> llm = Replicate(
... model="a16z-infra/llama13b-v2-chat:" +
... "df7690", #1
... input={
... "temperature": 0.5,
... "max_length": 100,
... "top_p": 1,
... })
Inspect the model configuration, including the API token, hosted model identifier, and generation settings. section 12.5.2
For a multi-turn bot, add conversation history to the prompt and use ConversationBufferMemory with a ConversationChain. A tutor prompt can specify the onboarding steps: ask whether the user is a parent or student, request a parent's explicit permission, and keep the next reply short. Memory supplies prior turns; the prompt describes the desired behavior. section 12.5.2
>>> class MathConversation(): ... def __init__(self, llm, prompt_string): ... self.llm = llm ... self.memory = \ ... ConversationBufferMemory( ... memory_key='history', ... ai_prefix='tutor', ... human_prefix="user") ... self.convo_chain = ConversationChain( ... llm=llm, memory=self.memory) ... self.convo_chain.prompt = PromptTemplate( ... input_variables=["history", "input"], ... template=prompt_string) ... ... def answer(self, user_input): ... return self.convo_chain.predict(input=user_input)
Inspect a small Python wrapper that joins a conversation chain, buffer memory, and a prompt with matching history and input variables. section 12.5.2
Test the replies rather than treating prompt instructions as guarantees. In a number-sequence quiz, the tutor praises an answer of 12 as correct while also saying that 10 comes next. That contradiction makes answer-checking especially important in a teaching application. section 12.5.2
>>> math_convo.answer("12")
"Great job! Your answer is correct. The next number
in the sequence would be 10.\n\nLet's move on to the
next question. Can you guess the next number in the
sequence: 38, 39, 40, 41?"
See the contradictory quiz response that exposes the limit of relying on fluent model output for math feedback. section 12.5.2
