Conversation history is the ordered record of user prompts and assistant responses. Adding earlier turns to a new prompt gives an LLM context from the interaction so far. section 8.1.2
History can make an interactive chatbot more useful, but it has practical limits:
- Long conversations can exceed the number of tokens the model can accept. section 8.1.2
- Summarization is shown as a way to compress chat history. section 8.1.2
- Using history does not ensure good responses; the model should be trained on conversation data. section 8.1.2
A Streamlit chatbot
Store role-tagged messages in session state, display previous turns, and use the history when forming the next prompt. section 8.1.2
Listing 8.2 An example application using chat history to improve results
import streamlit as st
import requests
import json
url = "http://localhost:8000/generate" #1
st.title("Chatbot with History")
if "chat_history" not in st.session_state: #2
st.session_state.chat_history = []
for chat in st.session_state.chat_history: #3
with st.chat_message(chat["role"]):
st.markdown(chat["content"])
if user_input := st.chat_input("Your question here"): #4
with st.chat_message("user"): #5
st.markdown(user_input)
st.session_state.chat_history.append( #6
{"role": "user", "content": user_input}
)
with st.chat_message("assistant"): #7
placeholder = st.empty()
full_response = ""
prompt = "You are an assistant who helps the user. " #8
"Answer their questions as accurately as possible. Be concise. "
history = [
f'{ch["role"]}: {ch["content"]}'
for ch in st.session_state.chat_history
]
prompt += " ".join(history)
prompt += " assistant: "
data = json.dumps({"prompt": prompt})
with requests.post(url, data=data, stream=True) as r: #9
for line in r.iter_lines(decode_unicode=True):
full_response += line.decode("utf-8")
placeholder.markdown(full_response + "▌") #10
placeholder.markdown(full_response)
st.session_state.chat_history.append( #11
{"role": "assistant", "content": full_response}
)
See the example application that stores and displays chat history. section 8.1.2
