A chat API lets an application exchange conversational messages with a language model. Role-based messages—system, user, and assistant—separate behavior instructions from user requests and replies. Clear instructions help keep responses useful and safe. section 1.1.2
For a Q&A chatbot, send the question with relevant document context, then let the user ask a follow-up to refine the answer. Retaining earlier turns helps the chatbot respond coherently, but conversation history consumes context-window space and may need summarizing. section 6.1.1 section 1.1.2
A chat API does not itself supply access to a company's documents. For a larger knowledge base, retrieve relevant content rather than putting everything into the prompt. The LangChain setup below sends retrieved context and a question to ChatOpenAI; it is an initial RAG chain, before message history is added. section 6.1.2 section 7.4
Listing 7.3 Initial RAG setup before incorporating message history
from langchain_core.runnables import RunnablePassthrough
from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
rag_prompt_template = """Use the following pieces of context
to answer the question at the end.
If you don't know the answer, just say that you don't know,
don't try to make up an answer.
Use three sentences maximum and keep the
answer as concise as possible.
{context}
Question: {question}
Helpful Answer:"""
rag_prompt = PromptTemplate.from_template(rag_prompt_template)
retriever = vector_db.as_retriever()
question_feeder = RunnablePassthrough()
chatbot = ChatOpenAI(openai_api_key=OPENAI_API_KEY,
model_name="gpt-5-nano")
rag_chain = {"context": retriever,
"question": question_feeder}|rag_prompt|chatbot
def execute_chain(chain, question):
answer = rag_chain.invoke(question)
return answer
The listing shows where a ChatOpenAI model fits after retrieval and prompt construction, and why this starting chain does not yet remember earlier messages. section 7.4
