Haystack is an open-source Python framework for indexing and searching documents, then connecting semantic retrieval with language models. Its main practical role is building retrieval-augmented generation (RAG) and question-answering pipelines that ground responses in retrieved text. section 10.3.6 section 10.3.7
A Haystack RAG workflow typically combines:
- A retriever that selects relevant documents for a query. section 10.3.11
- A PromptNode that instructs a generative model how to synthesize an answer from the retrieved text. section 10.3.11
- A Pipeline that connects the query, retriever, and prompt node while exposing settings such as top_k. section 10.3.11
- A grounding instruction that returns “I don't know” when the supplied sources cannot answer the question. section 10.3.11
A grounded question-answering pipeline
A PromptNode using google/flan-t5-base can be connected to an existing retriever. Asking “How are LSTM neural networks different from RNNs?” produces an answer from retrieved material. Asking “Where is the Louvre Museum?” returns “I don't know” when the retrieved sources do not support the answer, while a broader question about artificial intelligence produces a qualified response. section 10.3.11
Listing 10.23 Creating a Haystack PromptNode for RAG
>>> from haystack.nodes import PromptNode, PromptTemplate, AnswerParser
rag_prompt = PromptTemplate(
prompt="""Synthesize a comprehensive answer from the
following text for the given question.
Provide a clear response that summarizes the
key points and information presented in the text.
If you can't answer the question based on
provided sources, say 'I don't know'.
\n\n Related text: {join(documents)}
\n\n Question: {query}
\n\n Answer:""",
)
>>> prompt_node = PromptNode(model_name_or_path="google/flan-t5-base", default_prompt_template=rag_prompt)
Inspect the runnable PromptNode definition, including its synthesis instructions, document placeholder, question placeholder, and fallback response. section 10.3.11
