Retrieval-augmented generation (RAG) supplies an LLM with external knowledge retrieved at query time. The model generates an answer using selected passages in its context alongside its parametric memory. This allows it to address changing, domain-specific, or proprietary information without retraining. section 3.2
A RAG system has two workflows:
- Indexing runs ahead of requests, preparing external data and building a searchable index. section 3.2
- Generation runs on the request path: it searches the index, adds selected material to the prompt, and sends the augmented prompt to the LLM. section 3.2
Minimal keyword-based RAG
A Python implementation indexes documents as sparse TF-IDF vectors. For each question, it uses the same vectorizer, ranks documents by cosine similarity, and inserts the highest-scoring matches into a prompt instructing the model to answer only from that context. The augmented prompt is then sent to an OpenAI model. section 3.10.1
Listing 3.1 Minimal RAG in Python: TF-IDF indexing, similarity-based retrieval, and augmented generation
corpus = [doc["text"] for doc in documents] #1
vectorizer = TfidfVectorizer(stop_words="english")
doc_vectors = vectorizer.fit_transform(corpus) #2
def retrieve(query: str, k: int = 3) -> List[Dict[str, Any]]: #3
query_vec = vectorizer.transform([query])
similarities = cosine_similarity(query_vec, doc_vectors)[0]
top_indices = np.argsort(similarities)[::-1][:k]
results: List[Dict[str, Any]] = []
for idx in top_indices:
results.append({
"id": documents[idx]["id"],
"title": documents[idx]["title"],
"score": float(similarities[idx]),
"text": documents[idx]["text"],
})
return results
def query_model_with_rag(question: str,
k: int = 3,
model: str = "gpt-4o-mini") -> str: #4
retrieved_docs = retrieve(question, k=k)
context = build_context(retrieved_docs)
prompt = (
"You are a helpful assistant ...\n"
"Documentation:\n"
f"{context}\n\n"
f"User question: {question}"
)
# Query model (see full source code in GitHub)
Inspect the TF-IDF indexing and similarity-based retrieval code for the minimal pipeline. section 3.10.1
Grounding means factual claims are supported by material placed in the context; it does not guarantee accuracy or establish that the sources are trustworthy. Reliability also depends on source freshness and authority, chunking, ranking, permission filters, and prompt assembly. section 3.1 section 3.2
Retrieval selects passages dynamically, unlike cache-augmented generation, which preloads reusable context, or direct injection, which loads complete documents into the context window. Agentic RAG extends retrieval with an agent that can reformulate queries, select tools, and iterate over intermediate results. section 3.0 section 3.1
Local RAG stack
Ollama can run the LLM and embedding model locally, Qdrant can store and query embeddings, and Agno can orchestrate the pipeline. One configuration ingests a PDF manual into Qdrant, but fetches that manual from a remote URL at startup, so it is not fully offline as written. section 3.10.3
Listing 3.3 RAG system implemented in Python, using an Ollama model, Qdrant as a vector database, and Agno as an agent framework
from agno.agent import Agent
from agno.models.ollama import Ollama
from agno.knowledge.knowledge import Knowledge
from agno.vectordb.qdrant import Qdrant
from agno.knowledge.embedder.ollama import OllamaEmbedder
vector_db = Qdrant( #1
collection="my-vector-db",
url="http://localhost:6333",
embedder=OllamaEmbedder(id="nomic-embed-text"),
)
knowledge_base = Knowledge(vector_db=vector_db) #2
agent = Agent( #3
name="Local RAG",
model=Ollama(id="llama3.2:1b"),
knowledge=knowledge_base,
)
Inspect the configuration connecting the local model, embedder, vector database, and agent. section 3.10.3













































