Chapter 10: Query generation, routing, and retrieval postprocessing

Introduction

The introduction connects earlier RAG accuracy improvements to the chapter’s upcoming work on query generation, routing, and retrieval postprocessing.

• Earlier RAG accuracy improvements: Distinguish how advanced indexing and query transformations improve RAG answer accuracy: indexing makes embeddings more effective for broader chunks with richer context, while query transformations make vector-store retrieval more precise.
  Why it matters: This distinction provides context for why the chapter introduces additional RAG techniques.

Key terms: Retrieval-Augmented Generation (RAG), advanced indexing, query transformations, vector store retrieval

10.1 Content database query generation

The section surveys how LLM applications retrieve information from different content databases and how LLMs can bridge natural-language questions and database-specific queries.

• Natural Language Query Gap: Explain why answering a user question may require more than a vector store and how an LLM can help when another database requires a structured query.
  Why it matters: It explains why query generation is useful when an application draws on sources beyond a vector store.
• Dense And Sparse Retrieval: Distinguish embedding-based dense retrieval from word-based sparse retrieval, including how sparse retrieval finds and ranks matching chunks.
  Why it matters: The two approaches find relevant chunks in different ways, so they suit different kinds of questions.
• Relational Database Querying: Describe what a relational database stores and how text-to-SQL lets an LLM help retrieve its data from a natural-language question.
  Why it matters: Text-to-SQL is one concrete way an application can access structured facts using a user's ordinary question.
• Document Database Queries: Describe the data format used by document, key–value, or object databases and the kind of query an LLM can produce for them.
  Why it matters: It shows that the query an LLM generates must fit the particular database, not just be structured in any format.
• Knowledge Graph Representation: Explain how a knowledge graph represents data and why graph-specific queries can support questions beyond similarity search.
  Why it matters: The graph's explicit relationships make it a different source of answers from a similarity-based vector search.

Key terms: unstructured user questions, structured queries, content stores, dense retrieval, sparse retrieval, lexical retrieval, embeddings, inverted index, BM25, relational (SQL) database

10.2 Self-querying (metadata query enrichment)

This section explains how metadata attached to document chunks lets a vector store combine semantic search with keyword-based filtering, and how self-querying derives those filters from a user’s question.

• Metadata for chunk filtering: Describe how document chunks can acquire metadata or keywords that allow search results to be filtered, including explicit tags, algorithmic keyword extraction, and LLM-suggested keywords.
  Why it matters: The filters used in search depend on information attached to the chunks.
• Question-derived metadata filters: Explain how self-querying turns a user’s question into an enriched query containing both a metadata filter and a semantic component, so semantic search focuses on chunks matching the filter. Distinguish this inferred filter from one explicitly selected in a chatbot UI.
  Why it matters: It lets a question constrain which chunks are searched without requiring the user to set filters separately.

Key terms: document chunks, embedding, dense search, metadata tags, keyword-based indexing, TF-IDF, self-querying, self-metadata querying, metadata filter, semantic component

10.2.1 Ingestion: Metadata enrichment

This section prepares a ChromaDB collection of UK tourist destination chunks with metadata so later queries can filter search results.

• Metadata-enriched ingestion process: Describe how the UK tourist destination content is re-imported into a new ChromaDB collection: define a splitting strategy, then ingest each chunk with its metadata. Explain that the resulting collection supports metadata filtering.
  Why it matters: Adding metadata at ingestion time gives later searches fields they can filter on.
• Destination metadata fields: State what the source, destination, and region metadata fields represent, and describe how their values are prepared for each chunk.
  Why it matters: These fields give the collection concrete attributes for narrowing destination searches.

Key terms: metadata, document chunk, ChromaDB collection, metadata filters, source, destination, region

10.2.2 Q&A on a metadata-enriched collection

This section shows three ways to filter retrieval from a metadata-enriched vector collection: manually, with SelfQueryRetriever, or through a structured LLM-generated query.

• Explicit metadata filtering: Describe how an explicit filter uses metadata attached to chunks to restrict retriever results, and how the filter is changed.
  Why it matters: Explicit filtering gives you direct control when you already know which metadata value to search for.
• SelfQueryRetriever filter inference: Explain how SelfQueryRetriever obtains a metadata filter from a user question and the trade-off of doing so.
  Why it matters: It lets a user's question determine the metadata constraint without manually writing that filter.
• Structured query filter pipeline: Trace how a structured LLM function call turns a question into a content search and a ChromaDB metadata filter, and contrast its setup with SelfQueryRetriever.
  Why it matters: Separating the search text from inferred filters makes the generated query usable for a filtered ChromaDB search.

Key terms: explicit metadata filter, retriever, destination, SelfQueryRetriever, metadata attributes, filter criteria, structured LLM function call, DestinationSearch, content_search, ChromaDB filter

10.3 Generating a structured SQL query

The section introduces LLM-generated SQL and explains how schema-grounded prompts can reduce incorrect table and column references.

• Schema-Grounded SQL Generation: Explain how an LLM can turn a user question into a SQL query for a relational database, and why providing the target schema and sample records in a few-shot prompt can improve accuracy.
  Why it matters: Grounding the prompt in the real database helps prevent queries that refer to tables or columns that do not exist.

Key terms: text-to-SQL, few-shot prompts, schema, sample records, CREATE TABLE, hallucinations

10.3.1 Installing SQLite

This section gives the minimal steps for making SQLite available from the command line.

• SQLite setup without installation: Describe how to make SQLite available without a full installation: unzip the package into a folder and add that folder to the system’s Path environment variable.
  Why it matters: These steps prepare SQLite for the database work that follows.

Key terms: SQLite, Path environment variable

10.3.2 Setting up and connecting to the database

This section sets up the UkBooking SQLite database, explains its relational diagram, and verifies that LangChain can connect to it.

• UkBooking database setup and verification: Trace how to create and populate UkBooking in the SQLite terminal, check its Offer records, then connect from a Jupyter Notebook and verify the connection with a sample query.
  Why it matters: These checks establish that the database contains data and is accessible for programmatic queries.
• UkBooking relational diagram: Describe what the UkBooking relational diagram shows about its destinations, accommodations, and offers tables.
  Why it matters: The diagram gives you a way to see how the booking data is organized before querying it.

Key terms: UkBooking, SQLite terminal, SQL scripts, Offer table, Jupyter Notebook, LangChain, relational diagram, primary key (PK), foreign key (FK), destinations

10.3.3 Generating SQL queries from natural language

This section shows how to generate SQL from a natural-language question and clean its formatting before execution.

• Generated SQL formatting cleanup: Explain why a generated SQL response may need formatting cleanup before database execution and how an LLM chain performs that cleanup.
  Why it matters: Cleaning the generated response prevents formatting characters from breaking the database query.

Key terms: natural language questions, generated SQL query, backticks, LLM, chain

10.3.4 Executing the SQL query

This section explains how a combined chain retrieves relational data by generating and executing SQL, and notes a LangChain alternative for generating queries.

• Combined SQL generation and execution: Explain that the combined chain retrieves data from a relational database by handling both SQL query generation and execution, and that it can be integrated into a broader RAG setup.
  Why it matters: It connects a user-facing retrieval workflow to data held in a relational database.
• SQLDatabaseChain query generation: Describe SQLDatabaseChain as a LangChain option for generating SQL directly from user questions using an LLM and a database connection to create few-shot prompts automatically.
  Why it matters: It offers another way to generate relational-database queries for a RAG setup.

Key terms: sql_query_gen_and_exec_chain, relational database, RAG, SQLDatabaseChain, LLM, database connection, few-shot prompts

10.4 Generating a semantic SQL query

The section contrasts exact-match SQL queries with semantic SQL search, which aims to find results similar in meaning to a user’s intent.

• Exact versus semantic SQL search: Distinguish standard SQL search, which uses exact matches and traditional relational operations, from semantic SQL search, which aims to include results similar in meaning to the user’s intent.
  Why it matters: This distinction explains why generating SQL from natural language does not, by itself, make a search semantic.

Key terms: strict SQL, exact matching, semantic SQL search

10.4.1 Standard SQL query

This section contrasts exact-match SQL filtering with LIKE-based partial matching and shows that both can miss name variations.

• Exact and partial SQL matching: Distinguish an exact SQL name match from a LIKE search for names starting with “Rob,” and explain which name variations each misses.
  Why it matters: These limits explain why matching a name by its exact value or a shared string can leave out relevant variations.

Key terms: standard SQL query, exact matches, LIKE operator, partial matching

10.4.2 Semantic SQL query

Introduces semantic SQL search as embedding-based similarity search in relational databases, contrasting it with exact text matching.

• Semantic SQL search principle: Explain that semantic SQL search uses embeddings and vector similarity to find results by meaning rather than requiring exact text matches.
  Why it matters: This distinction explains what makes a semantic SQL search different from an exact-match search.

Key terms: semantic SQL search, SQL similarity search, embeddings, vector-based similarity searches

10.4.3 Creating the embeddings

This section explains how to prepare PostgreSQL fields for pgvector similarity search by storing embeddings alongside the values being searched.

• Embeddings for searchable columns: Describe how to enable similarity search on a table field: add a VECTOR column, calculate embeddings for the field’s values, and store them in that column. Explain that embeddings can be calculated within PostgreSQL using a supplied function or externally using a client such as LangChain.
  Why it matters: The database needs embeddings for the fields you want pgvector to search by similarity.

Key terms: pgvector, VECTOR, embeddings, similarity search, OpenAIEmbeddings

10.4.4 Performing a semantic SQL search

The section shows that a semantic SQL search can return name variations by ordering results according to similarity.

• Similarity-ranked name search: Explain that, after embeddings are set up, a semantic SQL search can return variations of a name by ordering results based on similarity. Mention that indexing the related column helps performance on big datasets.
  Why it matters: This connects the search results to similarity ranking rather than an exact-name match.

Key terms: embeddings, similarity search, indexing, similarity

10.4.5 Automating semantic SQL search

This section previews using a prompt and an LCEL chain to automate semantic SQL searches and pass results to an LLM.

10.4.6 Benefits of a semantic SQL search

Semantic SQL supports nuanced searches by combining semantic filters with exact SQL matching, including across joined tables.

• Flexible semantic SQL filtering: Explain how semantic SQL can combine semantic filtering with exact matching or multiple semantic filters, and why this is useful for complex queries.
  Why it matters: Combining semantic and conventional SQL filters lets you express searches that neither approach captures as precisely on its own.

Key terms: semantic filtering, exact matching, multiple semantic filters, joins, traditional SQL filtering

10.5 Generating queries for a graph database

The section explains how graph databases represent relationships and how LLMs can help turn natural-language questions into graph queries and answers.

• Graph Entities and Relationships: Explain how graph databases represent entities as nodes and relationships as edges, and why that representation suits knowledge-intensive applications.
  Why it matters: This explains why a knowledge graph may be useful when relationships are central to a question.
• Graph Formats and Query Languages: Distinguish RDF triples and SPARQL from other graph representations and query languages mentioned in the section.
  Why it matters: The database’s format determines what kind of query an LLM needs to generate.
• LLM-Assisted Knowledge Graph RAG: Trace the LLM roles described for building and querying a knowledge graph: extracting graph data from text, generating a query from a question, and turning results into an answer.
  Why it matters: It shows where LLMs help in a graph-backed question-answering workflow without replacing the graph database.
• Knowledge Graph Embedding Complement: Explain how adding textual descriptions and embeddings extends a knowledge graph’s retrieval options.
  Why it matters: It identifies a way to combine structured relationships with semantic retrieval.

Key terms: graph database, nodes, edges, knowledge graph, RDF, triples, subject-predicate-object, SPARQL, Cypher, Gremlin

10.6 Chain routing

Explains why a RAG application may need task-specific branches and how a routing chain selects one for a user question.

• Task-specific RAG chain routing: Explain why a RAG application may have separate chains for different data sources or tasks, and describe how a routing chain chooses the best-suited chain by analyzing the user's question.
  Why it matters: Routing lets an application send each question to a chain suited to its data and task.

Key terms: RAG architecture, routing chain, vector store, UkBooking database

10.6.1 Setting up data retrievers

This subsection sets up retriever chains for an existing vector store and relational database so questions can be sent to the appropriate data source.

• Data source retriever chains: Describe how the section reuses the existing vector store and relational database configurations to create corresponding retriever chains, each associated with its data source.
  Why it matters: The router needs source-specific retriever chains to send questions to.

Key terms: vector store, relational database, retriever chains

10.6.2 Setting up the query router

This section sets up an LLM-based router that chooses a data source from a question’s intent, then uses that choice to select a retriever chain.

• Intent-based data source routing: Explain how the LLM router uses a question’s content and a prompt describing each retriever to choose between tourist information and accommodation bookings. Describe how it returns that choice in a typed object’s datasource attribute.
  Why it matters: Choosing a data source that fits the question helps the system retrieve information relevant to the answer.
• Retriever chooser selection: Describe how the retriever chooser uses the router’s selected data source to select the corresponding retriever chain instance.
  Why it matters: The router’s decision only directs retrieval when it is used to select the matching chain.

Key terms: question router, datasource, tourist_info_store, uk_booking_db, retriever chooser, retriever chain, data source

10.6.3 Integrating the chain router into a full RAG chain

This section shows how a chain router fits into a RAG chain that selects a retriever, retrieves context, and generates an answer.

• Routed RAG answer workflow: Trace how the complete RAG chain routes an accommodation or tourist-information question to the appropriate retriever, then supplies the retrieved context and original question to the LLM to synthesize an answer.
  Why it matters: Routing lets one RAG chain answer different kinds of questions using the relevant retrieved context.

Key terms: chain router, RAG chain, retriever, context, original question, LLM

10.7 Retrieval postprocessing

This section introduces retrieval postprocessing as a way to filter retrieved document chunks before an LLM uses them to generate an answer.

• Purpose of retrieval postprocessing: Explain where postprocessing fits in RAG and why retrieved chunks may be filtered before answer synthesis.
  Why it matters: Retrieval can return chunks that need not all be included in the context used to answer a question.

Key terms: retrieval postprocessing, document chunks, answer synthesis

10.7.1 Similarity postprocessors

This section explains how a similarity-score cutoff filters retrieved chunks before they reach the LLM.

• Similarity threshold filtering: Explain how a similarity threshold reduces retrieved chunks, and describe how to configure and run a score-threshold retriever in LangChain.
  Why it matters: A cutoff keeps less relevant retrieved content out of the LLM’s input.

Key terms: similarity retriever, similarity score, similarity_score_threshold, search_kwargs, get_relevant_documents()

10.7.2 Keyword postprocessors

This section introduces keyword-based filtering of retrieved document chunks as a postprocessing step.

• Keyword filtering of retrieved chunks: Explain how a keyword postprocessor includes chunks containing required terms and excludes chunks containing forbidden terms, and state that the section demonstrates a Python implementation rather than a built-in LangChain postprocessor.
  Why it matters: Keyword filtering gives you a direct way to control which retrieved chunks remain available after retrieval.

Key terms: keyword postprocessor, required_keywords, excluded_keywords, retrieved document chunks

10.7.3 Time weighting

This section explains how retrieval can favor chunks that are both relevant and recently accessed.

• Recency-aware chunk ranking: Explain how access timestamps let a TimeWeightedVectorStoreRetriever rank chunks using both similarity and recency.
  Why it matters: It helps retrieval favor relevant content that is also timely.

Key terms: last_accessed_at, TimeWeightedVectorStoreRetriever, time decay factor, similarity score, recency

10.7.4 RAG fusion (Reciprocal Rank Fusion)

This section explains how RAG fusion uses multiple queries and Reciprocal Rank Fusion to rerank retrieved documents before answer synthesis.

• Multiple-query retrieval purpose: Explain why RAG fusion generates alternative queries from one user question before retrieving documents.
  Why it matters: The alternative queries provide the result sets that RRF combines.
• Reciprocal Rank Fusion ranking: Trace how RRF scores results from the generated queries and how the reranked documents are used in the RAG chain.
  Why it matters: RRF gives you finer control over which retrieved documents reach the LLM.

Key terms: multiple alternative queries, RAG fusion, Reciprocal Rank Fusion, RRF score, rank, k, RAG fusion retrieval chain

10.8 Summary

The section summarizes how RAG systems query different data stores, route questions to suitable sources, and combine keyword and semantic retrieval results.