10 Query generation, routing and retrieval post-processing

 

This chapter covers

  • Generate metadata queries directly from user questions
  • Convert user questions into database-specific queries (e.g., SQL, SPARQL)
  • Route questions to the appropriate handler based on intent
  • Enhance result relevance using Reciprocal Rank Fusion (RRF)

In chapters 8 and 9, you improved RAG answer accuracy using advanced indexing and query transformations. Indexing strengthens embedding effectiveness for broader chunks, adding richer context, while query transformations boost the precision of vector store retrieval.

Now, you’ll dive into three more advanced RAG techniques. First, you’ll learn to generate queries specific to the type of content store in use. For instance, you’ll see how to generate SQL from a user's natural language question to retrieve data from a relational database. Your setup might include several types of content stores—such as vector stores, a relational database, or even a knowledge graph database. You’ll use the LLM to direct the user’s question to the right content store.

Finally, you’ll refine the retrieved results to send only the most relevant content for synthesis, filtering out unnecessary data to maintain clarity and relevance.

10.1 Content Database Query Generation

10.2 Self-Querying (Metadata Query Enrichment)

10.2.1 Ingestion: Metadata Enrichment

10.2.2 Q & A on a Metadata-Enriched Collection

10.3 Generating a Structured SQL Query

10.3.1 Installing SQLite

10.3.2 Setting Up and Connecting to the Database

10.3.3 Generating SQL Queries from Natural Language

10.3.4 Executing the SQL Query

10.4 Generating a Semantic SQL Query

10.4.1 Standard SQL Query

10.4.2 Semantic SQL Query

10.4.3 Creating the Embeddings

10.4.4 Performing a Semantic SQL Search

10.4.5 Automating Semantic SQL Search

10.4.6 Benefits of Semantic SQL Search

10.5 Generating Queries for a Graph Database

10.6 Chain Routing

10.6.1 Setting Up Data Retrievers

10.6.2 Setting Up the Query Router

10.6.3 Integrating the Chain Router into a Full RAG Chain

10.7 Retrieval Post-Processing

10.7.1 Similarity Postprocessor

10.7.2 Keyword Postprocessors

10.7.3 Time Weighting

10.7.4 RAG Fusion (Reciprocal Rank Fusion)

10.8 Summary