chapter fourteen

14 Asking a KG questions with natural language

 

This chapter covers

  • Understanding the limitations of RAG in complex scenarios
  • Building an advanced question-answering system that mimics domain expertise on KGs
  • Transforming query results into meaningful, actionable summaries

In this chapter, we will explore how to build an advanced system that can answer questions effectively. Using a law enforcement example as our guide, we’ll compare the retrieval-augmented generation (RAG) approach and our new “expert emulation” method for capturing the expertise of skilled information retrieval.

The framework we’ll develop rests on several pillars:

  • Understanding and properly routing different types of user questions
  • Extracting and representing domain knowledge in a form that LLMs can use effectively
  • Implementing expert-like reasoning patterns for query construction
  • Ensuring that results are presented in meaningful, actionable ways

This framework is designed to integrate with a front-end layer, ensuring that the question-answering system can be presented to end users through a graphical interface. This integration-first approach influences many of our design decisions throughout the chapter, from how we structure query responses to how we handle data visualization.

14.1 Querying a knowledge graph in the policing domain

14.1.1 Enabling domain experts with knowledge graphs

14.2 RAG for KG querying: Capabilities and challenges

14.2.1 RAG effectiveness with complete context

14.2.2 RAG fragility with incomplete retrieval

14.3 Schema-based approach for querying KGs

14.3.1 Understanding and using graph schemas

14.4 Think like an expert: Using metadata for enhanced querying

14.5 Intent detection: Understanding user expectations

14.5.1 Classifying by visualization type

14.5.2 Is it data, documentation, or just complaining?

14.6 From schema to LLM-ready context

14.6.1 Schema extraction and representation

14.6.2 Enriching schemas with descriptive annotations

14.6.3 A practical approach to schema representation

14.7 It’s time to think: Understanding LLM reasoning