Chapter 14: Asking a KG questions with natural language

Introduction

The chapter introduces an expert-emulated approach to asking knowledge graphs questions and explains why its design accounts for presenting answers to end users.

• Front-end integration as design constraint: Explain why the question-answering framework is designed with a front end in mind and what that affects.
  Why it matters: Answers need to be designed for how people will receive and use them, not just for how queries are run.

Key terms: front-end layer, integration-first approach, query responses, data visualization

14.1 Querying a knowledge graph in the policing domain

The section explains why policing agencies need a knowledge graph to bring together data generated across investigations and roles.

• Unified policing knowledge graph: Explain why a policing agency uses a knowledge graph to integrate information from different roles and investigative activities.
  Why it matters: This explains why the agency needs connected information before analysts can draw insights from it.

Key terms: KG, single source of truth, roles, processes, and data

14.1.1 Enabling domain experts with knowledge graphs

This section explains why giving policing analysts direct, intuitive access to knowledge graphs makes better use of their expertise and removes technical bottlenecks.

• Analyst expertise in KG queries: Explain how domain experts’ contextual knowledge helps them ask nuanced questions of a knowledge graph and find insights that technical users might miss.
  Why it matters: Direct KG access lets analysts apply the expertise they already have rather than leaving every question to a technical intermediary.
• Benefits of removing query barriers: Explain why making KGs accessible without complex query-writing skills can improve decisions, workflows, and use of the graph.
  Why it matters: Removing the query-writing barrier lets analysts use the KG when they need it without making IT the gatekeeper for every question.

Key terms: domain expert, contextual knowledge, KGs, technical barriers, direct access

14.2 RAG for KG querying: Capabilities and challenges

The section explains why law enforcement experts need answers grounded in organizational data rather than generic LLM responses, while flagging incomplete retrieval as a limitation of RAG.

• Organizational data for expert questions: Explain why answers based on an LLM’s training data may offer little value to trained law enforcement professionals, and why RAG is used to answer questions grounded in an organization’s data.
  Why it matters: Grounding answers in organizational data makes the assistant more useful for experts’ actual work.
• Incomplete retrieval limitation: State the limitation the section identifies for RAG when retrieval is incomplete, without assuming the details of the forthcoming example.
  Why it matters: Grounding an answer in organizational data is not enough if relevant context is missing.

Key terms: LLMs, training phase, RAG, data available to an organization, incomplete retrieval, context

14.2.1 RAG effectiveness with complete context

The section shows how an LLM can answer case questions using retrieved witness statements and why broader questions require relevant context from additional sources.

• Complete context for case summaries: Explain why an LLM cannot meaningfully summarize an unfamiliar case on its own, but can summarize it when the relevant witness statements are provided with the question.
  Why it matters: It explains why retrieving case evidence is necessary before asking the LLM for a useful summary.
• Evidence-based assessment of a lead: Describe how the LLM uses supplied witness details to assess the left-handedness lead while keeping its conclusion tentative.
  Why it matters: It shows how a response can help an investigation without turning suggestive observations into certainty.
• Relevant context beyond witness statements: Explain why a question about the victim's criminal-organization connections calls for other sources, and describe how RAG supplies that context.
  Why it matters: The useful context depends on the question, not just on which records are easiest to retrieve.

Key terms: RAG, context, witness statements, left-handed, relevant pieces of information, retrieval, filtering, contextual augmentation

14.2.2 RAG fragility with incomplete retrieval

This section shows how incomplete retrieval can make a RAG system give a confident but incorrect answer, especially when evidence is spread across documents.

• Retrieval-dependent answer accuracy: Explain why a RAG answer depends on retrieving sufficient context, using the missing witness statement as an example.
  Why it matters: A convincing answer is not evidence that the retriever found everything needed to answer the question.
• Fragmented context from retrieval: Describe why passage size and relevance make complete retrieval difficult when an answer spans documents.
  Why it matters: Finding passages that look relevant may still leave out information needed for an accurate answer.

Key terms: RAG, retrieval process, incomplete context, passages, granularity, relevance, fragmented context

14.3 Schema-based approach for querying KGs

The section contrasts the limits of RAG-like context selection with an expert’s use of a graph schema to turn a natural-language question into a precise query.

• Limits of selected context: Explain why selecting relevant passages is not the same as applying domain expertise to a question.
  Why it matters: Relevant context alone may not support the rigorous approach a domain expert would take.
• Schema-guided graph queries: Describe how an expert uses a KG schema to turn a specific request into a traversal and refine it with constraints.
  Why it matters: A schema lets an expert ask for specific graph evidence rather than rely on a small selection of retrieved text.

Key terms: RAG-like approach, context, domain experts, schema, nodes, relationships, traversal, constraints, formal queries

14.3.1 Understanding and using graph schemas

Explains how knowledge of a graph schema turns a natural-language question into constrained traversals that can be composed into executable queries.

• Mapping questions to graph schemas: Explain how to map a natural-language question to the schema’s nodes, properties, relationships, and constraints, using the red Camaro example.
  Why it matters: Schema knowledge connects what a person asks for to what the graph can actually query.
• Constrained traversal composition: Describe how constrained traversals express a question as connected graph entities filtered by characteristics, and how they can be combined for more complex queries.
  Why it matters: Constrained traversals bridge a person’s question and a query the graph database can execute.

Key terms: graph schema, nodes, properties, relationships, constraints, constrained traversals, aggregations, structured queries

14.4 Think like an expert: Using metadata for enhanced querying

The section explains how an LLM can use a knowledge graph’s schema to formulate questions as precise queries rather than generate answers from retrieved context.

• Schema-guided expert reasoning: Explain how an LLM can use a knowledge graph’s schema to reason about entities and relationships and generate contextually appropriate queries.
  Why it matters: The schema gives the model a basis for asking the graph a more precise question.
• Questions as formal graph queries: Distinguish the section’s KG-querying approach from traditional RAG by explaining why it focuses on formulating a formal query rather than generating an answer from retrieved context.
  Why it matters: A well-formulated query lets the system retrieve facts from the graph instead of relying on an answer composed from context.

Key terms: expert reasoning, schema, entities, relationships, contextually appropriate queries, traditional RAG, formal queries, factual retrieval, KG

14.5 Intent detection: Understanding user expectations

This section explains why the question-answering system detects a user’s intent first: to choose a suitable presentation and processing path.

• Intent-guided answer presentation: Explain how detecting a user’s intent helps the system choose both an appropriate way to present an answer and the pipeline or chain to use. Use the red Camaros question to illustrate why an investigation board, table, or map might suit different needs.
  Why it matters: Intent detection helps the system give the user an answer in a useful form, not merely find relevant data.
• Early intent classification rationale: Justify placing intent detection at the beginning of the question-answering process: it mirrors an expert’s first step, and it gives the LLM a relatively straightforward semantic classification task that is easy to follow and debug.
  Why it matters: Starting with a tractable classification step provides a clear foundation for the question-answering system.

Key terms: Intent detection module, user’s intent, pipeline or chain, expert reasoning, semantic understanding, classification step

14.5.1 Classifying by visualization type

This section shows how to classify a knowledge-graph question by the visualization the user wants and how to refine that classification prompt when requests are ambiguous.

• Visualization intent categories: Explain how a question’s requested presentation maps to graph, table, chart, or map output, including what happens when no format is explicit.
  Why it matters: The classifier’s categories need to match the formats the application can actually display.
• Classification prompt structure: Describe the parts of the example classification prompt and how they make its output consistent.
  Why it matters: Clear rules, examples, and an output format make the classification usable by the application.
• Ambiguous cases and fallback: Explain how misclassified boundary cases can improve the prompt and why graph serves as its fallback.
  Why it matters: Ambiguous real-world questions reveal where examples and fallback rules need refinement.

Key terms: intent detection, graph, table, chart, map, classification prompt, task definition, few-shots section, JSON, reason field

14.5.2 Is it data, documentation, or just complaining?

This section broadens intent detection beyond KG data queries and explains how to design and assess prompts that route varied user requests.

• User request categories: Distinguish data-related questions, system-related questions, and feedback or complaints, including the documentation-related and schema-related subcategories of system questions.
  Why it matters: Not every legitimate user request should be treated as a graph-data query.
• Broader classification prompt design: Describe the prompt elements used for broader intent classification and the role of its reason field.
  Why it matters: A reasoned classification is easier to check and improve than an unexplained category label.
• Update frequency boundary case: Explain why a question about how often the KG is updated belongs with system documentation rather than KG data, and how a mistaken classification can guide prompt improvement.
  Why it matters: This boundary case shows how a plausible-looking data query can misroute a user’s actual question.
• Classification architecture trade-offs: Distinguish when a single broad classification prompt is preferable to a multistage approach and explain why the section recommends starting with one prompt.
  Why it matters: The choice balances implementation simplicity against the control needed for difficult or evolving requests.

Key terms: data-related questions, system-related questions, feedback and complaints, documentation-related questions, schema-related questions, task definition, few-shot section, reason field, boundary cases, expected output format

14.6 From schema to LLM-ready context

This section introduces schema extraction and expert-oriented metadata enrichment as ways to make a knowledge graph schema usable by LLMs.

• LLM-ready schema context: Explain why schema extraction and expert-useful metadata matter when preparing a knowledge graph schema for an LLM. Describe how information that makes a schema more understandable or actionable for an expert can also help an LLM support nontechnical users' queries.
  Why it matters: A usable description of the graph gives the LLM better context for helping people query it.

Key terms: schema extraction, metadata, enrichments, LLM-compatible formats

14.6.1 Schema extraction and representation

The section explains how to turn a technical Neo4j schema into a focused, structured conceptual schema that an LLM can use to generate Cypher queries.

• Technical and conceptual schemas: Distinguish the technical schema returned by apoc.meta.schema from the conceptual KG schema needed for natural-language querying. Explain that the conceptual schema retains domain-relevant entities and relationships while removing implementation details.
  Why it matters: An LLM needs the domain model, not every element maintained by the database.
• Purpose of schema filtering: Explain why filtering technical details helps an LLM map natural-language questions to graph elements and generate relevant Cypher queries.
  Why it matters: Filtering is not just cosmetic: it helps prevent irrelevant schema details from steering query generation.
• Conceptual schema representation: Describe the two ways given to derive a conceptual schema from APOC output, and contrast a narrative description with the concise, consistent representation intended for an LLM.
  Why it matters: The chosen representation determines whether the LLM can readily use the curated schema for query generation.

Key terms: apoc.meta.schema, technical database schema, conceptual KG schema, domain experts, cognitive load, query errors, skip list, narrative schema representation, LLM-friendly format, structured representation

14.6.2 Enriching schemas with descriptive annotations

This section explains why an LLM needs descriptive schema annotations, not just graph structure, to translate natural-language questions into accurate KG queries.

• Limits of schema structure alone: Explain how an LLM can generate an incorrect KG query when the schema does not explain stored values or the meanings of relationships.
  Why it matters: A query can look reasonable yet return misleading results if it does not match the KG’s actual values and meanings.
• Descriptive schema annotations: Describe how annotating node classes, relationship types, and properties gives the LLM an expert-like guide for constructing more accurate queries.
  Why it matters: Descriptions supply the context the LLM needs to make its queries reflect what the graph actually stores and means.

Key terms: schema structure, BLK, COMMITTED, CO_OFFENDS_WITH, annotated schema, descriptive annotations, inline comments, cheat sheet

14.6.3 A practical approach to schema representation

This subsection presents a YAML configuration for turning KG schema output into a focused, annotated representation that an LLM can interpret.

• YAML schema configuration roles: Explain how a YAML file refines the output of apoc.meta.schema: its skip section excludes schema elements outside the core conceptual KG, while its descriptions section annotates the purpose and semantics of retained elements.
  Why it matters: A focused schema with meaningful descriptions gives the LLM clearer context for interpreting the graph.
• Schema configuration benefits: Describe why keeping schema filtering and descriptions in a central YAML file supports customization, maintenance, and growth of the KG.
  Why it matters: The configuration can stay useful as domain needs and the knowledge graph change.

Key terms: YAML configuration file, apoc.meta.schema, skip, descriptions, core conceptual KG schema, Customization, Maintainability, Scalability, YAML file

14.7 It’s time to think: Understanding LLM reasoning

The section places Cypher query generation within the question-answering workflow and explains why prompts can encourage an LLM to work through complex questions before answering.

• Query generation inputs and feedback: Describe how query generation combines processed user intent with graph schema data, and how execution errors can feed back into query refinement.
  Why it matters: It connects the earlier work on intent and schema to the task of producing a usable database query.
• Intermediate reasoning prompts: Explain why complex questions can lead LLMs to jump to conclusions, and distinguish chain-of-thought prompting from scratchpad techniques as ways to elicit intermediate work before a final answer.
  Why it matters: Intermediate work is presented as a way to counter an LLM's tendency to jump to a plausible answer on a difficult question.

Key terms: query generation, processed user intent, schema data, execution error feedback, Cypher queries, chain-of-thought prompting, scratchpad techniques, intermediate steps, time to think

14.7.1 The order matters: Answer first vs. reasoning first

This section explains how the order of an LLM’s answer and reasoning affects its output, and why classification tasks can warrant a different order.

• Answer-first consistency bias: Explain why asking an LLM for an answer before its reasoning can lead it to justify that initial answer rather than assess the problem impartially.
  Why it matters: A convincing explanation after an answer may be a rationalization of an earlier mistake.
• Reasoning-first prompt order: Describe the prompt order the section recommends for encouraging more transparent and reliable reasoning, and contrast it with answer-first prompting.
  Why it matters: Prompt order can change whether the output develops toward a conclusion or defends one already stated.
• Classification justification exception: Explain when answer-first prompting can still be useful in classification and what the resulting reasoning is used for.
  Why it matters: A post-answer explanation can be useful for diagnosing a classifier even when it should not be treated as impartial reasoning.

Key terms: semantic consistency, cumulative context, error propagation, step-by-step reasoning, final answer, classification tasks, misclassifies, classification boundaries

14.7.2 Thinking in queries: From text to Cypher

This section builds a structured prompt that turns natural-language questions into schema-compliant Cypher queries and returns the result in a machine-readable format.

• Schema-grounded query prompt: Explain how the prompt frames the user's question and graph schema for Cypher generation, and how examples and explicit constraints keep queries grounded in that schema.
  Why it matters: Grounding the prompt in the actual graph schema helps prevent queries that invent graph elements.
• Intent-dependent result shapes: Distinguish the query requirements for graph or map results from those for table results.
  Why it matters: A query can match the right data yet return it in the wrong form for the user's requested view.
• Selected-node question context: Describe how the prompt handles a user's current node selection, including references to an empty selection.
  Why it matters: Selection context makes references such as “the selected person” usable without guessing which node the user means.
• Reusable context and reminders: Explain why KG-specific notes are separate from the core prompt and why the question, requirements, and optional failure information appear near its end.
  Why it matters: These additions preserve graph-specific meaning while helping the model stay focused and revise a failed query.
• Machine-readable query response: Describe the required JSON response and what its fields communicate about the proposed Cypher query.
  Why it matters: A predictable response lets an application distinguish a usable query from a failure and handle the result programmatically.

Key terms: Cypher, Graph Schema, annotations, examples, labels, relationships, intent detection, output_type, graph, map

14.7.3 Structuring output for reliable query generation

The section explains how ordering a model’s output fields and keeping examples consistent can make generated Cypher queries more reliable.

• Ordered query output fields: Explain why the model is asked to list relationships, provide reasoning, and then generate the Cypher query, in that order. Describe how the relationship-listing requirement can be adjusted to the model’s behavior.
  Why it matters: The field order helps ground the final query rather than asking the model to produce one immediately.
• Consistent prompt examples: Describe why prompt examples follow the response format and how placeholders or an initial prompt without reasoning can be used when preparing examples.
  Why it matters: Examples can reinforce the intended output structure without over-directing the model’s reasoning.

Key terms: relationships, reasoning field, Cypher query, schema description, response format, "..." notation, examples

14.8 Response summarization: From results to insights

This section explains how the final summarization step turns graph query results into a focused textual answer that complements the visual graph.

• Summarization’s role in the pipeline: Explain why summarization follows query execution and how it helps users understand results beyond the graph visualization.
  Why it matters: A graph can show connections without making the answer to the user’s question clear.
• Question-focused summary context: Describe the context supplied to the summarization prompt and why it filters returned data against the original question.
  Why it matters: Filtering against the question keeps the answer useful even when the displayed graph contains extra data.
• Intent-led analysis and output: Explain how the prompt decides whether to add analysis and how its JSON fields lead to the final summary.
  Why it matters: An explicit analysis decision helps the system provide requested insight without adding speculation the user did not ask for.

Key terms: summarization step, graph visualization, node properties, original question, executed Cypher query, query results, current selection, complete paths, results_analysis, reasoning

14.9 Summary