Chapter 1: Improving LLM accuracy

Introduction

This introduction frames GraphRAG as a way to improve LLM accuracy and currency by combining retrieval-augmented generation with knowledge graphs that connect structured and unstructured data.

• RAG for LLM limitations: Explain how retrieval-augmented generation combines an LLM with an external knowledge base at run time to provide more accurate and current responses and reduce, but not eliminate, hallucinations.
  Why it matters: RAG provides the basic approach the book uses to address unreliable or outdated LLM answers.
• Knowledge graphs in RAG: Describe knowledge graphs as representations of entities, attributes, and relationships, and explain how they let a RAG system connect unstructured text with structured records for context-rich, interconnected retrieval.
  Why it matters: Knowledge graphs extend RAG beyond isolated text retrieval so it can use relationships and structured domain information.
• GraphRAG learning scope: State that the book covers augmenting existing RAG architectures with knowledge graphs and building GraphRAG pipelines from scratch, including data modeling, graph construction, retrieval workflows, and evaluation.
  Why it matters: This roadmap clarifies that GraphRAG involves both building the graph and using and evaluating it in a RAG workflow.

Key terms: large language models, LLMs, retrieval-augmented generation, RAG, external knowledge base, hallucinations, knowledge graphs, entities, attributes, relationships

1.1 Introduction to LLMs

This section explains how LLMs generate text from learned language patterns, follow instructions, and produce factual-seeming answers without explicitly storing facts.

• Next-word prediction training: Explain that an LLM is trained on large, diverse text datasets primarily to predict the next word in a sequence, learning patterns such as grammar, context, and associations from that training data.
  Why it matters: This explains both why LLM output can sound natural and why its quality depends on its training data.
• Instruction-following capabilities: Describe how LLMs can follow detailed task instructions to generate varied outputs, such as a haiku in a requested topic and style, rather than acting as simple autocomplete systems. Explain that this capability can be used to tailor and refine RAG-pipeline responses for context, accuracy, and usability.
  Why it matters: Instruction following is what lets a RAG application direct an LLM to use retrieved context and produce an answer in the needed form.
• Learned weights versus fact storage: Explain that an LLM does not store and retrieve explicit facts or events from its training data like a factual database; it generates apparently knowledgeable answers from learned mathematical representations and neural-network weights used for next-word prediction.
  Why it matters: This distinction helps explain why an accurate-looking answer is not proof that the model has reliable factual memory.

Key terms: LLMs, training dataset, next word prediction, patterns, context, instruction-following, RAG pipelines, retrieval components, transformer, neural network

1.2 Limitations of LLMs

This section frames LLMs as powerful AI systems that also have important limitations and introduces the examination of those limitations and their implications.

• LLM limitations overview: Describe how the section positions LLMs as groundbreaking and capable across many applications while also having challenges and constraints whose implications need to be examined.
  Why it matters: Recognizing both LLM capabilities and limitations provides the motivation for the techniques introduced later in the chapter.

Key terms: LLMs, limitations, challenges, constraints, implications

1.2.1 Knowledge cutoff problem

This section explains that an LLM cannot know information published or occurring after the latest date represented in its training data.

• Knowledge cutoff date: Define an LLM’s knowledge cutoff date as the most recent point for which its training data contains information, and explain that the model draws on information from before that point when generating responses.
  Why it matters: The cutoff explains why an otherwise capable LLM may be unable to answer questions about recent events.
• Post-cutoff information limits: Explain why an LLM cannot provide information about events, developments, or research that occurred or were published after its knowledge cutoff date: that material was not included in the training dataset.
  Why it matters: This limitation motivates providing LLM applications with access to current external data.

Key terms: knowledge cutoff date, training dataset, training data, knowledge cutoff problem

1.2.2 Outdated information

This subsection explains that LLM answers can become outdated as real-world facts change after the model’s knowledge cutoff, motivating updated data or real-time information access.

• Outdated response limitation: Explain why an LLM can give an answer that was once correct but is no longer accurate when facts change after its training knowledge was established.
  Why it matters: A response can sound detailed and credible while still misrepresenting the current state of the world.
• Dallas Mavericks ownership example: Describe how the Dallas Mavericks example illustrates outdated information: a response naming Mark Cuban as sole owner became inaccurate after he sold his majority stake to the Adelson family and the Dumonts while retaining a minority share.
  Why it matters: This example shows that even a specific ownership detail can change enough to alter how people understand an organization.
• Current-information maintenance: Describe the two measures identified for keeping AI systems accurate in changing environments: regularly update model training data or enable access to real-time information.
  Why it matters: Keeping information current helps an AI system remain accurate and relevant as events and facts evolve.

Key terms: outdated responses, knowledge cutoff, recent developments, Mark Cuban, Dallas Mavericks, Adelson family, Dumonts, majority stake, minority share, updating training data

1.2.3 Pure hallucinations

This section explains pure hallucinations: LLMs can confidently fabricate incorrect facts because they predict likely text rather than verify truth.

• Pure hallucination definition: Explain that a pure hallucination is an LLM response that sounds plausible and confident but contains factually incorrect or entirely fabricated information, including for events before the model’s knowledge cutoff.
  Why it matters: Recognizing hallucinations prevents confidence and plausible wording from being mistaken for evidence.
• Next-token prediction mechanism: Describe why hallucinations occur: an LLM is a probabilistic language model that predicts likely next tokens from training-data patterns, rather than a reasoning engine that knows or verifies facts.
  Why it matters: This explains why fluent language can be useful without being a guarantee of factual understanding.
• External fact verification: State that users should critically verify LLM claims through reliable external sources, especially URLs, academic citations, identifiers, and other information for which precision is important.
  Why it matters: External verification can catch fabricated references and incorrect factual details before they are relied on.

Key terms: hallucinations, pure hallucinations, knowledge cutoff, probabilistic language models, next token, training data, statistical pattern matching, reasoning engines, reliable external sources, URLs

1.2.4 Lack of private information

This subsection explains why an LLM cannot answer accurately from private company information absent from its training data and why publishing that data is not a viable remedy.

• Private-information limitation: Explain why an LLM cannot reliably answer questions about internal or proprietary company information that was not included in its training data, even when that information predates the model’s knowledge cutoff.
  Why it matters: A useful company assistant needs access to private knowledge without assuming the base LLM already knows it.
• Privacy-preserving information access: Describe why making internal information public so it might enter future LLM training data is impractical and insecure, and state that the book instead pursues strategies that preserve privacy and control.
  Why it matters: Private data must remain protected while still being usable to improve answers in an application.
• Scope of LLM limitations: State that the book focuses on limitations affecting factually correct and up-to-date responses, while acknowledging without covering bias, lack of true understanding and context, prompt injection vulnerability, and inconsistent responses.
  Why it matters: Knowing the book’s scope helps distinguish the accuracy problems it addresses from important LLM risks it does not cover.

Key terms: private information, internal information, proprietary information, training data, knowledge cutoff, data privacy, data control, training dataset, bias in responses, lack of understanding and context

1.3 Overcoming the limitations of LLMs

This section introduces supervised finetuning and retrieval-augmented generation as approaches for injecting domain-specific knowledge into LLM applications that need precise, factually accurate outputs.

• Domain knowledge injection approaches: Describe why business-oriented, ChatGPT-like applications need domain-specific knowledge and identify supervised finetuning and RAG as approaches for injecting that knowledge into an LLM.
  Why it matters: These approaches address the need for precise, factually accurate answers when an LLM must work with specialized or current business knowledge.

Key terms: domain-specific knowledge, supervised finetuning, retrieval-augmented generation, RAG

1.3.1 Supervised finetuning

This section explains supervised finetuning as a comparatively inexpensive way to train LLMs with prompt–response examples, while noting why it is not yet a reliably simple solution for keeping production models factually current.

• Four-stage LLM training process: Describe the four training stages presented for an LLM: pretraining, supervised finetuning, reward modeling, and reinforcement learning. Explain the distinct purpose of each stage and identify pretraining as the especially resource-intensive foundational stage.
  Why it matters: This training context explains why updating a model through full pretraining is impractical and why finetuning was considered instead.
• Supervised finetuning examples: Explain how supervised finetuning supplies an LLM with input prompts and corresponding desired outputs so it learns to produce useful, accurate assistant-like responses. Use the NBA championship question-and-answer example to illustrate the format of a finetuning record.
  Why it matters: Supervised finetuning is the section's proposed alternative to expensive full retraining for adapting an LLM's responses.
• Finetuning reliability limitations: Explain why supervised finetuning is not presented as a dependable production solution for continuously adding factual knowledge: research findings are mixed, LLMs can struggle to learn new facts through finetuning, and reliable deployment remains challenging. Distinguish its lower resource cost from a guarantee that a model will reliably retain or answer new facts.
  Why it matters: These limitations motivate the book's move toward retrieval-based methods for handling knowledge limitations more efficiently.

Key terms: pretraining, supervised finetuning, reward modeling, reinforcement learning, input prompt, desired outputs, question–answer pair, high-quality conversations, factuality, new factual information

1.3.2 Retrieval-augmented generation

This section explains how retrieval-augmented generation retrieves external facts and supplies them with a user question to ground an LLM's answer.

• RAG purpose and grounding: Explain that retrieval-augmented generation (RAG) combines an LLM with an external knowledge base rather than relying only on the model's internal knowledge. Describe how supplying relevant facts in the prompt can improve accuracy, provide up-to-date information, and reduce hallucinations.
  Why it matters: RAG lets an LLM use its language abilities while grounding its answers in external information.
• Retrieval and augmented generation stages: Describe the two stages of the RAG workflow: retrieval locates relevant information in an external knowledge base or database, and augmented generation combines that information with the user's input so the LLM can produce a grounded response.
  Why it matters: Separating retrieval from generation clarifies how a RAG system finds evidence before asking the LLM to answer.
• Automatic prompt enrichment: Trace the behind-the-scenes RAG flow: a user provides only a question, the system turns it into a search query, retrieves suitable content from sources such as documents, knowledge articles, or databases, combines that content with the original question in an enriched prompt, and sends it to the LLM.
  Why it matters: This flow explains why RAG can improve answers without making users manually assemble the context for every question.
• Web search RAG example: Describe ChatGPT Web Search as an example of the RAG idea: the system can search for relevant web information before generating an up-to-date answer, although its exact retrieval and browsing implementation is not publicly disclosed.
  Why it matters: The example connects the abstract RAG workflow to a familiar interface while distinguishing known behavior from undisclosed implementation details.

Key terms: retrieval-augmented generation, RAG workflow, external knowledge base, internal knowledge, hallucinations, retrieval stage, augmented generation stage, retrieved information, grounded response, search query

1.4 Knowledge graphs as the data storage for RAG applications

This section explains why knowledge graphs and graph databases are well suited to RAG because they combine structured facts, relationships, and unstructured text in one connected store.

• Knowledge graph data model: Describe a knowledge graph as a data structure in which nodes represent concepts and entities and relationships connect those nodes, and explain its role as a storage option for a RAG application.
  Why it matters: This model lets a RAG application retrieve not just text but also the explicit connections among important entities.
• Unified structured and unstructured storage: Explain that a knowledge graph can store structured information such as attributes, statuses, and company hierarchies alongside unstructured information such as article text in the same database framework.
  Why it matters: Keeping both data types together supports RAG questions that need precise facts as well as rich textual context.
• Complementary query capabilities: Distinguish the question types supported by structured and unstructured data: structured attributes and relationships enable precise filtering, counting, aggregation, and relationship queries, while unstructured text supplies contextual depth for open-ended or fuzzy questions.
  Why it matters: This distinction explains why neither documents alone nor structured records alone can efficiently answer the full range of RAG questions.
• Cross-data retrieval connections: Explain that explicit links between unstructured text and structured graph data enable advanced retrieval, including linking entities mentioned in text to graph nodes and adding source-passage context to structured results.
  Why it matters: These links let a RAG system combine graph precision with evidence and context from the original text.

Key terms: knowledge graph, graph database, nodes, concepts, entities, relationships, structured information, unstructured information, article contents, structured data

1.5 Summary

This summary recaps LLM limitations and shows how RAG, especially RAG backed by knowledge graphs, supplies reliable external context for more accurate answers.