What you will learn in Building Reliable AI Systems

Applications and agents you can trust

by Rush Shahani

Read it on liveBook Get the book at manning.com

Chapter by chapter

Chapter 1: AI reliability: Building LLMs for the real world

The introduction contrasts rapidly expanding LLM capabilities with their unreliable performance in production, then defines reliability and its three layers.

1.1 From benchmarks to the real world · 1.2 The tangible impact of LLMs in the real world · 1.2.1 Legal industry · 1.2.2 Customer service · 1.2.3 Programming and development · 1.2.4 Enterprise AI · 1.3 Understanding hallucinations and reliable AI · 1.3.1 When AI hallucinates convincingly · 1.3.2 What is a hallucination? · 1.3.3 What is reliable AI? · 1.4 The AI reliability framework · 1.4.1 Layer 1: Reliable outputs · 1.4.2 Layer 2: Reliable agents · 1.4.3 Layer 3: Reliable operations · 1.5 The reliability toolbox · 1.6 Why reliable AI systems matter now · 1.7 Requirements for following along · 1.8 Summary

What you learn in chapter 1 →

Chapter 2: Generating trustworthy responses with prompt engineering

The introduction explains how prompt engineering guides LLM outputs and why plausible hallucinations make reliable prompting important.

2.1 Choosing the right model · 2.1.1 Reasoning vs. nonreasoning models · 2.1.2 The cost-capability spectrum · 2.1.3 Model selection as a reliability decision · 2.2 Tailoring LLM settings for maximum reliability · 2.2.1 Optimizing temperature for predictable outputs · 2.2.2 Top-P Sampling technique · 2.3 Limiting output and turns to reduce hallucination risk · 2.4 Applying frequency and presence penalties for balanced content · 2.5 Minimizing intrinsic randomness for stable performance · 2.6 Foundations of prompt engineering for reliable LLMs · 2.6.1 Designing components of a prompt for reliability · 2.6.2 Crafting basic prompts with a focus on dependability · 2.7 Prompt engineering techniques for preventing hallucinations · 2.7.1 Zero-shot prompting · 2.7.2 Few-shot prompting for contextual stability · 2.7.3 Chain-of-thought prompting for transparent reasoning · 2.7.4 Enhanced CoT prompt · 2.7.5 Automatic CoT for scalable reasoning · 2.7.6 Self-consistency for cross-verification · 2.7.7 Tree-of-thought prompting for structured decision-making · 2.8 Project: Creating a reliable weather assistant with OpenAI’s function calls · 2.8.1 Building a weather assistant · 2.8.2 Getting started and defining the weather function · 2.8.3 Integrating the chatbot · 2.9 Summary

What you learn in chapter 2 →

Chapter 3: Grounding outputs with RAG

This introduction explains why prompting and training alone cannot keep a chatbot current, and how RAG uses retrieved information to ground its answers.

3.1 What is RAG? · 3.2 RAG system architecture · 3.2.1 The role of the retriever · 3.2.2 The factual generators · 3.2.3 RAG system flow · 3.2.4 Example: E-commerce shopping assistant · 3.3 Reducing hallucinations with RAG · 3.3.1 Grounding the language model · 3.3.2 Grounding an e-commerce product query · 3.3.3 Advantages of RAG in reducing hallucinations · 3.3.4 Critical applications · 3.3.5 Enhancing transparency · 3.4 Data preparation and reliable indexing for RAG systems · 3.4.1 Introducing LangChain · 3.4.2 Structuring product catalogs · 3.4.3 Creating the searchable vector index · 3.4.4 Implementing an index · 3.5 Building an effective RAG system · 3.5.1 Building an e-commerce chatbot powered by RAG · 3.5.2 Processing user queries · 3.5.3 Crafting an accurate response using an LLM · 3.6 Evaluating and optimizing RAG systems · 3.6.1 The role of evaluator LLMs in assessing hallucinations · 3.6.2 Crafting evaluation datasets · 3.6.3 Practical metrics for RAG evaluation · 3.6.4 Implementing RAGAS evaluation · 3.6.5 Reducing hallucinations by filtering retrievals and using metadata · 3.6.6 Metadata-based filtering in a telecom chatbot · 3.6.7 Embedding chunk sizes and model choice · 3.7 Advanced techniques for improving RAG · 3.7.1 Fine-tuning embeddings for in-domain relevance · 3.7.2 Incorporating metadata to reestablish context · 3.7.3 Implementing hybrid retrieval · 3.7.4 Enhancing queries via rephrasing and augmentation · 3.7.5 Implementing query routing · 3.7.6 GraphRAG: Knowledge graph-enhanced retrieval · 3.7.7 When to use GraphRAG vs. traditional RAG · 3.7.8 Agentic RAG: Agent-controlled retrieval · 3.8 Building an RAG system for an e-commerce chatbot · 3.8.1 Step 1: Install the required libraries · 3.8.2 Step 2: Import libraries · 3.8.3 Step 3: Load the documents · 3.8.4 Step 4: Document transformers · 3.8.5 Step 5: Embed text · 3.8.6 Step 6: Choose a vector store · 3.8.7 Step 7: Prepare the LLM model · 3.8.8 Step 8: Add a retriever · 3.8.9 Step 9: Create a retrieval QA chain · 3.8.10 Step 10: Test the chatbot · 3.8.11 Step 11: Evaluate and prevent hallucinations with LangChain · 3.8.12 Challenges and adaptations of the chatbot · 3.9 Summary

What you learn in chapter 3 →

Chapter 4: Embeddings and vector search

The introduction shows how embeddings enable RAG retrieval and why poor embedding quality can make an otherwise capable chatbot unreliable.

4.1 What are embeddings? · 4.1.1 A mental model: Embeddings as coordinates in meaning space · 4.1.2 Why the model matters · 4.2 Embedding models in production · 4.2.1 Commercial models: Powerful but opaque · 4.2.2 Open source models: Flexible and transparent · 4.2.3 Domain-specific models: Purpose-built precision · 4.2.4 How to choose the right model · 4.2.5 Embeddings across industries · 4.2.6 Case study: Learning from Airbnb’s embedding journey · 4.3 Beyond simple vectors: Hybrid and multistage retrieval · 4.3.1 The limitations of pure vector search · 4.3.2 Hybrid retrieval: Combining dense and sparse approaches · 4.3.3 Multistage retrieval: Building precision through layers · 4.3.4 Building a hybrid retriever: Hands-on implementation for an employee chatbot · 4.4 Essential retrieval optimizations with embeddings and RAG · 4.4.1 Metadata filtering: Beyond pure semantic search · 4.4.2 Chunking strategies: Finding the right granularity · 4.5 Exploring vector storage and databases · 4.5.1 HNSW: The algorithm powering modern vector search · 4.5.2 FAISS: Industry-standard vector indexing · 4.5.3 Vector databases: Complete vector storage solutions · 4.5.4 Vector database project: Building a health-care policy assistant with Pinecone · 4.6 Practical challenges: Drift and compression · 4.6.1 Embedding drift explained · 4.6.2 Diving deeper into vector compression · 4.7 Summary

What you learn in chapter 4 →

Chapter 5: Fine-tuning LLMs for improved performance

The introduction explains why specialized tasks may call for fine-tuning and defines it in contrast to prompting and RAG.

5.1 Choosing the right approach: Prompting, RAG, or fine-tuning · 5.1.1 When to use each approach: A practical decision framework · 5.1.2 Hybrid approach: Combining RAG and fine-tuning · 5.2 Case studies: Fine-tuning in the real world · 5.2.1 Medicine: Med-PaLM 2 · 5.2.2 Finance: BloombergGPT and FinGPT · 5.2.3 Code generation: Codex · 5.3 Understanding the fine-tuning process · 5.3.1 Data preparation · 5.3.2 Fine-tuning closed-source models (OpenAI) · 5.3.3 Understanding key fine-tuning approaches · 5.3.4 Building a customer-support assistant · 5.4 Knowledge distillation for LLMs · 5.4.1 How knowledge distillation works in LLMs · 5.4.2 Best practices · 5.5 Summary

What you learn in chapter 5 →

Chapter 6: Creating effective AI agents

The chapter introduction contrasts a standalone LLM’s ability to respond with an agent’s ability to interact with real-world systems, and notes a production risk that reliable agents must control.

6.1 Why do we need agents? · 6.1.1 Static knowledge · 6.1.2 Inability to act · 6.1.3 No workflow management · 6.1.4 How AI agents solve these challenges · 6.1.5 Broader applications of AI agents · 6.2 Core components of reliable agents · 6.2.1 Memory: Context that sticks · 6.2.2 Balancing memory systems with LangChain · 6.2.3 Active tool use vs. passive retrieval · 6.2.4 Strategies for reducing hallucinations with tools · 6.2.5 Decision-making: Making AI agents intelligent · 6.3 Agentic RAG for reliable agents · 6.3.1 Project: Building an agentic RAG system with LangChain · 6.3.2 Setting up dependencies · 6.3.3 Building the knowledge base · 6.3.4 Creating interactive tools · 6.3.5 Configuring the agent’s behavior · 6.3.6 Implementing transparent decision-making · 6.3.7 Managing conversation context · 6.3.8 Making the system work in production · 6.4 Advanced techniques · 6.4.1 Cross-referencing · 6.4.2 Guardrails for security and preventing hallucinations · 6.4.3 Reinforcement Learning from AI Feedback · 6.4.4 Transparency in multistep reasoning · 6.4.5 Case studies for agents in production · 6.5 Summary

What you learn in chapter 6 →

Chapter 7: Tool integration and MCP

This introduction explains why connecting AI assistants to external systems becomes fragile at scale and presents MCP as a standardized alternative to one-off integrations.

7.1 What is MCP? · 7.2 The hidden complexity: The N×M problem · 7.3 The solution: MCP · 7.4 Building your first real MCP tool with a CSV-powered product catalog · 7.4.1 Loading your product catalog · 7.4.2 Setting up the MCP server · 7.5 Running and testing your server · 7.5.1 Why this matters · 7.5.2 Next steps · 7.6 Teaching your AI to use MCP tools · 7.6.1 How models learn what tools they can use · 7.6.2 A complete example: Model > MCP > answer · 7.6.3 What you didn’t have to write · 7.6.4 Tool design becomes interface design · 7.7 Adding a second tool: Inventory status · 7.7.1 How models chain tools · 7.7.2 Tool descriptions guide model behavior · 7.8 Handling failures gracefully · 7.8.1 Catching and communicating errors · 7.8.2 Why structured responses matter · 7.8.3 Handling empty results vs. errors · 7.8.4 Timeout and rate-limiting · 7.9 Beyond tools: Production agent harnesses and OpenClaw · 7.9.1 Why harnesses matter · 7.9.2 Designing agent-legible environments · 7.9.3 The security gap · 7.10 Summary

What you learn in chapter 7 →

Chapter 8: Multi-agent systems

The introduction explains why a customer request spanning images, products, and policies calls for coordinated specialist agents rather than isolated tools or one overloaded model.

8.1 Multi-agent systems · 8.1.1 Why multi-agent architecture matters · 8.1.2 Introducing LangGraph: The multi-agent framework · 8.1.3 Building intelligent agent coordination · 8.1.4 Real-world example: ShopBot multi-agent system · 8.1.5 Running your multi-agent system · 8.1.6 Adding vision: Image-based product search with VisionAgent · 8.1.7 What you built and why it matters · 8.2 Testing multi-agent workflows · 8.2.1 Category 1: Blended queries and multimodal interactions · 8.2.2 Category 2: Ambiguous intent and robust routing decisions · 8.2.3 Category 3: Agent failure and resilience verification · 8.2.4 Category 4: State flow and information preservation across the workflow · 8.2.5 Regression: Moving forward without breaking what worked · 8.3 Alternative framework: CrewAI · 8.3.1 LangGraph vs. CrewAI: Different philosophies · 8.3.2 ShopBot with CrewAI · 8.3.3 When to use which multi-agent framework · 8.3.4 The framework landscape · 8.4 Summary

What you learn in chapter 8 →

Chapter 9: Evaluation and performance for LLMs and agents

The chapter introduction explains why increasingly capable LLM and agent systems need attention to both silent hallucinations and performance under production load.

9.1 Identifying and measuring hallucinations · 9.1.1 Four steps to identify and measure hallucinations · 9.1.2 FActScore: fine-grained factual evaluation · 9.1.3 ROUGE for summarization evaluation · 9.1.4 LLM-as-a-judge: A holistic approach · 9.1.5 Red teaming and stress testing · 9.1.6 Using monitoring frameworks to detect hallucinations · 9.2 Essential architectural patterns for performance · 9.2.1 Token streaming: Presenting answers incrementally or at the same time · 9.2.2 Handling surges with batching: System-level vs. OpenAI’s Batch API · 9.2.3 Caching for efficiency · 9.2.4 Multimodel fallback: Matching each query to the right model · 9.2.5 Project: Building an e-commerce LLM service with batching, caching, and model fallback · 9.2.6 Further improvements · 9.3 Evaluating agent performance · 9.3.1 Core metrics for evaluating LLMs · 9.3.2 Evaluating agents: Beyond traditional LLM metrics · 9.4 Summary

What you learn in chapter 9 →

Chapter 10: Deploying and monitoring

This introduction explains why an LLM that succeeds in testing can still fail in production and why evaluation must be backed by operational infrastructure.

10.1 Introducing large language model operations · 10.2 Serving LLMs: Hosted APIs vs. open source models · 10.2.1 Using hosted APIs · 10.2.2 The open source alternative · 10.2.3 The hybrid solution: Best of both worlds · 10.3 Building LLM-native monitoring systems · 10.3.1 What really matters: The four questions · 10.3.2 Logging what matters · 10.3.3 Setting up alerts that help · 10.3.4 Catching cost explosions before they hurt · 10.3.5 Building dashboards that drive action · 10.3.6 Output-quality monitoring · 10.4 User experience and feedback monitoring · 10.4.1 Explicit feedback collection · 10.4.2 Implicit feedback signals · 10.4.3 Building actionable feedback loops · 10.5 Ensuring high-quality outputs in production · 10.5.1 The three-pillar quality framework · 10.5.2 Prompt engineering for consistent quality · 10.5.3 Continuous quality monitoring with automated testing · 10.6 Observability in practice: Introducing Langfuse · 10.7 Summary

What you learn in chapter 10 →

Chapter 11: Bias, privacy, and responsible AI

The introduction shows how historical inequalities in training data can become amplified discrimination in production AI systems.

11.1 The responsible AI imperative · 11.1.1 Regulatory pressure is accelerating · 11.1.2 User expectations have shifted · 11.1.3 Business risks have multiplied · 11.1.4 Real examples of AI bias in production · 11.1.5 The four failure modes · 11.1.6 The responsible AI defense system · 11.2 Data layer: Where bias begins · 11.2.1 The fine-tuning bias trap · 11.2.2 Detecting bias in chat logs · 11.2.3 The name experiment · 11.2.4 Three proven bias-mitigation strategies · 11.3 Model layer: Where bias evolves · 11.3.1 Why this matters for open source models · 11.3.2 Example: Adding fairness to a LoRA fine-tuning loop · 11.3.3 Anthropic’s constitutional AI: LLM-as-a-judge at training scale · 11.4 Safety layer: Your last line of defense · 11.4.1 Multilayered safety architecture · 11.4.2 Layer 3: Enhanced safety with commercial APIs · 11.5 Privacy layer: Protecting personal data · 11.5.1 Why LLM privacy failures are uniquely dangerous · 11.5.2 Building sensitive data detection · 11.5.3 Health-care privacy protection · 11.5.4 European data protection · 11.6 Real-world project: SafeMedAssist · 11.6.1 Why build a medical AI assistant? · 11.6.2 Professional testing with LangTest · 11.6.3 Production-deployment considerations · 11.6.4 The business case for responsible AI · 11.6.5 Completing the reliability framework · 11.6.6 The six principles of reliable AI · 11.6.7 Final thoughts · 11.7 Summary

What you learn in chapter 11 →

Concepts explained in this book

retrieval-augmented generationA retrieval-augmented generation approach that retrieves and synthesizes stored information without taking actions in the real world.
embeddingAn embedding is a numerical representation of content that captures semantic meaning in a form suitable for comparison.
metadata filterSelecting or excluding records according to their associated descriptive attributes.
semantic cachingA caching technique that reuses results for queries with similar meanings, rather than requiring exact textual matches.
vectorizationA model that maps input content to numerical vectors that capture semantic properties.
fine-tuningFine-tuning is the adaptation of a pretrained model by further training it, typically on task- or domain-specific data.
AI agentA software system that interprets information, makes decisions, and takes actions to pursue goals.
low-rank adaptationLow-Rank Adaptation is a parameter-efficient fine-tuning method that trains added low-rank matrices while keeping a pretrained model's weights frozen.
QLoRAQuantized Low-Rank Adaptation combines low-rank fine-tuning with quantized model weights to reduce memory requirements.
monitoringThe ongoing observation of a deployed AI system's behavior, performance, and quality.
memoryA memory mechanism stores and retrieves information so an AI system can retain context across task steps or interactions.
multimodel fallbackA model-routing strategy that sends simpler queries to a cheaper model and escalates harder or inadequately answered queries to a more capable model.
vector databaseA vector database is a database designed to store and retrieve vector representations, often by similarity.
batchingTemporarily grouping incoming application requests to manage concurrency and reduce processing overhead.
biasBias is a systematic tendency in an AI system that produces unfairly skewed or discriminatory outcomes.
hybrid searchA retrieval technique that combines multiple approaches to find relevant information.
reliabilityThe ability of an AI system to behave dependably and maintain accurate, safe performance under its intended operating conditions.
similarity searchA search technique that retrieves content based on meaning or semantic similarity rather than only exact keyword overlap.
prompt engineeringThe deliberate design of instructions and context to guide a language model's behavior and outputs.
hallucinationA hallucination is a model-generated output that presents fabricated or unsupported information as if it were true.
temperatureTemperature is a language-model generation parameter that controls the randomness of token selection.
evaluation datasetA collection of examples used to assess a model or system's behavior against defined criteria.
multi-agent systemA system composed of multiple AI agents that coordinate to perform tasks.
cachingStoring results so they can be reused instead of recomputed or retrieved again.
GPTCacheGPTCache is a library that intercepts requests to GPT-based endpoints and caches exact or semantically similar queries.
preprocessingThe cleaning and transformation of raw data before it is supplied to a machine-learning process.
evaluatorAn evaluation technique in which a large language model assesses the quality of another model's output.
ReActAn agent framework that interleaves reasoning, actions, and observations to guide task execution.
training datasetTraining data is the data used to fit a machine-learning model's parameters.
MCPA protocol for connecting AI applications and agents to external data sources and tools.
routingThe technique of directing different queries or tasks to selected models according to their requirements or cost.
prompt-completion paradigmA fine-tuning data pattern in which each example pairs an input prompt with an expected output completion.
validation setA held-out subset of data used to assess a model during development without fitting its parameters on that subset.
tool integrationThe process of connecting tools to an AI system so it can invoke them as part of its operation.
two-pass routingA method that evaluates a task with one model, then escalates to another model if the first result is inadequate.
Tree-of-thoughtA prompting method in which a language model explores and evaluates multiple reasoning paths before selecting a solution.
PineconePinecone is a managed vector database service for storing and searching vector embeddings.
approximate nearest neighbor algorithmA search method that efficiently finds vectors close to a query vector while allowing approximate rather than exact nearest neighbors.
working memoryAn agent memory that temporarily retains information, typically for the duration of a single session.
privacyThe protection of information about people from inappropriate collection, access, use, or disclosure.
state managementState management is the handling and maintenance of an application's current state as it executes and responds to interactions.
implicit feedbackImplicit feedback is evaluative information inferred from user behavior rather than directly supplied by users.
NormalFloat4NF4 is a four-bit quantization data type with quantization bins distributed according to a normal distribution.
grounding defect rateGrounding Defect Rate is a metric for quantifying the rate of unsupported claims in generated responses.
token streamingThe incremental delivery of generated text as tokens become available, rather than waiting for a complete response.
costThe monetary expense of using an API service, typically based on usage.
exact string matchingA matching technique that considers two strings a match only when their text is identical.
precisionRetrieval precision is the proportion of retrieved text chunks that are relevant to a query.
dual-encoder architectureA retrieval architecture with separate encoders that independently represent queries and candidate items in a comparable embedding space.
cross-encoderA cross-encoder is a model architecture that jointly processes a pair of inputs to produce a relevance or similarity score.
groundingA technique for anchoring an AI system's outputs in external information or evidence.
lexical searchA retrieval method that searches using sparse representations of content, such as term-based vectors.
tool usageAn AI capability in which a model invokes external tools to obtain information or perform actions.
chain-of-thought promptingChain-of-thought prompting is a prompting method that elicits explicit intermediate reasoning steps from a model.
chunkingA method for dividing source material into segments for processing or retrieval.
decision frameworkA structured approach that guides an AI agent in reasoning, selecting tools and actions, and sequencing tasks.
metadataDescriptive information associated with documents or other data, such as titles, filenames, and keywords.
distillationA training process that transfers the performance of a larger model to a smaller model.
feedback loopA process that uses information about a system's outputs to guide its later behavior or improvement.
RetrieverA retrieval component that finds relevant information from an external collection for use as model context.
promptInput instructions or context supplied to an AI model to guide its response.
evaluationThe systematic measurement of an AI system's behavior and quality against defined criteria.
transparencyThe degree to which an AI system's behavior, decisions, and relevant processes can be understood or inspected.
LLMOpsLLMOps is the engineering discipline of managing large language models in production, including their deployment, evaluation, and ongoing operation.
context windowThe maximum amount of input content a model can process in a single input.
message listAn ordered collection of messages that can retain a conversation's history.
top-p filteringTop-p sampling is a language-model generation setting that samples from the smallest set of tokens whose cumulative probability reaches a specified threshold.
fairnessThe property of an AI system treating people and groups equitably and avoiding unjustified disparities in outcomes.
reliable agentAn AI agent that performs tool use and multi-step workflows while taking safe actions.
gradient accumulationA technique that sums gradients across multiple mini-batches before updating model parameters.
end-to-end testingTesting that evaluates a system by simulating a complete task from its initial input through its final outcome.
BM25BM25 is a keyword-based ranking algorithm that scores documents according to query-term frequency while accounting for document length and term saturation.
hybrid deploymentA hybrid deployment architecture combines multiple deployment approaches or environments within one system.
semantic similarityA measure of how closely the meanings of two pieces of content correspond.
buffer memoryA conversation-memory pattern that retains the complete history of messages rather than compressing it.
summary memoryA conversation-memory pattern that maintains a compressed summary of the conversation instead of its complete message history.
content-aware chunkingA chunking method that uses content structure or meaning to create segments, potentially with shared content between adjacent segments.
HNSWHierarchical Navigable Small World is an approximate nearest-neighbor search algorithm that organizes vectors in a multilayer graph.
parameter-efficient fine-tuningFine-tuning methods that adapt a small subset or representation of a model’s parameters to reduce training resource requirements.
versioningA technique for associating entries with versions so entries from an outdated version can be invalidated.
action executionThe capability of carrying out tasks by interacting with external systems rather than merely describing possible solutions.
observability systemA software platform that collects and presents information about system behavior to support analysis and troubleshooting.
N×M integration problemThe N×M integration challenge is the scaling problem in which integrating N AI applications with M external systems requires a separate custom integration for each pairing.
chunk granularityThe size of a text segment produced when source content is divided for processing or retrieval.
adversarial test setA collection of challenging or ambiguous inquiries designed to expose likely model errors such as hallucinations.
dense vectorA numerical vector in which most or all dimensions have nonzero values.
grounding datasetA collection of authoritative information or evidence used to support or verify system responses.
vector spaceA mathematical space whose elements are vectors and in which vector operations are defined.
multi-stage retrievalA retrieval technique that finds and ranks results through a sequence of specialized filtering and ranking stages.
specialized agentA specialized agent is an AI agent configured to perform a focused class of tasks within a broader coordinated system.
generatorA model component that produces a response from a query and supplied context.
semantic relevance filteringA retrieval technique that filters information according to semantic similarity to a query or other reference.
data annotationThe process of adding human-provided labels, rankings, or other judgments to data.
overfittingA model-training outcome in which a model fits training examples too closely and generalizes poorly to new data.
information retrievalRetrieval is the process of finding relevant information from an external collection to supply as model context.
degradationA decline in the quality of a system's outputs compared with an earlier or expected level.
agentic RAGA retrieval-augmented generation approach in which an AI agent controls the retrieval process, including deciding when multiple retrievals are needed.
human-in-the-loopHuman supervision of an AI system's outputs, decisions, and operation.
MCP serverAn MCP server is a server component that implements the Model Context Protocol and exposes tools or data capabilities to compatible AI applications.
consistencyThe degree to which a language model produces stable and coherent outputs across responses.
open-source modelA model whose weights are made available under terms that permit users to obtain and run it, and potentially modify or adapt it.
shadow testingShadow testing evaluates a system or change alongside a live system without allowing the test results to affect user-facing behavior.
checkpointingThe practice of saving execution state so a process can be paused and later resumed.
LangGraphLangGraph is a library for building stateful, graph-based applications with language models and agents.
managed APIA hosted API is an application programming interface provided by an external service whose infrastructure runs the underlying model or software.
cosine similarityCosine similarity is a measure of the angular similarity between two vectors, calculated from the cosine of the angle between them.
stress testStress testing evaluates a system under unusually high loads or demanding conditions to identify weaknesses and failure modes.
booking conversion rateThe proportion of relevant user visits or interactions that result in a booking.
cold start problemThe difficulty of producing useful recommendations or representations for new items that lack interaction history.
de-biasing techniqueA method for reducing systematic distortions in data or model behavior so learned patterns better reflect the underlying signal.
position biasA distortion in interaction data caused by users being more likely to engage with results because of their displayed position.
branchConditional routing is the technique of directing execution to different agents or workflow paths according to the current request or state.
language modelA model that represents patterns in language to predict or generate sequences of text.
quantizationQuantization reduces the numerical precision used to represent model values, often lowering storage or computation requirements.
FActScoreFActScore is a metric that evaluates the factual accuracy of generated text by decomposing it into atomic facts and checking each fact.
customer satisfactionThe degree to which users are satisfied with a system or its outcomes.
model selectionThe process of choosing an AI model for a particular application or task.
red-teamingThe practice of deliberately probing a system for weaknesses, unsafe behavior, or failure modes.
rerankingReranking is the reordering of retrieved results using a further relevance-scoring step.
relevanceA measure of how directly an answer addresses the question or task posed.
embedding driftA gradual divergence between user queries and the vector representations of documents in an index, caused by changes in how people ask questions or in language and terminology.
product quantizationProduct quantization compresses vectors by dividing them into subvectors and representing them with entries from learned codebooks.
summarizationThe process of condensing information into a shorter representation that preserves important content.
user input leakageA user-input leak is the exposure of sensitive information supplied by a user to an AI system.
vector compressionA technique for reducing the memory footprint of high-dimensional embeddings while preserving most of their semantic properties.
rule-based systemA system that makes decisions by applying predefined logic or rules to inputs.
dialogueA record of the messages exchanged during a conversation.
agent harnessA harness is orchestration infrastructure that manages an AI agent's loop, state, tools, approvals, tracing, and output channels.
student modelA model trained to imitate a teacher model, often with the aim of being smaller or more efficient.
teacher modelA larger, more capable model whose behavior or outputs serve as the target for a smaller model to learn.
personal dataInformation relating to an identified or identifiable person.
PII detectionThe process of identifying information in data or outputs that can identify a specific person.
agent orchestrationThe coordination and sequencing of multiple task steps to accomplish a larger objective.
HIPAAThe Health Insurance Portability and Accountability Act is a United States federal law that includes requirements for protecting health information.
retrieved documentInformation selected from an external knowledge source and supplied to a model as response context.
LangChainLangChain is an open-source software framework for building applications that coordinate language models with prompts, data sources, tools, and retrieval workflows.
guardrailA constraint that defines behavior a model should or should not exhibit.
responsible AIPractices that address the safety, fairness, accountability, transparency, and societal impact of AI systems.
output formatAn output organized according to a predefined schema so that both people and software can interpret its contents consistently.
zero-shot promptingZero-shot prompting is instructing a model to perform a task without providing task-specific examples in the prompt.
testTesting is the systematic execution of queries or other inputs to assess a software system's behavior and identify defects.
text chunkA segment of text produced by dividing a larger document for processing or retrieval.
list_toolsAn MCP server endpoint that returns machine-readable descriptions of its available tools and their inputs.
CrewAICrewAI is a framework for building multi-agent systems with role-based agent definitions.
HaystackHaystack is an open-source framework for constructing modular search and language-model pipelines.
continuous improvementAn ongoing process of using system information to make iterative changes that improve performance or quality.
right to erasureA data-protection right allowing a person to request deletion of their personal data under applicable conditions.
deploymentThe release of software or a software change into an environment where it can operate.
RagasRAGAS is an open-source framework for evaluating retrieval-augmented generation systems.
multi-step reasoningReasoning that reaches a conclusion through multiple connected inferential steps.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.