Reliability
Reliability is the degree to which a system, process, or measurement performs or yields consistent, dependable results under specified conditions.
Agent Design Patterns
Reliability is the degree to which a system consistently produces correct and dependable results under expected conditions.
- How it is used
- Frames the design of agents and patterns that reduce workflow failures when models, tools, and structured processes work together.
- How deep it goes
- Defines reliability by internal failure rate, gives failure cases and a logging-based measurement approach, but does not specify a recovery procedure.
- Examples
- Uses skipped prompt instructions, malformed model responses, empty search results, and guarded personal-assistant behavior as concrete cases.
- What you can do after
- Evaluate/understand it by logging agent failures on test datasets or in real-world use and checking whether their rate decreases.
Inside AI
Reliability is the degree to which a system performs its intended functions consistently and correctly under specified conditions.
- How it is used
- Treats reliability as a requirement for AI predictions and high-stakes autonomous systems.
- How deep it goes
- Explains consequences and identifies detection and false-positive requirements, without detailing how to implement a reliable system.
- Examples
- Uses rejected payment transactions and an autonomous targeting incident to illustrate failures.
- What you can do after
- Evaluate reliability by considering both detection of fraud and mistaken rejection of legitimate transactions.
Apache Kafka in Action
Reliability is the likelihood that a system continues to perform correctly over a specified period and operating conditions.
- How it is used
- Frames Kafka reliability as preserving durable, available, and consistent messaging when brokers or other components fail.
- How deep it goes
- Defines three concerns and explains a page-cache acknowledgment trade-off, while pointing toward detailed treatment of ACKs, transactions, and replication.
- Examples
- Uses a three-broker failure scenario, page-cache writes acknowledged before disk commit, and a vertical-versus-horizontal scaling comparison.
- What you can do after
- Operate/configure a Kafka deployment with separated brokers and adequate capacity, then monitor replication health and client problems.
Grokking Software Architecture
The degree to which a system performs its intended function consistently over time, including remaining available when required.
- How it is used
- Frames reliability as a quality requirement and decision criterion alongside business constraints.
- How deep it goes
- Defines correct behavior under failure and examines a fragile dependency, without detailing a reliability implementation.
- Examples
- Uses a checkout flow dependent on an external payment processor and contrasts industry constraints in finance and healthcare.
- What you can do after
- Evaluate reliability risks and priorities when weighing architectural choices against business constraints.
Read how Grokking Software Architecture explains reliability →
Effective Data Analysis
Reliability is the consistency or stability of a measurement across repeated observations, items, or conditions.
- How it is used
- Supports trustworthy analytics, stakeholder decisions, and evidence-based practices by checking whether measurements remain consistent.
- How deep it goes
- Defines three reliability forms and gives procedural detail for test-retest analysis, while illustrating internal consistency through Cronbach’s alpha.
- Examples
- Uses depression scales, NPS responses, qualitative topic coding, employee well-being questionnaires, and weather data for illustration.
- What you can do after
- Evaluate measures by selecting test-retest, inter-rater, or internal-consistency checks suited to the measurement and its intended use.
AI Governance
Reliability is the degree to which a system performs consistently and correctly under expected conditions.
- How it is used
- Frames reliability as a prerequisite for trusting GenAI decisions and limiting harm from autonomous agents.
- How deep it goes
- Distinguishes three failure modes, explains their causes, and identifies first controls; notes the limits of exhaustive testing.
- Examples
- Illustrates failures with a circling robotaxi, insurance-policy queries, and a coding agent that deleted a production database.
- What you can do after
- Evaluate reliability failures and select grounding, perturbation testing, or configuration controls for different failure modes.
The Coder Cafe
Reliability is the ability of a system to continue performing its intended functions correctly over time and under expected failure conditions.
- How it is used
- Frames reliability as the user-facing goal guiding both system design and incident response.
- How deep it goes
- Defines a time-bound user expectation and outlines failure-design and debugging practices without prescribing one implementation.
- Examples
- Illustrates the stakes with air traffic and lost cloud photos, then makes expectations concrete with watch precision and datastore availability.
- What you can do after
- Practice/perform reliability design and incident diagnosis through dependency classification, graceful degradation, and segmented metrics.
Building Reliable AI Systems
The ability of an AI system to behave dependably and maintain accurate, safe performance under its intended operating conditions.
- How it is used
- Frames reliability as a production-wide concern spanning outputs, agent actions, and ongoing operations.
- How deep it goes
- Defines reliability dimensions and trade-offs, then outlines engineering capabilities for addressing them.
- Examples
- Uses medical exam performance and a multipart shopping assistant to illustrate benchmark limitations and graceful degradation.
- What you can do after
- Evaluate and understand AI reliability across output quality, action safety, efficiency, and performance over time.
Read how Building Reliable AI Systems explains reliability →
Generative AI in Action
Reliability is the ability of a system to perform its required functions consistently and correctly over time and under expected conditions.
- How it is used
- Guides production deployment of GenAI applications that must meet business needs and cope with failures.
- How deep it goes
- Identifies recovery, monitoring, testing, and scaling practices without providing detailed configuration steps.
- Examples
- Uses simulated user load, throughput and latency measurements, and traffic spikes to illustrate production reliability practices.
- What you can do after
- Evaluate production reliability by considering failure recovery, monitoring, model validation, and load testing.
AI Model Evaluation
The ability of a system to provide correct and available service consistently under expected operating conditions and failures.
- How it is used
- Evaluates whether AI-powered features can serve users quickly, reliably, and cost-effectively in production.
- How deep it goes
- Connects latency, load, failures, fallbacks, and trade-offs, but stops short of defining a single reliability formula.
- Examples
- Uses e-commerce recommendations, peak shopping traffic, fallbacks, validation reruns, and canary revalidation as concrete cases.
- What you can do after
- Evaluate AI features with production latency, load, failure, fallback, and infrastructure evidence alongside model quality.
Starting Data Analytics with Generative AI and Python
Reliability is the degree to which a system produces accurate and timely results consistently under varying conditions.
- How it is used
- Measures whether data-analysis systems deliver accurate, timely results consistently across conditions.
- How deep it goes
- Defines reliability and assesses failure or inaccuracy frequency and severity, without prescribing a calculation.
- Examples
- Contrasts observatory, hospital-monitoring, and chemical-plant systems to show how reliability priorities vary by domain.
- What you can do after
- Evaluate reliability by checking accuracy, timeliness, and the frequency and severity of failures or inaccuracies.
Read how Starting Data Analytics with Generative AI and Python explains reliability →
Machine Learning System Design
Reliability is the ability of a system to continue performing its required functions consistently despite faults, failures, or changing load.
- How it is used
- Guides production inference-platform and pipeline choices by weighing continued service against cost and failure consequences.
- How deep it goes
- Examines failure scenarios and tradeoffs but stops short of specifying technical recovery mechanisms or reliability calculations.
- Examples
- Uses unexpected load spikes, model bugs, hardware failures, and a hot-season platform outage to frame reliability decisions.
- What you can do after
- Evaluate inference designs by balancing reliability, operating cost, performance, scalability, flexibility, and security requirements.
Read how Machine Learning System Design explains reliability →
Effective Platform Engineering
System reliability is the ability of a software system to perform its required functions consistently over an expected period and operating condition.
- How it is used
- Measures platform value through system stability, observability coverage, incident performance, and reduced failures.
- How deep it goes
- Connects reliability to SRE practices, SLOs, SLIs, and error budgets without detailing their implementation.
- Examples
- Uses Epetech’s reduced downtime, fewer critical incidents, higher availability, faster recovery, and improved monitoring as evidence.
- What you can do after
- Evaluate reliability using availability, incident, recovery, telemetry, failure, and defect measures.
Read how Effective Platform Engineering explains reliability →
Architecting an Apache Iceberg Lakehouse
Reliability is the ability of a system to perform consistently and preserve correct results despite failures, retries, and interruptions.
- How it is used
- Guides ingestion architecture by matching delivery guarantees, retry behavior, source capabilities, and table-commit coordination to operational requirements.
- How deep it goes
- Explains delivery semantics, checkpointing, atomic commits, and recovery coordination, while deferring detailed resumability mechanics.
- Examples
- Uses streaming ingestion, batch archives, replayable sources, message queues, checkpointing, and Iceberg snapshot commits as concrete cases.
- What you can do after
- Evaluate ingestion guarantees and coordinate retries, checkpoints, commits, and recovery across processing and storage layers.
Read how Architecting an Apache Iceberg Lakehouse explains reliability →
Infrastructure as Code, Patterns and Practices: With examples in Python and Terraform
The capability of a system to perform its required functions consistently under stated conditions for a stated period.
- How it is used
- Uses active-passive infrastructure to limit disruption during IaC changes and protect business-critical applications.
- How deep it goes
- Defines reliability and works through regional standby construction and load-balancer failover, but only previews active-active refactoring.
- Examples
- Uses Python configuration files, separate regional environments, and weighted load-balancer services as concrete examples.
- What you can do after
- Operate/configure an active-passive deployment and redirect all production traffic to its standby environment after failure.
Data-Oriented Programming in Java
Software reliability is the likelihood that software continues to perform required functions correctly under stated conditions.
- How it is used
- Uses isolated integration tests to preserve confidence that software works against real external services.
- How deep it goes
- Explains infrastructure provisioning, injection, random naming, namespaced cleanup, and test interference, but stops short of reliability measurement.
- Examples
- Demonstrates queues, buckets, randomized names, namespaces, cleanup helpers, and a service tested against ephemeral infrastructure.
- What you can do after
- Build isolated integration suites that exercise real services while reducing cross-test interference and cleanup failures.
Read how Data-Oriented Programming in Java explains reliability →
How Large Language Models Work
Reliability is the degree to which a system consistently produces dependable behavior and results under expected conditions.
- How it is used
- Frames reliability as a deployment criterion for deciding whether LLM applications can safely replace existing services.
- How deep it goes
- Explains accuracy failures, unexpected inputs, liability, and remediation needs, but stops short of specifying reliability measurement methods.
- Examples
- Uses FAQs, email tickets, phone support, customer retention, and trial deployments to illustrate reliability decisions.
- What you can do after
- Evaluate an LLM deployment by weighing inaccurate outputs, liability, remediation effort, and customer-impact risks.
Read how How Large Language Models Work explains reliability →
Think Distributed Systems
The ability of a system to continue providing its intended service correctly over time and under expected conditions.
- How it is used
- Frames reliability as the ability to remain responsive while failures affect a distributed system.
- How deep it goes
- Develops a definition, emergent-system model, responsiveness measures, and application-specific guarantees.
- Examples
- Uses a company analogy and component-interaction scenarios involving crashes, absence, load, and system behavior.
- What you can do after
- Evaluate reliability by defining service-level objectives, measuring error rates, and setting acceptable error budgets.
Amazon Web Services in Action, Third Edition: An in-depth guide to AWS
Reliability is the ability of a system to perform its intended functions consistently over time.
- How it is used
- Uses managed AWS services and deployment automation to keep web applications available while infrastructure or application versions change.
- How deep it goes
- Traces reliability from service defaults through rolling updates and bootstrap failures, then points toward AMIs as mitigation.
- Examples
- Demonstrates reliability through an Etherpad rolling update, bootstrap scripts, repository outages, AMIs, and cloud-init logs.
- What you can do after
- Operate/configure CloudFormation rolling updates and recognize when AMIs can reduce deployment-script failure risk.
Rearchitecting LLMs
Reliability is the consistency with which a model produces correct or constraint-compliant outputs across inputs.
- How it is used
- Measures whether model outputs satisfy a required schema while tracking general capabilities before and after specialization.
- How deep it goes
- Defines field-level JSON checks and compares 40-sample baseline results with post-training compliance, while treating small-sample results as directional.
- Examples
- Tests medical-information extraction across clean, abbreviated, implicit, irrelevant, and typo-containing notes.
- What you can do after
- Evaluate schema adherence with field-level JSON validation and compare specialized models against a prompted baseline.
Vibe Engineering
Test reliability is the consistency with which tests produce correct and repeatable results for the same software and conditions.
- How it is used
- Establishes trustworthy test results as the prerequisite for CI, agent iteration, and safe code delivery.
- How deep it goes
- Identifies flakiness, circular tests, and oracle gaming, but does not provide implementation procedures for correcting them.
- Examples
- Uses flaky tests, circular tests, and agent shortcutting as examples of misleading validation.
- What you can do after
- Diagnose misleading test feedback before optimizing CI speed or expanding automated development.
Also covered in
- Geometry for Programmers
- Machines that ThinkIn early access (MEAP)
- Grokking Statistics
- Architecture Modernization: Socio-technical alignment of software, strategy, and structure
- Elixir in Action, Third Edition
- Fast by DefaultIn early access (MEAP)
- Bootstrapping Microservices, Second Edition: With Docker, Kubernetes, GitHub Actions, and Terraform
- Timeless Algorithms, The Foundational IdeasIn early access (MEAP)
- Natural Language Processing in Action, Second Edition
- Microsoft Azure in Action
- Systems Programming with ZigIn early access (MEAP)
- The Cloud at Your Service: The when, how, and why of enterprise cloud computing
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- availabilityAvailability is the degree or proportion of time that a system or service is operational and accessible when required.
- robustnessRobustness is the ability of a system or model to maintain acceptable, reliable behavior despite variations, disturbances, faults, or unfavorable conditions.
- scalabilityScalability is the ability of a system or process to accommodate growth in workload, size, or demand while maintaining acceptable operation.
Related concepts
- inter-rater reliabilityincludes · Inter-rater reliability is the degree to which independent raters give consistent judgments or labels to the same items, subjects, or events.
- internal consistencyincludes · Internal consistency is the degree to which components of a measure or artifact produce mutually consistent, noncontradictory results or statements.
- MapReduceaddressed by · MapReduce is a data-processing model that maps operations over input elements or partitions and combines their intermediate results through reduction or aggregation.
- nonfunctional requirementpart of · A nonfunctional requirement specifies a system quality, constraint, or operating condition rather than a particular function or behavior.
- test-retest reliabilityincludes · Test-retest reliability is the consistency of a measure when the same individuals are assessed repeatedly over time while the measured construct remains stable.
Questions about reliability
What is reliability?
Reliability is the degree to which a system, process, or measurement performs or yields consistent, dependable results under specified conditions.
Which Manning books explain reliability?
Agent Design Patterns, Inside AI, Apache Kafka in Action, Grokking Software Architecture, Effective Data Analysis, AI Governance, The Coder Cafe, Building Reliable AI Systems and 13 more.
What should I know before learning reliability?
Manning books that teach reliability usually introduce language model, availability, robustness and scalability first.
What is the difference between reliability and benchmark capability?
Reliability is the degree to which a system, process, or measurement performs or yields consistent, dependable results under specified conditions. Performance demonstrated by an AI system on standardized benchmark tasks.
What kinds of reliability are there?
Kinds of reliability include inter-rater reliability, internal consistency and test-retest reliability.
What is reliability part of?
Reliability is part of nonfunctional requirement. A nonfunctional requirement specifies a system quality, constraint, or operating condition rather than a particular function or behavior.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




















