Robustness
Robustness is the ability of a system or model to maintain acceptable, reliable behavior despite variations, disturbances, faults, or unfavorable conditions.
Machine Learning System Design
A software quality property of continuing to operate and produce some result despite errors or abnormal conditions, even when the result may be inaccurate.
- How it is used
- Guides error-handling choices and evaluation of dependable ML behavior in light of product risks.
- How deep it goes
- Defines the error-handling tradeoff and model consistency, then outlines perturbation tests and feature-selection risks.
- Examples
- Contrasts an entertainment filter with a self-driving car and uses lighting changes and synonym substitutions as model perturbations.
- What you can do after
- Evaluate/understand robustness tradeoffs and check whether model outputs behave as expected under relevant perturbations.
Read how Machine Learning System Design explains robustness →
AI Model Evaluation
Robustness is the ability of a model to maintain acceptable behavior across disturbances, unusual conditions, or changes in inputs.
- How it is used
- Diagnoses whether models remain useful under noisy inputs and rare production scenarios.
- How deep it goes
- Details input perturbations, ranked-output metrics, and a shadow-testing procedure with per-label monitoring and promotion criteria.
- Examples
- Uses perturbed movie recommendations and noisy clothing-search queries to reveal changes in output and relevance.
- What you can do after
- Diagnose brittle behavior by comparing clean and perturbed outputs and monitoring labeled edge cases in shadow tests.
Delivering Machine Learning Projects: From design to deployment
Robustness is the ability of a system to maintain acceptable performance across specified circumstances and perturbations.
- How it is used
- Uses robustness testing as evidence for acceptance to service and safer deployment decisions.
- How deep it goes
- Defines characterization and testing across circumstances, stopping short of specifying technical test methods.
- Examples
- Uses generative art, generative text, and facial recognition to compare intended-function and failure harms.
- What you can do after
- Evaluate/understand system behavior under specified circumstances and inform acceptance-to-service and mitigation decisions.
Read how Delivering Machine Learning Projects: From design to deployment explains robustness →
Sutskever's List
Robustness is a model's ability to maintain reliable behavior under perturbations, distribution changes, and atypical inputs.
- How it is used
- Tests whether benchmark success survives adversarial inputs and real-world distribution shifts.
- How deep it goes
- Identifies adversarial and deployment failures but does not develop robustness mechanisms or detailed evaluation procedures.
- Examples
- Uses ImageNet scores, imperceptible perturbations, and Google Photos’ gorilla-labeling incident.
- What you can do after
- Evaluate claims of human-level performance against adversarial inputs and atypical real-world cases.
Statistics Every Programmer Needs
Model robustness is the ability of a predictive model to maintain reliable performance across variation in data or conditions.
- How it is used
- Monitors process stability, detects deviations, and supports corrective action through visual quality-control charts.
- How deep it goes
- Explains control limits, means, standard deviations, chart selection, and implementation, but does not develop predictive-model robustness.
- Examples
- Uses orange-juice can defects, range variation, defects per unit, and synthetic intervals between rare defects.
- What you can do after
- Diagnose process instability with control charts and interpret points, trends, limits, and variation for corrective action.
Read how Statistics Every Programmer Needs explains robustness →
AI Governance
Robustness measures how consistently a model behaves appropriately when its inputs vary or differ from familiar contexts.
- How it is used
- Keeps model decisions reasonable and defensible when inputs or context vary in everyday use.
- How deep it goes
- Defines robustness, details phrasing and speaker sensitivity, and illustrates testing with perturbed queries and adversarial tools.
- Examples
- Uses a dirt-corrupted insurance scan, car-versus-vehicle paraphrases, dialect changes, and adversarial testing.
- What you can do after
- Evaluate robustness by testing paraphrased queries and adversarial cases with tools such as Giskard or ART.
Causal AI
Robustness is the property of maintaining acceptable model behavior when deployment conditions differ from training conditions.
- How it is used
- Frames robustness as a deployment requirement and a sensitivity check for causal estimates.
- How deep it goes
- Explains overfitting, underspecification, data drift, and confounding sensitivity, but does not develop model-training internals.
- Examples
- Uses elevation and temperature across cities, plus a DoWhy heatmap that varies unobserved-confounder influence.
- What you can do after
- Diagnose causal-estimate sensitivity with DoWhy by introducing an unobserved common cause and inspecting changed effect estimates.
The AI Pocket Book
- How it is used
- Improves software reliability by making validation central to development rather than a secondary task.
- How deep it goes
- Explains robustness procedurally through edge-case analysis, unit tests, persistent checks, and testing tradeoffs.
- Examples
- Uses edge-case tests, print statements, small code units, and validation-driven team workflows as concrete examples.
- What you can do after
- Apply it via test-driven development by designing tests for failure cases and keeping valuable checks in the codebase.
Evaluating AI Systems
The ability of a system to maintain suitable behavior under varied, unclear, or challenging inputs and conditions.
- How it is used
- Checks whether evaluation metrics remain trustworthy under small changes in systems, data, or environments.
- How deep it goes
- Defines metric robustness, contrasts fragile metrics, and outlines perturbation, cross-environment, and resampling checks.
- Examples
- Uses BLEU, perplexity, human ratings, and medical-chatbot input variations to illustrate metric sensitivity.
- What you can do after
- Evaluate metric stability by perturbing inputs, comparing environments, and resampling evaluation data.
Machines that Think
Robustness is the ability of a system to maintain acceptable behavior under variation, disturbance, or unfavorable conditions.
- How it is used
- Frames robustness as both tolerance to structural loss in neural networks and reliability supported by safeguards around generative systems.
- How deep it goes
- Explains pruning results and operational safeguards, while noting that safeguards cannot eliminate probabilistic variability.
- Examples
- Uses weight pruning, sparse subnetworks, structured prompts, output review, and stress-testing as concrete illustrations.
- What you can do after
- Evaluate robustness by distinguishing tolerance to structural loss from safeguards that manage, but cannot eliminate, output variability.
Generative AI in Action
Robustness is the ability of a machine-learning model to maintain acceptable behavior when inputs vary, are perturbed, or include difficult cases.
- How it is used
- Tests LLM resilience against adversarial, varied, and edge-case inputs during red-teaming and enterprise evaluation.
- How deep it goes
- Connects robustness to adversarial generation, data augmentation, and edge-case checks, without detailing metrics or remediation procedures.
- Examples
- Uses TextAttack, NL-Augmenter, edge-case inputs, and a SLURM pipeline evaluating Llama 2, Baichuan2, Mistral, and other models.
- What you can do after
- Diagnose robustness risks by generating adversarial inputs and comparing model behavior across varied and difficult cases.
Retrieval Augmented Generation, The Foundational Ideas
Noise robustness is the ability of a system to maintain performance when inputs or retrieved evidence contain irrelevant or corrupted information.
- How it is used
- Improves the reader side of RAG when irrelevant retrieved passages degrade answer quality.
- How deep it goes
- Explains reader noise sensitivity, mixed-passage fine-tuning, adapters, and k-range effects, but omits detailed training implementation.
- Examples
- Uses RAFT distractor mixtures, domain QA adapters, and RAGAS-guided failure diagnosis.
- What you can do after
- Diagnose RAG failures with RAGAS, then select reader fine-tuning, reranking, or context-compression interventions.
Read how Retrieval Augmented Generation, The Foundational Ideas explains robustness →
Also covered in
- Agent Design PatternsIn early access (MEAP)
- Inside Deep Learning: Math, Algorithms, Models
- Grokking AI Algorithms, Second Edition
- The Coder CafeIn early access (MEAP)
- Transformers in Action
- Architecture Modernization: Socio-technical alignment of software, strategy, and structure
- The Art of Code
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- inputAn input is information, a value, or an action supplied to a system, program, or computation for processing or to influence its behavior.
- hallucinationA hallucination is model-generated content that is false, fabricated, or unsupported by relevant evidence but presented as plausible or factual.
Related concepts
- adversarial inputmeasured by · An input crafted, selected, or modified to exploit a system’s weaknesses or induce incorrect, unsafe, or otherwise unintended behavior.
- accuracycontrasts with · Accuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- perturbation testmeasured by · A perturbation test evaluates a system by applying a controlled change to its inputs, components, or conditions and observing the response.
- property-based testingused by · Property-based testing is a software-testing technique that checks whether specified general properties hold across varied, often automatically generated, inputs.
- adversarial robustnessincludes · Adversarial robustness is the ability of a model or system to maintain acceptable, reliable behavior when exposed to deliberately crafted or hostile inputs.
Questions about robustness
What is robustness?
Robustness is the ability of a system or model to maintain acceptable, reliable behavior despite variations, disturbances, faults, or unfavorable conditions.
Which Manning books explain robustness?
Machine Learning System Design, AI Model Evaluation, Delivering Machine Learning Projects: From design to deployment, Sutskever's List, Statistics Every Programmer Needs, AI Governance, Causal AI, The AI Pocket Book and 4 more.
What should I know before learning robustness?
Manning books that teach robustness usually introduce language model, accuracy, input and hallucination first.
What is the difference between robustness and accuracy?
Robustness is the ability of a system or model to maintain acceptable, reliable behavior despite variations, disturbances, faults, or unfavorable conditions. Accuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
What kinds of robustness are there?
Kinds of robustness include adversarial robustness.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.











