Adversarial input
An input crafted, selected, or modified to exploit a system’s weaknesses or induce incorrect, unsafe, or otherwise unintended behavior.
LLMs in Production
The study and use of inputs or models designed to manipulate, evade, or expose weaknesses in machine-learning systems.
- How it is used
- Frames adversarial inputs as security risks for deployed language-model services.
- How deep it goes
- Explains attack mechanisms and safeguard trade-offs qualitatively, without an implementation walkthrough.
- Examples
- Uses a misclassified frog image, a fabricated cola recipe, and generated-text detection to illustrate adversarial behavior.
- What you can do after
- Evaluate and understand attack risks, suspicious outputs, and language-filtering trade-offs.
Sutskever's List
An adversarial example is an input deliberately perturbed in a way that is often imperceptible to humans but causes a model to make an incorrect prediction.
- How it is used
- Uses adversarial examples to challenge claims that benchmark success reflects human-like understanding or general robustness.
- How deep it goes
- Defines the failure and its significance, but does not explain how perturbations are constructed.
- Examples
- Uses imperceptible input perturbations and a separate Google Photos mislabeling incident to illustrate brittleness.
- What you can do after
- Evaluate why benchmark performance alone does not establish robust visual understanding.
Generative AI in Action
An adversarial example is an input modified or crafted to cause a machine-learning model to produce an incorrect or undesirable output.
- How it is used
- Tests LLM resilience against harmful, bad, and edge-case inputs during enterprise red-teaming.
- How deep it goes
- Defines adversarial attacks and names generation tools, but does not explain attack algorithms or mathematical internals.
- Examples
- Uses TextAttack and a SLURM pipeline to run attacks and evaluate completions across Llama 2, Baichuan2, Mistral, and other models.
- What you can do after
- Apply it via TextAttack and attack pipelines to evaluate model resilience against problematic inputs.
Read how Generative AI in Action explains adversarial input →
Deep Learning with R, Third Edition
An adversarial example is an input deliberately or naturally modified in a small way that causes a machine-learning model to produce an incorrect output.
- How it is used
- Diagnoses brittleness in image models and language models by showing how minor input changes can cause major errors.
- How deep it goes
- Explains the idea through gradient ascent and examples, but does not develop attack algorithms or defenses.
- Examples
- Uses a panda-to-gibbon image change and an Alice in Wonderland riddle with altered numerical values.
- What you can do after
- Evaluate how apparently harmless input changes can expose mismatches between model predictions and human judgment.
Read how Deep Learning with R, Third Edition explains adversarial input →
Deep Learning with Python, Third Edition
Inputs intentionally or effectively perturbed to cause a model to make a wrong prediction despite minimal change to the original example.
- How it is used
- Exposes the brittleness of deep learning models and the mismatch between their input-output mappings and human perception.
- How deep it goes
- Explains the gradient-ascent mechanism and a panda-to-gibbon example, without detailing attack algorithms or defenses.
- Examples
- Uses a panda image modified toward the gibbon class and a figure illustrating imperceptible classification-changing perturbations.
- What you can do after
- Evaluate/understand how small input perturbations can cause confident model misclassification.
Read how Deep Learning with Python, Third Edition explains adversarial input →
Also covered in
- AI GovernanceIn early access (MEAP)
- Transformers in Action
- Digital Twins in Action
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- convolutional neural networkA convolutional neural network is a neural-network architecture that processes structured inputs with convolutional operations, typically using learned local filters to capture patterns.
- fine-tuningFine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
Related concepts
- red-teamingused by · Red-teaming is an adversarial assessment practice that deliberately simulates attacks or uses challenging inputs to identify a system’s vulnerabilities, failures, or unsafe behavior.
- prompt injectionincludes · Prompt injection is an attack that introduces instructions through input or context to manipulate an AI system’s behavior or override its intended instructions.
- poisoning attackimplemented by · A poisoning attack deliberately corrupts data, a learning process, or model artifacts to compromise a machine-learning system’s behavior or outputs.
- robustnessmeasures · Robustness is the ability of a system or model to maintain acceptable, reliable behavior despite variations, disturbances, faults, or unfavorable conditions.
- TextAttackused by · TextAttack is an open-source Python framework for generating adversarial examples, augmenting data, training NLP models, and evaluating their robustness.
- inputkind of · An input is information, a value, or an action supplied to a system, program, or computation for processing or to influence its behavior.
Questions about adversarial input
What is adversarial input?
An input crafted, selected, or modified to exploit a system’s weaknesses or induce incorrect, unsafe, or otherwise unintended behavior.
Which Manning books explain adversarial input?
LLMs in Production, Sutskever's List, Generative AI in Action, Deep Learning with R, Third Edition and Deep Learning with Python, Third Edition.
What should I know before learning adversarial input?
Manning books that teach adversarial input usually introduce language model, accuracy, convolutional neural network and fine-tuning first.
What kinds of adversarial input are there?
Kinds of adversarial input include prompt injection.
What is adversarial input a kind of?
Adversarial input is a kind of input. An input is information, a value, or an action supplied to a system, program, or computation for processing or to influence its behavior.
What is adversarial input also called?
Adversarial input is also called adversarial example.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




