Adversarial input

As explained in Deep Learning with R, Third Edition

An adversarial example is an input deliberately or naturally modified in a small way that causes a machine-learning model to produce an incorrect output.

An adversarial input is a sample changed in a small, often deliberate way to make a machine-learning model produce the wrong output. It exposes a gap between model sensitivity and human perception. section 19.1.2

A panda classified as a gibbon

Gradient ascent can slightly modify a panda image to maximize the model’s prediction for “gibbon.” The altered image may look unchanged to people while overturning the model’s classification. section 19.1.2

Related sensitivity appears in several forms:

• Image models can be tricked by carefully designed pixel changes. section 19.1.2
• LLMs can fail when names, places, numbers, or code variable names change slightly. section 19.1.2

Figure 19.3 An adversarial example: imperceptible changes in an image can upend a model’s classification of the image.

Use the figure to inspect how imperceptible image changes can reverse a model’s classification. section 19.1.2

How Deep Learning with R, Third Edition covers it

How deep it goes

Explains the idea through gradient ascent and examples, but does not develop attack algorithms or defenses.

Examples

Uses a panda-to-gibbon image change and an Alice in Wonderland riddle with altered numerical values.

What it assumes

Assumes familiarity with gradient ascent and convolutional-network filter visualization.

What you can do after

Evaluate how apparently harmless input changes can expose mismatches between model predictions and human judgment.

How it connects

Connects adversarial examples with gradient ascent, filter visualization, image classification, LLMs, and prompt engineering.

liveBookmanning.com

Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  LLMs in ProductionThe study and use of inputs or models designed to manipulate, evade, or expose weaknesses in machine-learning systems.
• 

  liveBookmanning.com
  Sutskever's ListAn adversarial example is an input deliberately perturbed in a way that is often imperceptible to humans but causes a model to make an incorrect prediction.
• 

  liveBookmanning.com
  Generative AI in ActionAn adversarial example is an input modified or crafted to cause a machine-learning model to produce an incorrect or undesirable output.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionInputs intentionally or effectively perturbed to cause a model to make a wrong prediction despite minimal change to the original example.

Compare how 5 books explain adversarial input →

Related concepts

• red-teamingused by · Red-teaming is an adversarial assessment practice that deliberately simulates attacks or uses challenging inputs to identify a system’s vulnerabilities, failures, or unsafe behavior.
• TextAttackused by · TextAttack is an open-source Python framework for generating adversarial examples, augmenting data, training NLP models, and evaluating their robustness.
• poisoning attackimplemented by · A poisoning attack deliberately corrupts data, a learning process, or model artifacts to compromise a machine-learning system’s behavior or outputs.
• robustnessmeasures · Robustness is the ability of a system or model to maintain acceptable behavior or performance despite variations, disturbances, faults, or challenging conditions.
• prompt injectionincludes · Prompt injection is an attack that introduces instructions through input or context to manipulate an AI system’s behavior or override its intended instructions.
• predicted valueproduces · A predicted value is an estimated outcome or output that a model produces for given input values.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.