How deeply does Generative AI in Action cover adversarial input?

Defines adversarial attacks and names generation tools, but does not explain attack algorithms or mathematical internals.

Adversarial input

As explained in Generative AI in Action

An adversarial example is an input modified or crafted to cause a machine-learning model to produce an incorrect or undesirable output.

An adversarial input is crafted or modified to test whether a machine-learning model produces an incorrect, unsafe, or otherwise undesirable output. In enterprise GenAI, adversarial inputs support robustness and security testing rather than ordinary user interaction. section 13.0 section 13.4.2

Use adversarial inputs as part of red-teaming:

Generate adversarial inputs with TextAttack, a Python framework supporting adversarial attacks, examples, and NLP data augmentation. section 13.4.2
Evaluate how consistently an LLM handles bad inputs, edge cases, and other conditions that may produce unexpected outputs. section 13.4.2
Combine attack testing with model evaluation, fairness audits, explainability, and data augmentation to investigate broader risks. section 13.4.2

# Run all compatible attacks against Llama 2 7B Chat using a SLURM cluster
python ./scripts/run_pipeline.py --methods all --models 
llama2_7b --step all --mode slurm

# Generate and evaluate completions using a SLURM cluster
python ./scripts/run_pipeline.py --methods all --models 
baichuan2_7b,mistral_7b,llama2_70b --step 2_and_3 --mode slurm

The supplied pipeline listing shows how compatible attacks and completion evaluation can run across several language models on a SLURM cluster. section 13.4.2

How Generative AI in Action covers it

liveBook manning.com

Read Generative AI in Action on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

LLMs in ProductionThe study and use of inputs or models designed to manipulate, evade, or expose weaknesses in machine-learning systems.
liveBook manning.com

Sutskever's ListAn adversarial example is an input deliberately perturbed in a way that is often imperceptible to humans but causes a model to make an incorrect prediction.
liveBook manning.com

Deep Learning with R, Third EditionAn adversarial example is an input deliberately or naturally modified in a small way that causes a machine-learning model to produce an incorrect output.
liveBook manning.com

Deep Learning with Python, Third EditionInputs intentionally or effectively perturbed to cause a model to make a wrong prediction despite minimal change to the original example.

Compare how 5 books explain adversarial input →

Related concepts

red-teamingused by · Red-teaming is an adversarial assessment practice that deliberately simulates attacks or uses challenging inputs to identify a system’s vulnerabilities, failures, or unsafe behavior.
TextAttackused by · TextAttack is an open-source Python framework for generating adversarial examples, augmenting data, training NLP models, and evaluating their robustness.
poisoning attackimplemented by · A poisoning attack deliberately corrupts data, a learning process, or model artifacts to compromise a machine-learning system’s behavior or outputs.
robustnessmeasures · Robustness is the ability of a system or model to maintain acceptable behavior or performance despite variations, disturbances, faults, or challenging conditions.
prompt injectionincludes · Prompt injection is an attack that introduces instructions through input or context to manipulate an AI system’s behavior or override its intended instructions.
predicted valueproduces · A predicted value is an estimated outcome or output that a model produces for given input values.

This explainer was generated by Manning's knowledge graph from the text of Generative AI in Action. The section links point to the parts of the book it draws on.