How deeply does LLMs in Production cover poisoning attack?

Defines the attack, illustrates its effects, and gives approximate image counts, but does not detail mitigation procedures.

Poisoning attack

As explained in LLMs in Production

Data poisoning is an attack that inserts malicious or manipulated examples into training data to alter a model's behavior.

A poisoning attack inserts malicious training data to change a model’s behavior. Poorly vetted datasets make large language models especially exposed. section 6.3.6

Small changes, large effects

As few as 50 to 300 poisoned images can make image models generate cats for dog prompts or cows for car prompts. A poisoned language-model dataset could teach the model to ignore security protocols when a code word or hash appears. section 6.3.6

Recognize the practical implications:

Treat training-data vetting and cleaning as security controls, not merely quality checks. section 6.3.6
Distinguish poisoning attacks from prompt injection, which manipulates model inputs rather than its training dataset. section 6.3.6

How LLMs in Production covers it

liveBook manning.com

Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Introduction to Generative AI, Second EditionData poisoning is an attack that inserts malicious or misleading examples into training data so a model learns harmful or exploitable behavior.
liveBook manning.com

The Art of AI Product DevelopmentAn attack that deliberately corrupts or manipulates data used to train or operate a machine-learning system in order to compromise its behavior or outputs.

Compare how 3 books explain poisoning attack →

Related concepts

training datasetuses · A collection of examples or observations supplied to a learning procedure to fit or adapt a model or learn its patterns or parameters.
validationaddressed by · The process of checking data, behavior, systems, models, or results against defined criteria to assess their correctness, suitability, or acceptability.
backdoor attackproduces · A backdoor attack uses or introduces a concealed mechanism that bypasses intended controls to enable unauthorized access or trigger behavior under specific conditions.
adversarial inputimplements · An input crafted, selected, or modified to exploit a system’s weaknesses or induce incorrect, unsafe, or otherwise unintended behavior.

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.