Many-shot jailbreaking

As explained in Generative AI in Action

Many-shot jailbreaking is a prompt attack that places numerous fabricated examples of an AI answering harmful questions before a final malicious request.

Many-shot jailbreaking is a prompt-injection technique that places a fabricated human–assistant dialogue, containing harmful answers, before a final malicious request. Larger context windows make it possible to include many such examples and pressure the model toward unsafe behavior. section 13.2.1

Figure 13.8 Many-shot jailbreaking

Inspect the supplied illustration of a fabricated dialogue followed by a final question. section 13.2.1

Apply defensive controls around models exposed to this attack:

• Validate and sanitize model responses before passing them to backend functions. section 13.2.1
• Encode model output returned to users to reduce unintended code execution, including JavaScript execution. section 13.2.1

How Generative AI in Action covers it

How deep it goes

Explains the prompt structure and context-window rationale, but does not detail model internals or attack construction.

Examples

Uses a fabricated harmful dialogue, a final prohibited request, and a Gemini Pro 1.5 low-safety demonstration.

What it assumes

Builds from prompt injection and assumes an LLM with a sufficiently large context window.

What you can do after

Diagnose/debug with it by recognizing fabricated-example attacks and adding validation, sanitization, and output-encoding controls.

How it connects

Connects prompt injection with context windows, response validation, sanitization, and output encoding.

liveBookmanning.com

Read Generative AI in Action on liveBook Get the book at manning.com

Related concepts

• jailbreakingkind of · Jailbreaking is an adversarial input or interaction strategy intended to make an AI system bypass its instructions, safeguards, or behavioral restrictions.
• context windowuses · A context window is the bounded sequence of tokens or other input information a model can access when processing or generating an output.

This explainer was generated by Manning's knowledge graph from the text of Generative AI in Action. The section links point to the parts of the book it draws on.