Guard model

As explained in Transformers in Action

A guard model is a compact classifier that labels prompts or outputs for safety-related properties such as benignness or maliciousness.

A guard model is a compact classifier that labels prompts or outputs as benign or malicious, forming a protective layer around larger language models. It filters jailbreak attempts, prompt injections, policy violations, hallucinations, or unsafe generations before they reach users. section 7.6 section 7.1

Guard models provide a practical safety layer with several design properties:

• Use encoder-based classifiers such as Llama Guard or Prompt Guard 2 to return binary or categorical labels quickly, without full text generation. section 7.6
• Deploy them in real-time pipelines where larger generative models handle reasoning while the guard model performs fast filtering or verification. section 7.1 section 7.6
• Curate training data and calibrate decisions carefully because broad or noisy datasets can overblock benign prompts. section 7.6

Distinguishing harmful and benign prompts

A classifier may flag a prompt containing “attack” even when it discusses cybersecurity defense. This illustrates the trade-off between blocking unsafe requests and preserving legitimate use. section 7.6

Figure 7.1 SLMs as auxiliary components in a larger AI pipeline

Use the pipeline figure to trace how a small guard model operates as an auxiliary protective component around a larger generative system. section 7.1

How Transformers in Action covers it

How deep it goes

Explains classifier labels, deployment trade-offs, and calibration risks, but stops short of model-training procedures.

Examples

Uses Llama Guard, Prompt Guard 2, and cybersecurity-defense prompts to illustrate deployment and false positives.

What it assumes

Assumes familiarity with larger language models, prompts, outputs, and classifier-based labeling.

What you can do after

Operate a lightweight real-time filter that labels potentially harmful inputs or outputs before delivery.

How it connects

Connects guard models to small language models, encoder classifiers, prompt injection, jailbreak attacks, and generative defenses.

liveBookmanning.com

Read Transformers in Action on liveBook Get the book at manning.com

Related concepts

• small language modelincludes · A small language model is a comparatively compact language model with limited parameters or capacity and typically lower computational and storage requirements than larger models.
• prompt injectionaddresses · Prompt injection is an attack that introduces instructions through input or context to manipulate an AI system’s behavior or override its intended instructions.
• language modelpart of · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.

This explainer was generated by Manning's knowledge graph from the text of Transformers in Action. The section links point to the parts of the book it draws on.