liveBook · Manning

LLM Guard

As explained in Transformers in Action

LLM Guard is a security toolkit that provides configurable scanning and protection for inputs and outputs of large language model systems.

LLM Guard is a configurable security toolkit for identifying, editing, and cleansing prompts and model responses. It provides real-time protection for production LLM applications, helping enforce application-specific safety, security, and regulatory requirements. section 10.4

Protection is organized around input and output scanners. Input scanners can block unwanted substrings, competitor names, selected topics, toxic language, and prompt injection attempts. Output scanners check factual consistency, malicious URLs, and sensitive entities such as people and email addresses, with optional redaction. section 10.4

Financial-services chatbot

A customer-onboarding chatbot can restrict conversations to product information, risk-tolerance assessment, and account setup. A safeguard function scans the prompt, records validity and scores, blocks failed inputs with a fixed response, and otherwise continues toward model inference and output scanning. Example tests compare a prompt mentioning Fidelity, an ordinary question about dogs, toxic language, and a request for personal data with and without safeguards. section 10.4

How Transformers in Action covers it

How deep it goes
Demonstrates scanner configuration, threshold selection, inference integration, blocking, result recording, and guarded test cases.
Examples
Uses a financial chatbot, competitor-name filtering, banned topics, toxic prompts, personal-information requests, and guarded versus unguarded inference.
What it assumes
Assumes a loaded Llama 3 model, tokenizer, GPU-backed inference setup, and familiarity with Python model-generation code.
What you can do after
Operate/configure LLM Guard scanners, apply them around inference, inspect scores, block invalid prompts, and redact selected output entities.
How it connects
Connects input validation with prompt injection, toxicity, topic filtering, privacy protection, factual consistency, and malicious-URL detection.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Transformers in Action. The section links point to the parts of the book it draws on.