Azure Content Safety

As explained in Generative AI in Action

Azure Content Safety is a Microsoft Azure service for analyzing and moderating potentially harmful, ungrounded, or protected content in AI inputs and outputs.

Azure Content Safety is an Azure AI service that analyzes text and images in prompts and model outputs, assigns severity scores, and supports enterprise content moderation. section 13.5.1

Its controls include:

• Configurable harm-category filters and blocklists for organizational, regulatory, and industry requirements. section 13.5.1
• Prompt Shields for detecting prompt-injection attacks in user prompts and documents. section 13.5.1
• Groundedness detection for identifying responses that are not supported by supplied factual sources. section 13.5.1
• Protected material detection for recognizing potentially copyrighted content. section 13.5.1

Checking a prompt and documents

A Python client can send a user prompt and document list to the text:shieldPrompt endpoint, then inspect attackDetected results before continuing model processing.

Figure 13.13 Azure AI Content Safety

Architecture view showing Azure AI Content Safety in the moderation workflow. section 13.5.1

How Generative AI in Action covers it

How deep it goes

Explains API-based analysis, severity scoring, filters, blocklists, and specialized detectors without detailing model internals.

Examples

Demonstrates Python requests for prompt shields, groundedness detection, and protected material detection.

What it assumes

Assumes familiarity with generative-AI prompts, outputs, documents, and authenticated HTTP requests.

What you can do after

Apply it via Azure APIs to inspect content, prioritize moderation, and choose actions using returned detection results.

How it connects

Connects content moderation with prompt injection, blocklists, groundedness detection, and copyright protection.

liveBookmanning.com

Read Generative AI in Action on liveBook Get the book at manning.com

Related concepts

• Azure Content Moderatoralternative to · Azure Content Moderator is Microsoft's cloud service for automatically detecting potentially inappropriate content in text, images, and video.
• prompt shieldhas part · A prompt shield is a content-safety mechanism that detects, blocks, or neutralizes malicious, manipulative, or unsafe instructions in prompts.
• protected material detectionhas part · Protected material detection is the identification of generated content that matches copyrighted or otherwise owned source material.
• Microsoft Azure Foundryuses · Microsoft Azure AI Foundry is Microsoft's Azure platform for accessing AI models and developing, customizing, deploying, and orchestrating AI applications and agents.

This explainer was generated by Manning's knowledge graph from the text of Generative AI in Action. The section links point to the parts of the book it draws on.