liveBook · Manning

Chapter 6: Tools and Guardrails: Enabling safe, managed AI actions

What you will learn in chapter 6 of Designing AI Systems

This introduction explains why AI actions need platform-managed tools and guardrails, and outlines the services that provide them.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction explains why AI actions need platform-managed tools and guardrails, and outlines the services that provide them.

Key terms: agentic AI, blast radius, function calling, Tool Service, Tool Registry, Credential Store, Execution Layer, tool definitions, Guardrails Service, Guardrails System

6.1 Reframing tools as platform-managed capabilities

This section reframes AI tools from application-embedded functions into independently managed platform capabilities that can be shared, governed, and secured centrally.

Key terms: platform-managed capabilities, registration, discovery, credential management, execution, identity, unique name, version history, owner, access policy

6.1.1 The tool service contract

This section defines the Tool Service gRPC contract and explains how its lifecycle operations let applications safely register, discover, validate, and execute platform-managed tools.

Key terms: Tool Service, gRPC contract, registration, discovery, execution, validation, Register, RegisterMcpServer, Discover, tool registry

6.1.2 Tool definitions beyond function schemas

This section expands tool definitions from model-facing function schemas into platform-managed entities with identity, schema, and operational metadata for safe execution and governance.

Key terms: ToolDefinition, identity fields, schema fields, operational metadata, function calling, name, version, owner, description, parameter definitions

6.1.3 The SDK interface

This subsection shows how the SDK hides tool-management complexity by letting applications register tools once and discover model-ready tool definitions for reuse.

Key terms: SDK, tool registration, tool service contract, definition schema, behavioral metadata, managed capabilities, authorized application, tool registry, build_model_tools, function calling

6.2 The tool registry: namespacing and discovery

This section explains how a tool registry uses hierarchical namespaces to make platform capabilities discoverable, access-controlled, and owned by distributed teams.

Key terms: tool registry, authoritative catalog, searchable inventory, platform-managed capabilities, namespacing, hierarchical naming, fully qualified name, business domain, capability area, specific operation

6.2.1 Capability-based discovery

Capability-based discovery lets applications and AI agents find permitted tools by functionality across namespaces instead of relying on hardcoded namespace knowledge.

Key terms: capability-based discovery, namespaces, DiscoverToolsRequest, authentication context, namespace access control, application identity, tool registrations, capabilities, tags, tool descriptions

6.2.2 Version control

This section explains how a tool registry uses semantic versioning, version constraints, compatibility enforcement, and structured deprecation to let tools evolve without breaking applications.

Key terms: tool registry, version history, version constraints, pinning, gradual migration, semantic versioning, major version, minor version, patch version, compatibility rules

6.3 Tool execution: adapters, credentials and reliability

This section introduces tool execution as a platform concern that hides external connectivity, authentication, failure handling, and delayed responses from application developers.

Key terms: tool execution, platform, external systems, authenticating with APIs, failures, responses

6.3.1 The adapter pattern

This section explains how adapters hide complex external APIs behind stable, AI-friendly tool interfaces.

Key terms: adapter pattern, adapter, Tool Service, tool contract, tool interface, request translation, authentication, response transformation, tool arguments, API format

6.3.2 Credential isolation

This section explains how the platform isolates credentials from application code and injects them securely during tool execution.

Key terms: credential isolation, secure boundary, Tool Service, SDK, credential_ref, credential store, out of band, admin console, CLI, infrastructure-as-code pipeline

6.3.3 The credential store interface

This section defines a credential-store abstraction that delegates secret management to existing systems while keeping tool credentials isolated, authorized, and rotatable without redeployment.

Key terms: CredentialStore, secrets manager, HashiCorp Vault, AWS Secrets Manager, access controls, authorization checks, Tool Service, credential_store.retrieve(), outbound request, SDK

6.3.4 Synchronous and asynchronous execution

This section explains how the Tool Service selects synchronous or asynchronous execution for external tools and centrally applies reliability policies so AI conversations remain responsive.

Key terms: synchronous execution, asynchronous execution, Tool Service, task identifier, AI application, conversation, timeout policy, retry policy, tool definition, expected latency

6.4 Interoperability: the Model Context Protocol (MCP)

This section introduces MCP as an open standard that replaces the N×M burden of custom third-party tool integrations with a shared interoperability protocol.

Key terms: N×M integration problem, custom integrations, bespoke connectors, third-party tool providers, Model Context Protocol, MCP, open standard, standard interface, MCP-compatible, discover

6.4.1 From NxM to N+M, and then to M

This section compares direct integrations, MCP standardization, and a platform Tool Service to show how integration cost falls from N×M to N+M to M.

Key terms: N×M, N+M, M, MCP, Tool Service, organizational multiplexing, registry, credential isolation, policies, tool wrappers

6.4.2 The architecture of MCP: hosts, clients and servers

This section defines MCP’s host, client, and server roles and shows how they connect within the platform’s Tool Service.

Key terms: MCP server, MCP client, host, Tool Service, subprocess, cloud infrastructure, deployment model, dedicated connection, tool providers, open-source SDKs

6.4.3 The three MCP primitives

This section defines MCP's three capability primitives—tools, resources, and prompts—and explains how each maps to platform services.

Key terms: tools, resources, prompts, MCP server, JSON Schema, tools/list, tools/call, Tool Service, context management, prompt template management

6.4.4 What MCP does not cover

This section defines MCP’s limits as a wire protocol and explains why the platform must supply the security, governance, credential, and audit controls around it.

Key terms: MCP, wire protocol, tool schemas, invocations, credential management, static API keys, personal access tokens, environment variables, rotation, scoping

6.4.5 Integrating MCP with the platform

This section shows how the Tool Service integrates external MCP servers as managed platform tool sources while preserving a uniform application interface and platform controls.

Key terms: Tool Service, MCP client, MCP servers, platform tools, platform namespace, MCP transport, SDK, discovery API, execution call, native adapter

6.5 Tool execution safeguards

This section introduces execution isolation as the safeguard that bounds each tool invocation so a catastrophic failure cannot spread through the platform.

Key terms: Tool Service, tool invocation, execution isolation, bounded resources, adapter, infinite loop, malformed response, unbounded memory allocation, multi-gigabyte payload

6.5.1 Resource limits

This section explains how the Tool Service enforces per-invocation execution limits, isolates tool adapters, and turns limit violations into controlled model-visible failures.

Key terms: container-based isolation, tool adapter, execution limits, ToolDefinition, ToolBehavior, RateLimits, structured error, raw exception, metrics, is_idempotent

6.5.2 Circuit breakers

This section explains how circuit breakers prevent repeatedly failing tools from burdening unhealthy external systems while allowing controlled recovery checks.

Key terms: circuit breaker, closed, open, half-open, failure threshold, recovery timeout, tripped circuit breaker, tool returning an error, discovery results, alternatives

6.6 Guardrails: not just filters, but execution policies

This section reframes guardrails as platform-enforced execution policies and locates their enforcement across five points in an AI interaction flow.

Key terms: content guardrails, execution policies, business logic, content filters, Guardrails Service, AddPolicy, ValidateInput, CheckPolicy, FilterOutput, ReportViolation

6.7 Guardrails in practice: input, output and behavioral policies

This section introduces input, output, and behavioral guardrails by identifying what each category protects and where it acts.

Key terms: input guardrails, output guardrails, behavioral guardrails, malicious requests, invalid requests, harmful responses

6.7.1 Input guardrails: protecting the system

Input guardrails inspect messages and tool-call arguments before model or external-system access to block attacks, out-of-scope requests, and invalid actions.

Key terms: input guardrails, inspection point 1, prompt injection, jailbreaking, pattern matching, classifier models, off-topic detection, topic classifier, application's domain, declarative configuration

6.7.2 Output guardrails: the last line of defense

This section explains how output guardrails inspect AI responses before delivery, detect safety and factual problems, and choose proportionate actions from redaction to blocking.

Key terms: output guardrails, inspection point 5, input guardrails, tool policies, PII detection, sensitive identifiers, redact, [REDACTED], flag for review, hallucination detection

6.7.3 Behavioral guardrails: controlling actions and logic

This section explains how behavioral guardrails govern AI tool actions through cross-tool consistency checks, session-aware authorization, and human approval gates.

Key terms: behavioral guardrails, inspection points 2 and 3, tool invocation, business logic, cross-tool policy, cross-action consistency, race conditions, conflicting operations, session state, session-level permissions

6.8 Declarative policy and observability

This section introduces declarative policy as separating application logic from centrally configured safety and operational constraints enforced by the platform at runtime.

Key terms: declarative policy, application logic, configuration, separation of concerns, runtime, YAML, prohibited medication topics, prompt injection pattern, rate limits, platform enforcement

6.8.1 Policy as configuration

This section explains how declarative, version-controlled guardrail policies enable independent governance, automatic enforcement, and safe progressive rollout without application redeployment.

Key terms: declarative policies, configuration files, YAML, input guardrails, output policies, PII redaction, version-controlled repository, compliance team, security engineers, automatic enforcement

6.8.2 Making safety visible

This section explains how comprehensive observability, metrics, and audit trails make guardrail and tool-safety decisions transparent, debuggable, and compliant.

Key terms: guardrail evaluation, inspection point, confidence score, content hashes, input, pre-tool, hard stop, silent failure, PII, redacting PII

6.9 Summary

This section summarizes how a platform manages tool capabilities and enforces observable, declarative safety policies for AI actions.

Where this is used later

Concepts explained in this chapter

← Chapter 5: The Data Service: teaching AI what your organization knows Chapter 7: The Observability Service: seeing what your AI does →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.