liveBook · Manning

Tool service

As explained in Designing AI Systems

A platform service that registers and discovers external tools, APIs, and services while handling authentication, rate limiting, recovery, and caching.

A tool service turns external APIs and services into managed capabilities that AI applications can discover and invoke. Instead of embedding integration code and credentials in each application, teams register tools once and delegate validation, execution, authentication, and operational safeguards to the platform. This boundary matters when model-requested actions have real consequences, such as booking appointments or submitting insurance checks. section 1.4 section 6.0

A tool registration describes its endpoint, schema, documentation, and metadata. Metadata can include rate limits, cost, side effects, and whether an operation is idempotent. A credential reference names the secret to resolve at execution time; the secret itself stays out of application code and logs. A versioned, hierarchical registry lets authorized applications find capabilities by namespace or purpose. section 2.1.5 section 6.1.1 section 6.9

The service contract covers four lifecycle phases:

  • Register a tool or import tools from an MCP server into the registry. section 6.1.1
  • Discover definitions by namespace or capability and pass them to model APIs for function calling. section 6.1.1
  • Validate arguments against schemas and business constraints before committing to an action. section 6.1.1
  • Execute with credential injection and standardized results. For long-running work, start an asynchronous task and query its progress. section 6.1.1
Listing 6.1 The tool service gRPC contract
service ToolService {
  rpc RegisterTool(RegisterToolRequest)
      returns (RegisterToolResponse);    #A
  rpc DiscoverTools(DiscoverToolsRequest)
      returns (DiscoverToolsResponse);    #B
  rpc ExecuteTool(ExecuteToolRequest)
      returns (ExecuteToolResponse);    #C
  rpc ValidateTool(ValidateToolRequest)
      returns (ValidateToolResponse);    #D
 
  // Long-running executions
  rpc ExecuteToolAsync(ExecuteToolRequest)
      returns (ExecuteToolAsyncResponse);    #E
  rpc GetTask(GetTaskRequest) returns (GetTaskResponse);    #E
 
  // MCP integration
  rpc RegisterMcpServer(RegisterMcpServerRequest)
      returns (RegisterMcpServerResponse);    #F
}

The gRPC contract names the lifecycle operations, asynchronous task calls, and MCP server registration. section 6.1.1

Appointment scheduling

Register an appointment scheduler with an endpoint, a credential reference, and parameters for checking availability or scheduling a visit. An application builds model-facing tool definitions, sends them with a chat request, executes any returned tool calls, and adds their results to the conversation. section 2.1.5

Listing 2.6 Registering an external tool via the Tool Service
platform.tools.register(
   name="appointment_scheduler",
   description="Check appointment availability and schedule patient 
➥visits",
   endpoint="https://scheduler.coolmedical.com/api/appointments",
   credential_ref="scheduling-api-prod",   #A
   parameters={
      "action": {"enum": ["check_availability", "schedule_appointment"]},
      "patient_id": {"type": "string", "required": True},
      "preferred_date": {"type": "string", "format": "date"}
   }
)

The registration example shows the endpoint, named credential, and parameter schema together. section 2.1.5

For MCP integrations, the service acts as a client: it discovers server tools, maps them into a platform namespace, and routes calls through the MCP transport. Applications use the same discovery and execution interface for native adapters and MCP-backed tools. The platform adds credential management, policy enforcement, invocation tracking, and other organizational controls above the protocol. section 6.4.5 section 6.9

Listing 6.16 Registering an external MCP server as a platform tool source
platform.tools.register_mcp_server(
    server_url="https://mcp-github.internal:3000/mcp",    #A
    namespace="devtools.github",    #B
    credential_ref="github-api-token",    #C
    policy_overrides={"requires_confirmation": True},
    rate_limit_overrides={"requests_per_minute": 30}    #D
)
 
tools = platform.tools.discover(
    namespace="devtools.github"
)    #E
# Returns: ["devtools.github.create_issue",
#           "devtools.github.list_repos", ...]

The MCP registration example combines a server URL and namespace with credential, confirmation, and rate-limit settings. section 6.4.5

Execution safeguards keep one faulty integration from disrupting other tools or sessions. Each adapter runs in its own process with per-invocation limits; tool-level settings operate beneath platform-wide ceilings. When a limit is exceeded, execution is terminated, a structured error is returned, and metrics are emitted. Retry decisions should account for idempotency; circuit breakers can temporarily disable consistently failing tools. section 6.5 section 6.5.1 section 6.9

Listing 6.17 Execution limits applied per tool invocation
message ExecutionLimits {
    int32 timeout_seconds = 1;      // Hard kill after this duration
    int32 memory_limit_mb = 2;      // OOM-killed if exceeded
    int32 cpu_limit_millicores = 3; // Prevents CPU starvation
    int32 max_response_size_kb = 4; // Truncate oversized responses
    int32 max_retries = 5;          // Retry budget for transient failures
}

The execution-limits schema specifies timeout, memory, CPU, response-size, and retry bounds. section 6.5.1

How Designing AI Systems covers it

How deep it goes
Defines a gRPC lifecycle contract and elaborates registration, discovery, execution, MCP integration, and per-invocation safeguards.
Examples
Uses an appointment scheduler and an MCP-backed GitHub integration to demonstrate registration and discovery.
What you can do after
Apply it via the platform SDK to register and discover tools, execute model-requested calls, and configure MCP integration.
How it connects
Connects function calling, credential management, guardrails, adapters, rate limiting, and circuit breakers.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.