liveBook · Manning

Model registration

As explained in Designing AI Systems

Custom model registration is the process of declaring a custom model so that a model service can identify and invoke it.

Model registration makes a self-hosted or fine-tuned model a first-class model-service resource instead of a special application-side integration. Applications discover it, request it by name, and receive the same interface used for commercial models. section 3.1.4 section 3.1

A registration request supplies the information the platform needs to route, constrain, authenticate, and monitor the deployment:

  • Name the model with a stable identifier used by discovery and workflow requests. section 3.1.4
  • Specify the internal endpoint and credential reference so the service can reach the deployment without embedding credentials in application code. section 3.1.4
  • Declare capabilities such as context-window size, vision support, and tool support so callers can select compatible models. section 3.1.4
  • Provide a health-check path and receive a provisioning status with a registration timestamp. section 3.1.4

Using a registered model

A workflow requests the registered model by name rather than calling its internal endpoint directly. The Model Service routes the request, records usage and interaction data, monitors health, and can fall back to another model or support an A/B test through configuration. section 3.1.4

Listing 3.4 Custom model registration request and response
Request:
  name: "healthfirst-medical-v1"
  endpoint: "http://ml-inference.
➥internal:8000/v1"    #A
  capabilities: {    #B
    context_window: 8192,
    supports_vision: false,
    supports_tools: true
  }
  health_check: "/health"    #C
  credential_ref: "ml-inference-prod" #D
 
Response:
  name: "healthfirst-medical-v1"
  status: "provisioning"
  registered_at: "2024-03-01T14:00:00Z"
 

Inspect the concrete registration request and response, including endpoint, capabilities, health check, credential reference, and provisioning status. section 3.1.4

How Designing AI Systems covers it

How deep it goes
Defines registration fields and resulting platform behavior, but stops short of showing the registration service implementation.
Examples
Uses a fine-tuned medical model with an internal endpoint, declared capabilities, a health check, and a production credential reference.
What it assumes
Assumes an accessible custom inference endpoint and credential, while defining required capabilities and health-check metadata.
What you can do after
Operate/configure a custom model as a named model-service resource with routing, health monitoring, usage tracking, and fallback.
How it connects
Connects model service, model discovery, provider adapters, fallback logic, health monitoring, and observability.

Also explained in

Compare how 2 books explain model registration →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.