liveBook · Manning

Model discovery

As explained in Designing AI Systems

Model discovery is the retrieval of available AI models and their supported capabilities from a model provider or service.

Model discovery retrieves available models and their capabilities from the Model Service, replacing brittle model-name assumptions with runtime selection. section 3.1.2 section 3.1

A discovery response gives workflows the metadata needed to choose an appropriate model:

  • Model identity and provider, such as gpt-4o from OpenAI or an internally hosted model. section 3.1.2
  • Capability metadata, including context-window size, vision support, and tool support. section 3.1.2
  • Runtime filtering, such as selecting a model with more than 100,000 tokens of context or with vision support. section 3.1.2
Listing 3.2 Model discovery response with capabilities
Response:
  models: [
    {
      name: "gpt-4o",
      provider: "openai",
      context_window: 128000,    #A
      supports_vision: true,
      supports_tools: true
    },
    {
      name: "claude-sonnet-4-5",
      provider: "anthropic",
      context_window: 200000,
      supports_vision: true,
      supports_tools: true
    },
    {
      name: "llama-3-70b-medical",    #B
      provider: "internal",
      context_window: 8192,
      supports_vision: false,
      supports_tools: false
    }
  ]
 

The response shape shows discovery as a list of model records with provider and capability fields. section 3.1.2

Treat the platform as the source of truth: workflows ask which models exist instead of hardcoding names that become stale. Cache the model list when the SDK client initializes and refresh it periodically, because availability changes infrequently and repeated discovery adds a hot-path network round trip. section 3.1.2

Discovering a custom model

A registered internal model can appear alongside commercial models, exposing its context window and tool support through discovery. Workflows request it by name through the same interface, while the platform retains unified logging, cost tracking, fallback support, health monitoring, and observability. section 3.1.4

How Designing AI Systems covers it

How deep it goes
Defines capability-based filtering and refresh caching, but stops short of implementing the discovery client.
Examples
Uses context-window and vision filtering, commercial provider listings, and a registered internal medical model.
What it assumes
Assumes a Model Service contract and model metadata, while treating provider differences as platform concerns.
What you can do after
Apply via a Model Service by filtering discovered models against workflow capability requirements.
How it connects
Connects model discovery with the Model Service, provider adapters, model capabilities, caching, and custom model registration.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.