liveBook · Manning

Api model

As explained in Machine Learning System Design

A model API is a programmatic interface that exposes machine-learning model operations such as generating embeddings and scoring candidates.

A model API exposes model operations to the surrounding system. In demand forecasting, it extracts predictions directly from the model; in image search, it extracts query embeddings and scores retrieved candidates. section 13.5.1 section 13.5.2

Keep transport and business responsibilities separate. An HTTP handler parses JSON, forwards requests, wraps responses, and handles errors; the model API performs model-specific work. Search systems can place vector-database retrieval and result ranking in adjacent APIs rather than inside the model API. section 13.5.2

A practical model-API design distinguishes:

  • Prediction extraction for forecasting services, with responses containing items such as SKU, demand, entity, and period. section 13.5.1
  • Embedding extraction and candidate scoring for search, followed by filtering and relevance ranking in neighboring services. section 13.5.2
  • Independent release rhythms: infrastructure wrappers require integration and compatibility testing, while model updates may be less frequent. section 13.5.1 section 13.5.2
GET /predictions?query=<query_string>&parameters=<parameters>&version=
<version>
&limit=<limit>&request_id=<request_id>&sku=<sku>&entity_id=<entity_id>
&group=<group_type>

Inspect a versioned demand-prediction endpoint with query, parameters, limit, request, SKU, entity, and grouping fields. section 13.5.1

```
GET /search?query=<query_string>&filters=<filters>&version=<version>
&limit=<limit>&request_id=<request_id>
```

Inspect a versioned search endpoint that accepts a query and filters and returns ranked result identifiers and scores. section 13.5.2

How Machine Learning System Design covers it

How deep it goes
Specifies responsibilities, JSON boundaries, release cadence, and layering, but stops short of implementation details.
Examples
Uses demand-prediction and photo-search endpoints returning prediction arrays or scored search-result arrays.
What you can do after
Operate/configure model-serving boundaries by separating transport, model work, candidate retrieval, ranking, and release responsibilities.
How it connects
Connects HTTP API handlers, vector databases, ranking APIs, embeddings, candidate retrieval, model releases, and fallback models.

Also explained in

Compare how 3 books explain api model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.