Model registration makes a model discoverable and usable by a model-serving system. It commonly records model files and metadata such as name, ID, version, training algorithm, dataset information, and execution metrics. section 7.5.1
A metadata store creates a metadata object, indexes it by model name and version, and supports lookup by model ID. Artifact files remain in object storage, while metadata stores a pointer to their location. Training services can register models automatically, and data scientists can register locally built models manually. section 7.5.1
Figure 7.13 The internal storage design of the metadata store; model metadata are stored as object files with lookup tables in front of them.
Use the internal storage design to trace the relationship between model metadata, lookup records, and artifact locations. section 7.5.1
Serving runtimes impose separate registration requirements. Behind a load balancer, instances may need model files copied into local stores and registered individually. Kubernetes sidecars can hide these details by downloading models, placing them in a serving repository, registering them, and forwarding predictions. section 7.2.7 section 7.4.5
Use the Kubernetes architecture to see how a proxy sidecar prepares TorchServe instances for arbitrary models. section 7.2.7
Integrating registration without changing the public API
A Kubernetes prediction service can place Triton pods behind its existing API and give them a shared model repository. A sidecar downloads a model from cloud storage, registers it with Triton, and converts the existing prediction request into a Triton API call. section 7.4.5
Figure 7.11 A proposal to integrate a list of Triton server instances into an existing serving system
Use the integration architecture to inspect how sidecars connect cloud model storage, Triton registration, and an unchanged prediction interface. section 7.4.5
