A deep learning system is infrastructure for moving deep learning work from data exploration and model prototyping into production. Rather than being a model itself, it supports the people, workflows, and operational tasks needed to develop, deploy, and monitor models efficiently and at scale. section 1.0 section 1.3
Figure 1.3 An overview of a typical deep learning system that includes basic components to support a deep learning development cycle. This reference architecture can be used as a starting point and further tailored. In later chapters, we discuss each component in detail and explain how it fits into this big picture.
Use the reference architecture as a starting point, then tailor component boundaries and relationships to project goals, constraints, and team needs. section 1.2 section 1.2.1
A basic system groups the following responsibilities behind network-accessible services:
- An API provides access for user interfaces, applications, and other systems; it may be a gateway or the combined APIs of individual components. section 1.2.2
- A dataset manager collects, organizes, describes, and stores training data, supporting exploration, quality work, and dataset lineage. section 1.2.2 section 1.4
- A model trainer accepts training requests and uses limited compute resources efficiently. section 1.4
- Model serving receives inference requests, runs models, and returns inferences to callers. section 1.4
- A metadata and artifacts store records metadata and preserves outputs produced across the system. section 1.4
- Workflow orchestration and an interactive data science environment support coordinated workflows and hands-on development work. section 1.2.1
Designing under production constraints
When several models cannot fit in one machine’s memory, cache models between memory and disk, accepting added inference latency, or use smaller machines per model if budget and model size allow. Compliance requirements may instead make controlled data-access frameworks the governing design concern. section 1.2
The aim is to reduce operational work and cross-team coordination so data scientists can build scalable models, establish data pipelines, and deploy and monitor models without becoming DevOps or data-engineering specialists. Teams may build components, adopt open-source Kubeflow, or use hosted Amazon SageMaker alternatives. section 1.1.4 section 1.2.1
