- Using workflows to connect different machine learning system components (data ingestion, distributed model training, and model serving).
- Composing complex but maintainable structures within machine learning workflows with the fan-in and fan-out patterns.
- Accelerating machine learning workloads with concurrent steps that leverages the synchronous and asynchronous patterns.
- Improving performance and avoiding duplicate workloads in the workflows with the help of the step memoization pattern.
5.1 What is workflow?
5.2 Fan-in and fan-out patterns: Composing complex machine learning workflows
5.2.1 Problem
5.2.2 Solution
5.2.3 Discussion
5.2.4 Exercises
5.3 Synchronous and asynchronous patterns: Accelerating workflows with concurrency
5.3.1 Problem
5.3.2 Solution
5.3.3 Discussion
5.3.4 Exercises
5.4 Step memoization pattern: Skipping redundant workloads via memoized steps
5.4.1 Problem
5.4.2 Solution
5.4.3 Discussion
5.4.4 Exercises
5.5 References
5.6 Summary