5 Workflow patterns

 

This chapter covers

  • Using workflows to connect machine learning system components
  • Composing complex but maintainable structures within machine learning workflows with the fan-in and fan-out patterns
  • Accelerating machine learning workloads with concurrent steps using synchronous and asynchronous patterns
  • Improving performance with the step memoization pattern

Model serving is a critical step after successfully training a machine learning model. It is the final artifact produced by the entire machine learning workflow, and the results from model serving are presented to users directly. Previously, we explored some of the challenges involved in distributed model serving systems—for example, how to handle the growing number of model serving requests and the increased size of those requests—and investigated a few established patterns heavily adopted in industry. We learned how to achieve horizontal scaling with the help of replicated services to address these challenges and how the sharded services pattern can help the system process large model serving requests. Finally, we learned how to assess model serving systems and determine whether an event-driven design would be beneficial in real-world scenarios.

5.1 What is workflow?

5.2 Fan-in and fan-out patterns: Composing complex machine learning workflows

5.2.1 The problem

5.2.2 The solution

5.2.3 Discussion

5.2.4 Exercises

5.3 Synchronous and asynchronous patterns: Accelerating workflows with concurrency

5.3.1 The problem

5.3.2 The solution

5.3.3 Discussion

5.3.4 Exercises

5.4 Step memoization pattern: Skipping redundant workloads via memoized steps