8 Overview of relevant technologies

 

This chapter covers

  • Getting familiar with model building using TensorFlow
  • Understanding key terminologies on Kubernetes
  • Running distributed machine learning workloads with Kubeflow
  • Deploying container-native workflows using Argo Workflows

In the previous chapter, we went through the project background and system components to understand our strategies for implementing each component. We also discussed the challenges related to each component and discussed the patterns we will apply to address them. As previously mentioned, we will dive into the project’s implementation details in chapter 9, the book’s last chapter. However, since we will use different technologies in the project and it’s not easy to cover all the basics on the fly, in this chapter, you will learn the basic concepts of the four technologies (TensorFlow, Kubernetes, Kubeflow, and Argo Workflows) and gain hands-on experience.

Each of these four technologies has a different purpose, but all will be used to implement the final project in chapter 9. TensorFlow will be used for data processing, model building, and evaluation. We will use Kubernetes as our core distributed infrastructure. On top of that, Kubeflow will be used for submitting distributed model training jobs to the Kubernetes cluster, and Argo Workflows will be used to construct and submit the end-to-end machine learning workflows.

8.1 TensorFlow: The machine learning framework

8.1.1 The basics

8.1.2 Exercises

8.2 Kubernetes: The distributed container orchestration system

8.2.1 The basics

8.2.2 Exercises

8.3 Kubeflow: Machine learning workloads on Kubernetes

8.3.1 The basics

8.3.2 Exercises

8.4 Argo Workflows: Container-native workflow engine

8.4.1 The basics