In the previous chapter, we went through the project background and system components to understand our strategies for implementing each component. We also discussed the challenges related to each component and discussed the patterns we will apply to address them. As previously mentioned, we will dive into the project’s implementation details in chapter 9, the book’s last chapter. However, since we will use different technologies in the project and it’s not easy to cover all the basics on the fly, in this chapter, you will learn the basic concepts of the four technologies (TensorFlow, Kubernetes, Kubeflow, and Argo Workflows) and gain hands-on experience.
Each of these four technologies has a different purpose, but all will be used to implement the final project in chapter 9. TensorFlow will be used for data processing, model building, and evaluation. We will use Kubernetes as our core distributed infrastructure. On top of that, Kubeflow will be used for submitting distributed model training jobs to the Kubernetes cluster, and Argo Workflows will be used to construct and submit the end-to-end machine learning workflows.