Now that you know the basic concepts and background of distributed machine learning systems, you should be able to proceed to this part of the book. We will explore some of the challenges involved in various components of a machine learning system and introduce a few established patterns adopted heavily in industries to address those challenges.
Chapter 2 introduces the batching pattern, used to handle and prepare large datasets for model training; the sharding pattern, used to split huge datasets into multiple data shards that spread among multiple worker machines; and the caching pattern, which could greatly speed the data ingestion process when a previously used dataset is re-accessed for model training.