7 The self-serve data platform
This chapter covers
- Discussing why a self-serve platform enters the Data Mesh picture
- Learning the key concepts for building a self-serve data platform
- Understanding how our MVP fits into that
- Applying these concepts to build a self-serve data platform in three iterations
- Defining a platform and the architectural perspective of a platform
- Defining platform thinking, X-as-a-Service, and the “modular plane architecture”
- Managing a data platform as a product
In the first part of this book, you learned all the basics of the Data Mesh. That included building a Data Mesh MVP for Messflix LLC within a month in chapter 3. The Data Mesh MVP contained a very minimal self-serve platform.
Once Messflix LLC starts to build data product after data product, a lot of duplication of efforts happens in terms of ingestion tooling and other data-related technologies. This is a very specialized skill set. Thus, once we start to grow the number of data products, it is time to recall an already known idea: self-serve platforms. The idea of a self-serve platform is to take the duplication and specialized skills out of the many development teams, and put them into one platform. Because the platform will be used by many teams, we usually strive to make it “self-serve”, meaning it keeps as much of the autonomy of the development teams intact as possible.