This chapter is all about metadata: in other words, data about data. This is one aspect of data governance. We will cover two other important aspects in the following chapters: data quality in chapter 9 and compliance in chapter 10. Figure 8.1 highlights our current area of focus. We won’t view this map of our data platform again until the last chapter, which covers data distribution.
Figure 8.1 Data governance deals with multiple aspects of managing data, including metadata, data quality, access control, compliance with laws and standards, etc.
We’ll start by outlining the information architecture challenges a big data platform encounters and how metadata can help address them. We’ll introduce two important concepts: data dictionaries and data glossaries. Using these, we can inventory our datasets and queries.
Next, we’ll look at Azure Purview. Azure Purview is the Azure data governance service that helps us manage our metadata. We’ll spin up a new instance of Azure Purview and go over some of its key features.