Data Governance - Azure Purview - 01

Much ink and voice have been spilled to describe and explain the theory behind Data Governance. I will try to share my view on this in laymen terms later in a different post. The objective of this series is to get our hands dirty in setting up a full-fledged Data Governance suite in Azure Purview.

In this article, we will focus about the building blocks:

1. Azure Resource Group

2. Data Lake - ADLS Gen2

3. Azure SQL Database

4. Azure Synapse Workspace

5. Azure Data Factory

The above image is from my Azure environment depicting the list of services we have spun up. Below is a comprehensive, click-by-click guide for this step.

Phase 1: The Infrastructure Layer (Build the "Estate")

Goal: Create the raw resources before we govern them.

1. Create the Resource Group

• Go to Azure Portal > Resource groups > Create.

• Name: rg-enoch-01

• Region: North Europe (or your preferred region).

  
  

2. Create Data Lake (ADLS Gen2)

• Search Storage accounts > Create.

• Name: adlsenoch (must be unique).

• Important: On the Advanced tab, check Enable hierarchical namespace.

• Click Review + create.

• Post-Deployment: Go to the resource > Containers > + Container > Name it raw-data.

  Upload the customers_raw.csv and sales_raw.csv files:

customers_raw.csv

Download CSV • 1KB

sales_raw.csv

Download CSV • 1KB

3. Create Azure SQL Database

• Search SQL Database > Create.

• Server: Click Create new. Name: sql-server. Authentication: Use SQL authentication (User: sqladmin, Pass: <strong password>!).

• Database Name: CustomerDB.

• Networking: Select Public endpoint. Crucial: Check "Allow Azure services and resources to access this server".

• Click Review + create.

  
  

4. Create Azure Synapse Workspace

• Search Azure Synapse Analytics > Create.

• Resource Group: rg-enoch-01.

• Workspace Name: synapse-enoch-01.

• Data Lake: Select the ADLS account you just created (adlsenoch).

• SQL Password: Use the same admin credentials.

• Click Review + create.

  
  

5. Create Azure Data Factory (ADF)

• Search Data factories > Create.

• Name: adf-enoch-01.

• Git configuration: Select "Configure Git later".

• Click Review + create.

Now we have the real-estate ready. We will pick from here in the next article. Stay tuned.