There are many scenarios when we work with data inside organizations where we see the need to share the data with external vendors, customers, or different departments within the organization.
Consider the below example where we have our Data in Azure Storage and Azure SQL database. We have our requirement to share data with external customers (Azure) and internal teams inside the organization.
Note — This solution is for Azure specific , considering all external customers are also using azure cloud solutions. Ideally this might not be the case but for our article we will assume everything is in azure :)
Supported data stores
+-------------------------------------------------------------+---------------------------------+----------------------------------------+------------------+
| Data store | Sharing based on full snapshots | Sharing based on incremental snapshots | Sharing in place |
+-------------------------------------------------------------+---------------------------------+----------------------------------------+------------------+
| Azure Blob Storage | ✓ | ✓ | |
| Azure Data Lake Storage Gen1 | ✓ | ✓ | |
| Azure Data Lake Storage Gen2 | ✓ | ✓ | |
| Azure SQL Database | ✓ | | |
| Azure Synapse Analytics (formerly Azure SQL Data Warehouse) | ✓ | | |
| Azure Synapse Analytics (workspace) dedicated SQL pool | ✓ | | |
| Azure Data Explorer | | | ✓ |
+-------------------------------------------------------------+---------------------------------+----------------------------------------+------------------+
Azure Data Share supports two types of sharing
- Snapshot-based sharing
- In-place sharing(via Azure Data Explorer only).
Share from a storage account
Supports sharing of container, folder and files from ADLS Gen1 and Gen2 and blobs from blob storage. It also supports the sharing of blobs, folders, and containers from Azure Blob Storage.
Lets see a demo using storage account.
Create Azure Data Share after login to the portal as below
Go to the service and click start sharing your data
Enter the name and select share type as snapshot as below and click continue.
Select ADLS Gen2 as below and click next
Add recipient email as below with expiration optional
Add snapshot settings like daily or hourly like below
You will receive an email in few minutes like below
Now login to the other azure account to view the data share , there might be additional steps needs to validate the email address with verification code as below
Then you would be able to see the invitation as below
Create resource group and data share in other azure environment and accept the invitation as below
Lastly we need to configure dataset and configure schedule and trigger full copy as below
Now I am able to see the files synced in the target subscription, its as simple as that.
This way we can copy all the data from source container to target container and periodically sync between the two data sets on an daily or hourly basis. In case of network failure in incremental copy it starts from the last successful run.
