Guidance for using Azure Storage Explorer with Azure AD authorization for Azure Storage Data Access

Published in

Microsoft Azure

11 min readDec 31, 2020

--

The purpose of this article is to share some practical guidance (or best practices, usually I refrain from using the term best practices because everything is relatively new) around Azure AD Authorization for users to access data stored in Azure Storage Account when using Azure Storage Explorer.

Objective

Azure Storage Explorer is an easy to use GUI tool for working with Azure Storage data, behind the scenes it uses AzCopy for all data transfer operations. In the enterprise environments with strict Access Control and Audit requirements the default easy to use options of Azure Storage Explorer might require more permissions than deemed acceptable by the Security Administrators. So, in this post I will go over some intricate details on how Azure Storage Explorer operates and share guidance around levels of Azure AD permissions needed by users for the various options available in Azure Storage Explorer.

Background — Azure Blob Storage (including Azure Data Lake Gen2)
Background — Authorization for Azure Blob Storage (including ADLS Gen2)
Background — Management (or Control) Plane Operation — RBAC Roles
Background — Data Plane Operations — RBAC Roles and ACLs (ACLs only for ADLS Gen2)
Only Management Plane Role Assigned but Data Operations still work when using Azure Storage Explorer
Recommendation 1 — Combination of Management Plane Reader Role and Data Plane Permissions for Data Access using Azure Storage Explorer
Recommendation 2 — Data Plane Permissions Only for Data Access using Azure Storage Explorer
Avoid mixing Management Plane RBAC Contributor (or Owner) and Data Plane Permissions
Conclusion

Background

I will touch up on some concepts to build the background in case you are new to Azure Platform or Azure Storage, even if you are familiar with Azure Storage concepts it might be a good idea to have a quick glance over this section.

Azure Blob Storage (including Azure Data Lake Gen2)

Azure Storage Accounts include Azure Files, Azure Queues, Azure Tables and Azure Disks (you can read more about different Azure Storage services here) but in this article here I focus primarily on Azure Blob Storage including Azure Data Lake Gen2 (referred to as ADLS Gen2).

ADLS Gen2 can be thought of a specific configuration of Azure Blob Storage with a few additional features meant for enterprise big data analytics. As far as this article goes the most important aspect of ADLS Gen2 is that it allows much more fine-grained access control (ACL) in comparison to Azure Blob Storage Account with Hierarchical Namespace setting disabled.

ADLS Gen2 is nothing but Azure Blob Storage with Hierarchical Namespace setting enabled, you can check if an existing Storage Account is ADLS Gen2 or plain Blob storage from Configuration Setting tab as shown in the screenshot below.

Authorization for Azure Blob Storage (including ADLS Gen2)

The authorization for Azure Blob Storage can be put into two broad categories:

Shared Access Key — This is the simpler method where an Access key can be used to access data from the Storage Account. The challenge with this method is that it’s a “shared” key, if two distinct users have access to the Shared Key and access the data in Storage Account there is no clear way to distinguish between access by User A or User B in the audit logs (accurate audit trail is a common requirement in enterprise scenarios especially Regulated Industries).
Azure AD — This is the preferred method in enterprise scenarios where individual users have identities in a centralized Identity and Access Management system and use their own identity to access the data in the Storage Account hence accurate audit trail can be captured.

In certain documentation you might see Shared Access Signature is another category but I view that as another layer on top of the above top categories and I don’t see it as a separate category. You can read more about Shared Access Signature method here.

Management (or Control) Plane Operation - RBAC Roles

RBAC stands for Role Based Access Control and Azure Platform has quite a few Built-In Roles for a variety of purposes.

Contributor, Storage Account Contributor, etc. are example built-in roles for Management Operations, the full list is documented here.
RBAC Roles can be assigned from Azure Portal using Access Control (IAM) tab (shown in the screenshot below).

RBAC Roles can be assigned at the Storage Account level or individual containers within the Storage Account.
RBAC Roles cannot be assigned lower than the Container level, container is the top level folder within the Blob Storage Account (including ADLS Gen2) and there can be multiples of those in a single account.

The Management Operations are of the nature like creation of the storage account, enabling of audit logs, changing network settings, etc. and do not give permissions for direct data operations but there is an extremely important detail to keep in mind that some of the roles give users access to Access Keys hence they indirectly grant access to data.

Example of User with Storage Account Contributor management plane RBAC Role

Let’s look at this case where user Chris Green was assigned Storage Account Contributor Role and using the Azure Portal is able to gain access to data files inside the container.

Storage Account Contributor — User able to access data with Management Plane Role

How does Chris have access to the data file? Azure Portal has convenience built-in, since Chris belongs to Storage Account Contributor Role Azure Portal used Management Plane role permissions to lookup Access Key and then used Access Key to make data available to Chris. Hence, Shared Access Key was used to access the data not really Azure AD Identity.

Azure Portal does give option to switch to Azure AD Identity for accessing data, since user Chris does not belong to any Data Plane role it results in error as shown in the screenshot below.

Data Access Failed using Azure AD Identity

Data Plane Operations — RBAC Roles and ACLs (ACLs only for ADLS Gen2)

RBAC Roles are applicable to both Azure Blob Storage as well as the ADLS Gen2 — Storage Blob Data Owner, Storage Blob Data Contributor and Storage Blob Data Reader are the RBAC roles meant for Data Operations. You can read more about Data Plane RBAC roles here.
ACLs only available for ADLS Gen2 can be used to apply fine grained access control rules on levels below container (or FileSystem is another term used for containers in ADLS Gen2 Storage Accounts).
For ADLS Gen2 Storage Accounts authorization is based on combination of RBAC Roles and ACLS, you can read more details on ADLS Gen2 Access Control Model here and here.
This Github Repo is another excellent resource explaining the Access Control Model as well as how to setup ADLS Gen2 Access Control Model for an Enterprise Data Lake (Data Lake as a pattern).

The easiest method to assign ACL Permissions for ADLS Gen2 Storage Account is to use Azure Storage Explorer as shown in the next two screenshots.

Now, background context is out of the way let’s get to the main topics of this post which is to walk through few Azure Storage Explorer usage options.

Only Management Plane Role Assigned but Data Operations still work when using Azure Storage Explorer

The most common method I have observed used is that the user is assigned a Management Plane RBAC Role like Owner, Contributor or Storage Account Contributor.

User sets up the connection using the default Azure Account method, this prompts user to login using Azure AD credentials as shown in the screenshot below.

Common Method — Default Connect to Azure Storage Method

User sees the Azure Subscription and list of Storage Account they have access to as shown in the screenshot below.

User allowed Data Operations when they only had Management Plane Roles

User was not in any of the Data Plane RBAC Roles but still can perform uploads or download of data files.

If you pay close attention to the “Activities” section of the above screenshot, it shows Shared Access Signature was used for the data operations (upload and downloads of files). Hence, to be more accurate Data Access was based on Shared Access Signature created from Access Key and not the user Azure AD Identity. Nothing wrong with this approach but if your goal is to use Azure AD for all data operations (accurate audit trail) it might have looked like Azure AD identity is being used for data access but to be accurate it is not. The following screenshot shows access log records for the data plane operations.

User has Management Plane Role — Log records show Data Operations used SAS rather than Azure AD Identity

If using this method, it might be a good idea to review the SAS (Shared Access Signature) Duration setting used by Azure Storage Explorer which is by default 30 days.

Azure Storage Explorer — SAS Duration Setting

Recommendation 1 — Combination of Management Plane Reader Role and Data Plane Permissions for Data Access using Azure Storage Explorer

Users can be assigned a combination of Management Plane and Data Plane Role roles for easy usage of Azure Storage Explorer

Management Plane Role — Built-In RBAC Reader Role for the entire Storage Account, this role only allows listing of the Storage Account and the Containers but not the data operations. Reader role does not give users permissions to retrieve Access Key.
Data Plane Permissions — These could be assigned using Built-In RBAC Roles like Storage Blob Data Reader/Contributor or in case of ADLS Gen2 ACL Permissions.

Using this method, Azure Storage Explorer Connection is still established using the default option as in the Common method described in the previous section but all the data operations are carried out using Azure AD Identity of the user. Screenshot below shows convenient browsing of the Storage Accounts and data operations using Azure AD Identity.

Convenient browsing of Storage Accounts and Data Plane Operations using Azure AD Identity

Screenshot below shows log records with user identity for data access.

Log Records — Data Plane Operations used Azure AD Identity

Another example of how this method can be used and its implication:

Reader RBAC Role assigned at Storage Account Level.
Data Operations only permitted for container1 by assigning Storage Blob Data Contributor Role (ACL possible for ADLS Gen2 Storage Account).
No Data Operations permitted on container2.
Screenshot shows user has access to perform data operations on container1, data operations not allowed on container2 but user still can see that a container with that name exists. If your security requirements need to be more strict that the users shouldn’t even be able to see the name of containers they don’t have permissions to perform data operations then you will need to use the method described in the next section.

Data Operation not allowed but Management Plane Role allows viewing the name of the container

Last thing to mention on this method, it should be possible to create a single custom RBAC role which is a mix of Reader and the appropriate Data Plane Role rather than assigning two roles. Please see my article on guidance around creation of Custom RBAC Roles here.

Recommendation 2 — Data Plane Permissions Only for Data Access using Azure Storage Explorer

In cases where users are given only Data Plane Permissions like Storage Blob Data Contributor, Storage Blob Data Reader or just ACL in case of ADLS Gen2 users will need to use Add a resource via Azure Active Directory (Azure AD) option to setup Storage Account connection in Azure Storage Explorer.
This option requires users to know the Storage Container URL when setting up the connection, even if the Storage Blob Data Contributor role is assigned at the Storage Account Level user will not be able to see the Storage Account in Azure Portal or browse in the Storage Explorer using the default option because they don’t have the needed management plane permissions to browse the Storage Account list. An Administrator will need to share these settings with the end user and also the end user will need to create a connection to individual containers within the Storage Account.

Following screenshots show the connection setup wizard screens.

Azure Storage Explorer — Add a resource via Azure AD

Resource Type needs to be selected and Container URL needs to be provided

Connected Storage Account will display under the Local & Attached node as shown in the screenshot below.

Connected Storage Account under Local & Attached node

Container or Path URL field will need to be looked up and shared with end users by an administrator. The following screenshots show Container Properties on Azure Portal where these settings are available.

Avoid mixing Management Plane RBAC Contributor (or Owner) and Data Plane Permissions

In cases where Management Plane RBAC Role like Storage Account Contributor (or any role which grants permissions to Shared Access Key) is assigned along with Data Plane RBAC Role like Storage Blob Data Contributor, the default Azure Storage Explorer method results in Shared Access Signature used for data operations although user has Data Operation RBAC Role. In other words, Shared Access Signature used for data operations instead of Azure AD Identity.

Example:

User assigned Storage Account Contributor Role or Contributor role which are Management Plane Roles.
User also assigned Storage Blob Data Reader or Storage Blob Data Contributor Role which are Data Operation Roles.
Storage Explorer will use Management Plane role to look up Shared Access Key and generate Shared Access Signature based on the Access key.

Conclusion

If you have strict access control and audit requirements with need to have accurate audit trail in the logs for users who accessed data, use Recommendation 1 or Recommendation 2 above to setup access control for the users.