Python for Azure: Enable Immutability on Azure blob containers for business-critical data

Introduction: Immutable Storage is an important feature of Microsoft Azure Blob Storage as it enable end-users to store business-critical data in a “Write Once, Read Many” (WORM) manner. Thus by enabling “immutability-policy” data cannot be modified or deleted.

There are two types of immutability policies:

1. Time-based retention policies: User can set immutability-policy for a specified period of time. Under this policy, objects can be created or read but not modified or deleted. After the retention-period for this policy is expired that can be deleted but not overwritten.
2. Legal-hold policies: This policy stores data in an immutable-way until the legal hold is explicitly set. Under this policy, objects can be created or read but not modified or deleted.

Types of Data-Immutability policies [Source]

Comparison between the two Data-Immutability policies [Source]

The following table provides a summary of protections provided by container-level immutability policies:

Protections provided by container-level immutability policies [Source]

Points to Remember:

• Microsoft recommends enabling soft delete for additional protection before any immutability policies are applied, its adds extra protection and security.
• There is no additional capacity charge for using immutable storage.
• Creating, modifying, or deleting a time-based retention policy or legal hold on a blob version results in a write transaction charge.
• You cannot rename or move a blob when the blob is in the immutable state.
• Microsoft recommends that you configure immutability policies mainly for block blobs and append blobs.
• Storage account deletion fails if there is a container with at least one blob present with data-immutability policy enabled.
• With immutable storage once something is written, it cannot be changed or deleted.
• All blob access tiers support immutable storage.
• All redundancy configurations support immutable storage.
• The downside is, if you can’t delete the data, you’re stuck with storing it for some amount of time, which could bring additional costs.

Hands-On Implementation via Azure Portal & Python SDK for Azure

Prerequisites

• Python 3.6 or later is required to use this package
• You must have an Azure subscription and an Azure storage account to run the python code below.

Setup

• Install all the requirements Azure libraries for Python with pip:
• Clone or download this project repository: azure_blob_immutabilty
• Open the project folder in Visual Studio Code or your IDE of choice.
• From the root location of the project folder run the following command.

pip install -r requirements.txt

Workflow

1. In this workflow demo, I have firstly created a Resource group named “RG_Demo_Immutability” and further created a Storage account named “demo00immutability”.

Note: Remember to whitelist your IP in the “Networking” config settings of the storage account. Also, in the “Access Control (IAM)” config settings, add proper “role assignment” to yourself for successful execution of this demo workflow.

Storage account created where will apply immutability policy to the blob container

2. The script below demonstrates the usage of Python SDK for Azure for implementing the above said workflow i.e., immutability policy on Azure blob containers.

3. Before running the script, in the terminal of the IDE do the following steps:

• Log in to your Azure account

az login --tenant <tenant_id>

• Select the correct subscription

az account set --subscription <sub_id/sub_name>

[Info]: Now, the “_get_credential” method using “DefaultAzureCredential” library can do the authentication properly.

• After selecting the correct ‘Python Interpreter’ & correct ‘Configuration’ for the scope of your project like “Working Directory” etc. , run the script “blob_immutability_policy.py”.
• Following is the Python run-console with the workflow logs, please observed the highlighted text below.

Python console with work-flow logs

4. After the script is successfully executed, we can observe on the Azure portal side a container named “container-imt-policy” is created and an immutable storage policy of the type “Legal Hold” is applied with the tag value “tstimmutability”.

Implementation of Legal-Hold Immutability policy

Note: I have not shown the implementation of Time-based retention Immutability policy but only of the type Legal Hold. Also. this policy can easily be cleared via deleting the tags assigned.

Key Observation from the Workflow:

• With Legal-Hold Immutability at container level, the data in it can neither be deleted nor modified. But additional data can be created or uploaded in the container. Also this immutability policy can easily be cleared or removed to make container mutable again.
• With Time-Based Immutability at container level, which is a bit more strict policy, data can neither be deleted nor modified. But additional data can be created or uploaded in the container. Also with this policy with ‘locked’ mode can also not be deleted until the retention time expires, which mean also the container and parent resource i.e., storage account cannot be deleted.
• However meta-data of the container can be created, modified or deleted wr.t. both the policies enabled.