Azure Storage Development with Azurite — Part I

6 min readMar 25, 2024

Developers use containers for local development and testing, be it databases of all flavors, authorization server, messaging middleware or whatsoever. This article will show the usage of Azurite, a lightweight server clone of Azure Storage. The first part will be on setup in order to use the BlobStorage, TableStorage and Queue-APIs, the second part will be on testing using Azurite testcontainer.

When you develop applications for Azure Cloud you might have used the Blob Storage for easy persistence of unstructured resp. binary data. The API is quite easy, you may access the data via URL, even without security. Maybe you have also used Storage Queues or the Table Storage, which is a fast solution for accessing tabular data by key. Developing against these APIs is straightforward, all you need is some credential, preferably an identity. But if you develop against a storage account you have to share the storage, which means you have to manage access so you don’t interfere with each other.

But currently we are used to access our external dependencies like databases by utilizing Containers: just start it on your local machine and run your application and tests agains it. This has a lot of advantages: you do not interfere with others, and you do not even need an internet connection, so you can develop wherever you are, be it on the train or in the middle of nowhere. There are images for all kinds of stuff that makes local development easy: databases of all flavors, messaging middleware…and there is Azurite, a storage emulator with support for Blob Containers, Queues and Tables.

In this first article we will use Azurite to develop a Kotlin Spring Boot sample application, the second part will show how to run medium size Unit and Spring Integration Test with an Azurite testcontainer. We will also use the Azure Storage Explorer, a tiny UI based tool that makes introspection of the storage quite easy. The source code is provided on Github.

Starting Azurite

The Azurite open-source emulator provides a free local environment for testing your Azure Blob, Queue Storage, and Table Storage applications. There are multiple choices to run the emulator:

We will utilize the docker image, just run the image:

docker run -it -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite

2024-03-06 05:52:14 Azurite Blob service is starting at http://0.0.0.0:10000
2024-03-06 05:52:14 Azurite Blob service is successfully listening at http://0.0.0.0:10000
2024-03-06 05:52:14 Azurite Queue service is starting at http://0.0.0.0:10001
2024-03-06 05:52:14 Azurite Queue service is successfully listening at http://0.0.0.0:10001
2024-03-06 05:52:14 Azurite Table service is starting at http://0.0.0.0:10002
2024-03-06 05:52:14 Azurite Table service is successfully listening at http://0.0.0.0:10002

So all we gotta do is map the ports, 10000 for the BlobStorage, 10001 is the TableStorage and 10002 the StorageQueue. If you want your storage data to persist just mount the volume /data

docker run -it -p 10000:10000 -p 10001:10001 -p 10002:10002 -v ./data:/data mcr.microsoft.com/azure-storage/azurite:3.29.0-arm64

Connecting to Azurite

In order to connect to the azurite storage we need to provide an endpoint, an account-name and an account-key. The account-name is set to devstoreaccount1, also the account-key is fixed, you will find them both in the documentation. Since we are running the container on our local machine, the endpoint resolves to

http://127.0.0.1:<port>/devstoreaccount1

In this demo we are going to use the package spring-cloud-azure-starter-storage, in order to configure it, we will provide a dedicated stage application-azurite.yaml containing the needed configuration data for the blob storage:

spring:
  cloud:
    azure:
      storage:
        blob:
          account-name: devstoreaccount1
          endpoint: http://127.0.0.1:10000/devstoreaccount1
          account-key: "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
          # this is a public documented key for testing purposes with azurite, see https://learn.microsoft.com/de-de/azure/storage/common/storage-use-azurite?tabs=visual-studio%2Cblob-storage#connect-to-azurite-with-sdks-and-tools

Don’t mind to check in the account-key. Since it is a well known public secret, even Github security won’t complain.

The spring-cloud-azure-starter-storage lib will automagically setup a BlobServiceClient which will be injected in our service class AzureBlobStorage. Our demo application uses this class to upload a blob to the storage.

Be aware that this demo is stupid simple: no timeout, retry handling etc., it is stripped down to the bare bones. Don’t do this at work ;-)

@Service
class AzureBlobStorage(val blobServiceClient: BlobServiceClient,
    @Value("\${demo.storage.blob.container}") val containerName: String) {

    private val blobContainerClient: BlobContainerClient by lazy {
        blobServiceClient.createBlobContainerIfNotExists(containerName) ?: blobServiceClient.getBlobContainerClient(
            containerName
        )
    }

    private fun getBlobClient(blobName: String): BlobClient {
        return blobContainerClient.getBlobClient(blobName);
    }

    fun updateBlob(blobName: String, byteArray: ByteArray) =
        getBlobClient(blobName).upload(BinaryData.fromBytes(byteArray), true)

    fun downloadBlob(blobName: String): ByteArray = 
            getBlobClient(blobName).downloadContent().toBytes()
  ...
}



@Component
class StorageAccessDemo(val blobStorage: AzureBlobStorage,
...
    fun blobStorage() {
        val blobName = "testblob"
        logger.info("creating blob '$blobName'")
        blobStorage.updateBlob(blobName, "Here we go".toByteArray(Charsets.UTF_8))

        logger.info("reading blob '$blobName'")
        val content = blobStorage.downloadBlob(blobName).toString(Charsets.UTF_8)
        logger.info("content is: $content")
    }

If we run the demo we will get the following log output

StorageAccessDemo: creating blob 'testblob'
StorageAccessDemo: reading blob 'testblob'
StorageAccessDemo: content is: Here we go

Azure Storage Explorer

Ok, log output is quite fine, but I want to see the real thing, er… blob. This is where the Azure Storage Explorer enters the stage. It is neat little UI based tool that let’s you inspect and alter the content of an Azure Storage. And this works both for your real Azure Storage accounts as well as for your local Azurite instance. Just download and start it, and you will be connected to your azurite instance right ahead. So let’s have look at it: in the Emulator section you will find Blob Containers, Queues and Tables of your Azurite instance. In the Blob Containers node you will find our testcontainer containing our testblob. Doubleclick on the blob will download the content, and we will find our expected content:

Sending Messages via Storage Queues

Our little Demo also contains a handler that receives message from a storage queue named testqueue. This is implemented using spring-cloud-azure-starter-integration-storage-queue. The configuration is quite similar to the blob config, here is an excerpt from the application-auzurite.yaml:

spring:
  cloud:
    azure:
      ...
        queue:
          account-name: devstoreaccount1
          endpoint: http://127.0.0.1:10001/devstoreaccount1
          account-key: "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
          # this is a public documented key for testing purposes with azurite, see https://learn.microsoft.com/de-de/azure/storage/common/storage-use-azurite?tabs=visual-studio%2Cblob-storage#connect-to-azurite-with-sdks-and-tools
          queue-name: testqueue

You can look up the configuration of Spring Integration in the Github repository; let’s have a look at the message handler:

@Component
class DemoMessageHandler() : MessageHandler {

    override fun handleMessage(message: Message<*>) {
        logger.info("New message received: '${message.text()}'")
  
        // checkpoint the message, aka acknowledge
        message.checkpoint()
    }

The handler just logs the incoming message and checkpoints it (aka acknowledge). You can use the Storage Explorer to inspect the messages on the queue, and also to send and delete messages. Let’s test our message handler by sending a message to our testqueue:

Right after adding the message our Demo client receives the message and logs it :-)

DemoMessageHandler: New message received: 'Hi there, anybody listening?'

So the Storage Explorer comes quite handy when you have to deal with Blob-, Queue- and TableStorage, be it locally on Azurite or in the real cloud, just give it a try.

Conclusion

So this first part gave you a brief introduction on how easy it is to use Azurite as a local replacement for Azure Cloud Storage for development. The Azure Storage Explorer is neat tool that helps you to inspect and manipulate the storage — both Azurite and Azure Cloud — in order to interact with your application. In the next part we will use testcontainers for writing Unit tests for our Demo application.

In terms of long-term durable storage, the human mind, paradoxically, is pretty good, but it’s very fragile.
Jonathan Nolan