Azure Storage Development with Azurite — Part II Testing

7 min readApr 8, 2024

The first part was about setting up Azurite and using it for local development, and how the Azure Storage Explorer can be helpful in interacting with the storage. In this second part we will use Azurite as a testcontainer for medium size Unit and Spring Integration tests.

In the first part we were using Azurite as an Azure Storage replacement for local development, and we used the Azure Storage Explorer to inspect and interact with blobs and queues. Now we are using Azurite as a testcontainer in order to write medium size Unit and Spring Integration tests. The example project shown here is provided on Github.

Setting up a Testcontainer

I assume you are familiar with testcontainers, if not, the Getting Started and the Guides are a good starting point. In short, they allow you to set up external dependencies like databases, messaging middleware etc. as a Docker Container in order to run integration tests against it. To run Testcontainers-based tests, you need a Docker-API compatible container runtime as a prerequisite.

To get started we will write a base class which sets up the testcontainer and provides some useful other stuff needed by the actual test.

class AzuriteContainer : GenericContainer<AzuriteContainer>("mcr.microsoft.com/azure-storage/azurite:3.29.0")

abstract class AzuriteTestcontainer()  {

    companion object {
        val container = AzuriteContainer().withExposedPorts(10000, 10001, 10002)
...
        init {
            container.start()
        }
...
    }

}

So we are creating a testcontainer using the image mcr.microsoft.com/azure-storage/azurite, and expose the ports 10000, 10001 and 10002 which are the ports for the blob-, queue- and table-storage respectively. In the init block of the companion object we start the container. So the container will be started just once if you are running a multiple tests, so it is quite cheap after all.

Testcontainers do not map the given ports right away to the host system, but use random ports. This is done to avoid port collisions, see more details in the network section of the documentation. So in order to provide the correct ports to our tests, we add the following code:

val blobUrl by lazy {
    "http://${container.host}:${container.getMappedPort(10000)}"
}
val queueUrl by lazy {
    "http://${container.host}:${container.getMappedPort(10001)}"
}
val tableUrl by lazy {
    "http://${container.host}:${container.getMappedPort(10002)}"
}

We retrieve the mapped port — means the random port actually chosen — for the corresponding original port. Instead of localhost (or 127.0.0.1) we also use the term container.host. In local environments this usually is localhost, but on CI systems it may be a different host. For our convenience, we also provide the resulting connection strings

val blobConnectionString by lazy { "DefaultEndpointsProtocol=http;AccountName=${accountName};AccountKey=${accountKey};BlobEndpoint=${blobUrl}/devstoreaccount1;" }
val queueConnectionString by lazy { "DefaultEndpointsProtocol=http;AccountName=${accountName};AccountKey=${accountKey};QueueEndpoint=${queueUrl}/devstoreaccount1;" }
val tableConnectionString by lazy { "DefaultEndpointsProtocol=http;AccountName=${accountName};AccountKey=${accountKey};TableEndpoint=${tableUrl}/devstoreaccount1;" }

AzureBlobStorage Test

Now we will write a test for the AzureBlobStorage, which provides access to the blobs in a given container; let’s recap the implementation:

@Service
class AzureBlobStorage(val blobServiceClient: BlobServiceClient,
    @Value("\${demo.storage.blob.container}") val containerName: String
) {

...

    fun listBlobNames():List<String> = ...
     
    fun downloadBlob(blobName: String): ByteArray = ...

So our service class needs a BlobServiceClient and the name of the container as constructor parameters, which will be injected by Spring at runtime. Our Unit Test will set up the service and therefore needs the appropriate objects, which will be created right away using the blobConnectionString provided by our AzuriteTestcontainer base class:

class AzureBlobStorageTest : AzuriteTestcontainer() {

    val container = "test"
    val client = BlobServiceClientBuilder()
        .connectionString(blobConnectionString)
        .buildClient()
    val blobContainerClient = client.getBlobContainerClient(container)
    val storageService = AzureBlobStorage(client, container)

    @BeforeEach
    fun setup() {
        if (blobContainerClient.exists()) {
            blobContainerClient.listBlobs().map { it.name }.forEach {
                blobContainerClient.getBlobClient(it).delete()
            }
        }
    }

Since we are reusing the same Azurite instance for all tests, we have to properly clean up everything before each test.

Now we are ready to write the first tests, let’s start with the listBlobNames() function. It is expected to return a list with the names of the blobs in the given container.

@Test
fun listBlobNames() {
    storageService.listBlobNames() shouldBe emptyList()

    blobContainerClient.getBlobClient("a").upload("aaa".byteInputStream())
    blobContainerClient.getBlobClient("b").upload("bbb".byteInputStream())
    blobContainerClient.getBlobClient("c").upload("ccc".byteInputStream())

    storageService.listBlobNames() shouldBe listOf("a", "b", "c")
}

Initially we expected the service to return an empty list, as there are no blobs in the container. We manually add some blobs using the blobContainerClient created for our test setup, and now expect a list of the blob names. If we run the test, the base class will start the Azurite container, you will find log statements on that like the following. At the end of the test suite the Ryuk container that is started by Testcontainers core will take care of stopping the singleton container.

20:31:04.822 ... Creating container for image: mcr.microsoft.com/azure-storage/azurite:3.29.0
20:31:06.235 ... Container mcr.microsoft.com/azure-storage/azurite:3.29.0 started in PT1.413205S

So if we run our test it sets up the our AzureBlobStorage using the connection string and our test will run against Azurite testcontainer instance, and…. tada, green :-) This test is not a big deal and you may wonder if all this is overkill, but when it comes to testing edge cases and failure handling, testing against the real thing by utilizing testcontainers is quite valuable.

Spring Integration Test

In our AzureBlobStorageTest we didn’t use any Spring magic, we created our setup in order to connect to Azurite by hand. In order to test our MessageReceiver we will write a Spring Integration Test, which will start up a complete Spring Boot Application with all the configuration and injection magic that Spring performs.

@ActiveProfiles("azurite")
@SpringBootTest
class MessageReceiverTest:AzuriteTestcontainer() {

    @MockkBean()
    lateinit var messageHandlerMock: DemoMessageHandler

    @Autowired
    lateinit var storageQueueTemplate: StorageQueueTemplate

    val messageSlot = slot< Message<*>>()

    @BeforeEach
    fun setup() {
        clearAllMocks()
    }

    @Test
    fun `receive message`() {
        every { messageHandlerMock.handleMessage(capture(messageSlot)) } just Runs

        storageQueueTemplate.send(DEMO_QUEUE, MessageBuilder.withPayload("Yeehaw").build())

        val message = messageSlot.awaitMessage()
        message.text() shouldBe "Yeehaw"
    }
....

Due to the SpringBootTest annotation the complete Spring application will be started during the test. We demand to use the azurite profile, so our application under test will be set up using the configuration data provided in application-azurite.yaml. The we mockk our DemoMessageHandler so we can setup custom behavior in our test, resp. in this case we will set up the mock handler to capture the message passed, so we can inspect it.

In order to test our message handler, we send a message to our demo queue, wait for the handler to receive the message, and check if the content is the same. If you run the test you will notice that it takes quite a bit more time than our other tests. This is due to the fact that the complete application ist started which will be reflected in the logs. Run the test, and…red…eh? If you look at the logs you will find something like:

Connection refused: /127.0.0.1:10002

Ah, ok. We used the azurite profile for our test, and in the application-azurite.yaml the configuration for azurite is set up to localhost:10000–10002…but — as already mentioned — the testcontainer uses random ports. So how do we get those ports into our test setup? We will use DynamicPropertySource to inject (resp. overwrite) the values in the Spring environment in order to use the correct hosts and ports, by adding the following code to our AzuriteTestcontainer base class:

abstract class AzuriteTestcontainer()  {

    companion object {
        ...

        @JvmStatic
        @DynamicPropertySource
        fun registerAzuriteProperties(registry: DynamicPropertyRegistry) {
            registry.add("demo.storage.table.endpoint") { "${tableUrl}/devstoreaccount1" }

            // overwrite endpoints using the testcontainers ports
            registry.add("spring.cloud.azure.storage.blob.endpoint") { "${blobUrl}/devstoreaccount1" }
            registry.add("spring.cloud.azure.storage.queue.endpoint") { "${queueUrl}/devstoreaccount1" }
        }

Now run the test again, and….tada, green :-) In productive code we would add some more tests in order to check failure- and timeout-handling etc.

Playtika Spring Boot Testcontainers

If you write medium sized SpringBootTests like our latter one, it is worth to have a look at the https://github.com/PlaytikaOSS/testcontainers-spring-boot project. It provides out-of-the-box testcontainers with autoconfiguration for Spring Boot. So you just need to add the dependency, and the testcontainer will be ready to be used in your SpringBootTest. And they have images for all kinds of stuff: databases of all flavors, mailserver, messaging and cloud stuff of all three major vendors. So just give it a try. As an example we will now rewrite our MessageReceiverTest in order to make use of the Playtika variant of the Azurite testcontainer:

@ActiveProfiles("playtika")
@EnableAutoConfiguration
@SpringBootTest
class PlaytikaMessageReceiverTest {

    @MockkBean()
    lateinit var messageHandlerMock: DemoMessageHandler

    @Autowired
    lateinit var storageQueueTemplate: StorageQueueTemplate

    val messageSlot = slot< Message<*>>()

    @BeforeEach
    fun setup() {
        clearAllMocks()
    }

    @Test
    fun `receive message`() {
        every { messageHandlerMock.handleMessage(capture(messageSlot)) } just Runs

        storageQueueTemplate.send(DEMO_QUEUE, MessageBuilder.withPayload("Yeehaw").build())

        val message = messageSlot.awaitMessage()
        message.text() shouldBe "Yeehaw"
    }

This variant is quite similar to our first MessageRetrieverTest, except for the following points:

It does NOT inherit from a common base class that deals with the container set up.
It uses a dedicated profile playtika
We have to enable auto-configuration

So where is the container setup done? This is done by to Spring auto-configuration, provided by the dependency we are adding in our build.gradle.kts. The first adds the embedded Azurite which does the setup of the container (and some more stuff), the second is needed to get things working, see the documentation for information on that:

testImplementation(("com.playtika.testcontainers:embedded-azurite:3.1.5"))
testImplementation("org.springframework.cloud:spring-cloud-starter-bootstrap")

The embedded-azurite provides the azurite specific host and ports in some dedicated properties in the Spring environment. We just have to wire them up, that’s we are using a dedicated profile application-playtika.yaml:

demo:
  storage:
    table:
      endpoint: ${embedded.azurite.table-endpoint}

spring:
  cloud:
    azure:
      storage:
        blob:
          account-name: ${embedded.azurite.account-name}
          endpoint: ${embedded.azurite.blob-endpoint}
          account-key: ${embedded.azurite.account-key}
        queue:
          account-name: ${embedded.azurite.account-name}
          endpoint: ${embedded.azurite.queue-endpoint}
          account-key: ${embedded.azurite.account-key}
          queue-name: testqueue

Just run the test and you will see quite the same output as our original MessageReceiverTest produced. As already said you will find images for all kinds of stuff like database etc., perfectly prepared and ready to run, very nice. A drawback is, that it is built to be used in full fledged SpringBootTests, but you may not use it in more light-weight tests as we did in our first example.

Conclusion

Azurite can be quite helpful when it comes to developing applications that uses the Azure Storage APIs, be it as a local replacement in order to run your app, or for writing medium sized or even full fledged Spring Integration Tests by using Azurite as a testcontainer. Have an eye on the Playtika Spring Boot testcontainers if you intend to write integration tests in general, as they provide out-of-the-box solutions for all kinds of external dependencies, that are easily integrated by auto-configuration. And check out the Azure Storage Explorer if you have not already used it yet; it is a free tool that comes in handy, just give it a try.

Experience is a hard teacher because she gives the test first, the lesson afterward.
Vernon Law