Building distributed apps using Dapr, locally and on Azure

In this article, I’ll introduce Dapr for building distributed applications. We’ll look at the different features offered, the components that can be utilized and finally, look at a working example that I built to summarize and consolidate the knowledge in this article.

Photo by Amanda Phung on Unsplash

What is Dapr?

From the website,

Dapr helps developers build event-driven, resilient distributed applications. Whether on-premises, in the cloud, or on an edge device, Dapr helps you tackle the challenges that come with building microservices and keeps your code platform agnostic.

To put it more simply, Dapr allows you to build distributed applications without having to worry about how the different services would talk to each other, what configurations to make for setting up state stores, pubsub, secret store or for observability, among other things.

This means that any developer, irrespective of the choice of language and framework, can very quickly and easily build apps across a cluster, focusing more on the business logic and less on the infrastructure woes.

Why is it important?

There are many challenges that come with building distributed apps. I’ll list a few of them below and we’ll see how Dapr comes in and solves these problems for a developer.

Limited choice of languages
Some platforms for building apps have a narrow language support and as such, developers have little choice in language selection.

With Dapr, you can be coding in any language and still use all of the features and services provided. This is because Dapr building blocks can be accessed using an HTTP or gRPC API. Clients exist for languages like Go, Python etc. or you can directly make the network calls too.

Keeping up with modern practices is hard
A common problem with implementing a full solution on your own is upgrading the architecture when new approaches are discovered. As such, there’s a gap in the existing code and best practices.

Dapr codifies the best practices for building microservice applications into open, independent, building blocks. Each building block is completely independent and you can use one, some, or all of them in your application.

Limited portability
A lot of the time, code written for any app relies on specific infrastructure environment. This dependency limits the portability of the app and any change would require a rewrite, which is undesirable.

Dapr is platform agnostic meaning you can run your applications locally, on any Kubernetes cluster, and other hosting environments that Dapr integrates with. This enables you to build microservice applications that can run on the cloud and edge.

All the three points summarized

How does Dapr achieve all of that?

Sidecars

Dapr has a sidecar architecture. Every service that you run with Dapr gets a sidecar attached to it. This unit is responsible for communicating with other sidecars to facilitate actions like calling methods on their respective services, or communicating with statestore, pubsub or input/output binding components.

The application code only talks to the sidecar that is attached to its environment. All inter-service calls are handled by Dapr, securely through the sidecars.

Dapr sidecar architecture

For registering a service with Dapr, we have two ways, depending on the choice of deployment. The image below shows a sample for enabling the sidecar.
In Kubernetes, you add a few annotations (in bold) and for local deployment, you can use the Dapr CLI to run your app, which automatically registers a sidecar for your application.

Registering a sidecar in Dapr

Building blocks

A very crucial part to Dapr are its building blocks. Each of them have been built according to the best practices and are ready to be used, with little configuration.

All building blocks follow a similar architecture for use:

1) There’s an endpoint that you have to call, which specifies what feature you’re calling and where the Dapr server is.

2) A set of options of components that you can choose from for that building block. These are components that are either built by Dapr or the community and you can plug those into your application using a simple YAML file.

I recommend reading through the official documentation for building blocks before proceeding to aid comprehension.

In any case, you’ll learn more about the building blocks by seeing them in action below!
Using an example I built which is also listed under Dapr’s official samples repository, I’ll demonstrate service invocation, pubsub, state management and output bindings.

Distributed Calendar

This is a sample application built using Dapr as a proof-of-concept. I have used multiple languages for writing the different parts of this calendar app, but the bulk is in Javascript (NodeJS). This demonstrates the language-agnostic nature of Dapr and the flexibility that it brings to developing applications.

Contents

• Motivation
• Setup
• Architecture
• How to run
• Test using Postman

Motivation

I had built a SpringBoot app on MVCS architecture before; it was a monolith application, all written in Java. Building a roughly similar architecture as a distributed application would intuitively require some additional work pertaining to service discovery, inter-pod communication and network security. Things could get complicated if I needed additional checks, state stores or other controls which I would have to implement on my own. This, in addition to the actual application itself.

I wanted to find out how Dapr simplified this process and what additional work I would have to put in to get a distributed version of the same application using Dapr.

Setup

Dapr can be run across a range of platforms. You can choose to run it on Azure or similar platforms or on your local machine. The code remains the same and only slight modifications in the YAML configuration is required to make it run anywhere. This ensures that your application code is unaffected by your choice of deployment.

Note: Using Dapr requires some knowledge of Kubernetes concepts like configuration files, pods and lots of other minute details. If you’ve never worked with distributed systems or orchestration platforms, I have a hands-on course that is built to get you started with cloud computing. Find it here and videos at my channel here!

We’ll develop this app locally and then I’ll guide you on how to very quickly deploy it to Azure.

The following steps will get you started with using Dapr locally:

• Install the Dapr CLI

wget -q https://raw.githubusercontent.com/dapr/cli/master/install/install.sh -O - | /bin/bash

You can verify the installation by typing dapr on your terminal, which should produce an output like the following:

Output of dapr command

You can use the guide here as reference for getting started.

For running on Azure, we ‘ll use the Azure Kubernetes Service (AKS):

AKS is Azure’s managed Kubernetes service. It offers

• Elastic provisioning of capacity without the need to manage the infrastructure and with the ability to add event-driven autoscaling and triggers through KEDA
• Faster end-to-end development experience through Visual Studio Code Kubernetes tools, Azure DevOps and Azure Monitor
• Most comprehensive authentication and authorization capabilities using Azure Active Directory, and dynamic rules enforcement across multiple clusters with Azure Policy

If you want to get started with Kubernetes, you should check out this amazing resource, built to help you learn in just 50 days!

50 day course from Microsoft Azure

For almost all scenarios pertaining to product development, you can leverage the power of AKS to make things simpler.
You can use it for building complete end-to-end machine learning pipelines, run data streaming jobs, build microservices, and much more!

Steps to follow to run the Dapr sample on Azure:

• Create an AKS cluster.
• Install kubectl on the environment that you plan to use to connect to your cluster. You can connect using the Azure command-line tool, as follows:

az aks get-credentials --resource-group myResourceGroup --name myAKSCluster

After the second step, follow the steps from the local installation.
Finally, clone this repository so that you can follow along.

Once the steps above are done, we can move ahead with designing and running our app.

Running the App

The project has three different apps, in Go, Python and Node. We have to build them and then utilise the dapr run command to start these apps.

First, make sure that your component definitions are present under $HOME/.dapr/components if you're on Linux and under %USERPROFILE%\.dapr\components if you're using Windows. This is because the dapr run command makes use of the YAML definitions provided here at runtime.

Verify that components are present.

You can find the component definitions I’ve used in this project under components of the root directory. On an Azure Kubernetes Service (AKS) cluster, you can directly perform:

kubectl apply -f components/statestore.yaml

to deploy the respective component.

The password and api keys from the YAML definitions have been removed and will need to be provided at runtime. On an AKS cluster, you can make use of Secrets for these values. Check out secret stores component of Dapr!

After the components are created with the correct fields, we can build and run the individual apps.

In a Kubernetes environment, put the commands under each of the headings inside a Dockerfile, one for each of the services. These will be used to create an image to be run inside your cluster.

A sample Dockerfile for Node.JS : here
A sample YAML config file: here

Node

• Install dependencies.

npm install

• Start Dapr.

dapr run --app-id controller --app-port 3000 --dapr-http-port 3500 node node_controller.js

Go

• Go inside the go directory and build the project Make sure you have gorilla/mux package installed. If not, run the following command:

go get -u github.com/gorilla/mux

• Build the app.

go build go_events.go

• Run Dapr

dapr run --app-id go-events --app-port 6000 --dapr-http-port 3503 ./go-events

Python

• Install required dependencies.

pip3 install wheel python-dotenv flask_cors flask dapr

• Set environment variable for Flask.

#Linux/Mac OS:
export FLASK_RUN_PORT=5000
#Windows:
set FLASK_RUN_PORT=5000

• Start Dapr.

dapr run --app-id messages --app-port 5000 --dapr-http-port 3501 flask run

Architecture

I have tried to model this system on the Model View Controller Service (MVCS) architecture, as already mentioned.

Architecture showing the different components involved.

Controller (written in Javascript)

• The controller supports creation of new events and deletion of existing events. It forwards these requests to the Go code using service invocation. Shown below is the add event flow.

where the invokeURL is defined as:

const invokeUrl = `http://localhost:${daprPort}/v1.0/invoke/${eventApp}/method`;

• On creation of a new event, it publishes a message to a pubsub topic which is then picked up by the Python subscriber.
  Publishing to the topic

where the publishURL is:

const publishUrl = `http://localhost:${daprPort}/v1.0/publish/${pubsub_name}/${topic}`;

Building blocks used:

• Service invocation: Detailed guide here. Try to relate that guide to the code above.
• PubSub: Detailed guide here.

Reminder to keep drinking water.

Services

The services handle the requests forwarded by the controller. Each of the tasks listed with the controller is handled by a service written in a different language. I’ll detail the implementation below.

• Event Service (written in Go): This service uses the statestore component Redis for storing and deleting events from memory. The code snippet shown below is from go_events.go and demonstrates adding an event to the state store.

where the stateURL is defined as:

var stateURL = fmt.Sprintf(`http://localhost:%s/v1.0/state/%s`, daprPort, stateStoreName)

• Messaging Service (written in Python):

This service subscribes to the topic that we post messages to, from the controller. It then uses the SendGrid output binding to send an email about creation of a new event. I have used the Dapr client for Python while writing this service.

The code below shows how the service registers as a subscriber with Dapr for a specific topic.

The Dapr runtime calls the /dapr/subscribe endpoint to register new apps as subscribers. The other way to do this would be defining a configuration file, linked here.

The following code receives the message posted to the topic and then calls the send_email function.

The send_email functions calls the SendGrid binding with the message payload:

where invoke_binding is a library function from the Dapr client. In the previous cases, we had called the endpoints directly; here we use a function already implemented for us.

Building blocks used:

• Output Bindings: Detailed guide here.
• PubSub: Detailed guide here.

And with that, the explanation of all the code is complete! Congratulations, if you made it here 🎉🎉

Test using Postman

Posting a new event to the controller:

• Postman client used to send request body at /newevent endpoint.

• The Dapr logs inside the controller showing the data that is being passed to the Go app for persisting in storage.

• The Dapr logs inside the go-events app showing the data received along with the status response.

There are more tests on the repository, if you are interested!

Conclusion

We first understood the common challenges facing developers while building distributed apps and then saw how Dapr attempts to solve those.

We also found out that Dapr achieves all that it does using a sidecar architecture and a range of building blocks. We looked at a working example to better comprehend the working of the building blocks!

For folks who prefer to watch me explain all of the stuff above, I took a session for the Microsoft Student Ambassadors in January ’21 that might help.

Feel free to reach out to me in case you have any questions 💖

LinkedIn and Twitter