Clean Architecture and use case-driven workflow with Camunda 8

Combining DDD, Clean Architecture, and BPMN engine

There is a GitHub repository where you can find complete source code for the example application which we are going to discuss below.

CamundaCon 2022

While I am writing this, Camunda is holding it CamundaCon 2022 conference. One speaker, Luc Weinbrecht, just gave an excellent presentation about exactly the same topics: Camunda 8 and Clean Architecture. You’ll find a link to the GitHub repository he mentioned in this presentation in the references section at the end of this article.

The idea

Camunda team recently released version 8 of their workflow engine based on Zeebe. It is an excellent product which integrates wonderfully into Spring ecosystem. It is also pretty simple to test with Docker Compose. All this has made me curious about how one can design and implement an application using a BPMN workflow engine for process modelling, but adhering to the principles of Clean Architecture and DDD, as well. I would like to share with the community some of my ideas and questions, which I describe in this article and which are also reflected in the example application.

At first glance, we have an apparent contradiction here: BPMN stands for “business process modeling”, meaning that it must be used to model business, in some way. On the other hand, DDD, is all about modeling business too. So, in a project which uses a workflow engine and which tries to leverage domain-driven design, how are we to make sure that these two models: the one described by the BPMN and the one we have in the middle of our hexagon, do not contradict each other or get in the way of each other? There is a way to resolve this contradiction. It is here that Clean Architecture comes to the rescue. But first, let us see what do we gain by combining BPMN and DDD together.

TripFlow example, BPMN

To illustrate the ideas and concepts in this article, I’ve implemented an example application using Camunda 8 (self-managed) and Spring Boot. It address a canonical problem often used as an example of modeling a workflow: it is a trip planning problem. Here are the most important specifications for the application.

• A trip planning proceeds in steps which must be executed in the well-defined order.
• First, a customer selects a flight to the desired destination city.
• Then, the customer, selects among several hotels available in the destination city.
• Afterwards, the customer reviews the summary for the trip with the total of the expenses related to the cost of the flight and the cost of the hotel reservation.
• System, then, checks if the total price of the trip does not exceed the credit limit for the customer.
• Finally, the trip is confirmed or refused depending on the outcome of the credit check.
• At any point during the trip’s booking process, the customer may consult a list of all trips created by her where the booking has not been completed yet. She must have a possibility to continue with any task for any of the open trip bookings.

We can immediately see that these specifications lend themselves particularly well to a design based on workflow modeling. We have well defined steps, a logical sequence. We also have two actors (“customer” and “system”) with clearly defined roles and responsibilities. Here is how this trip booking workflow may be modeled in BPMN. I’ve used Camunda Modeler for this, of course.

Trip booking BPMN workflow

So BPMN modeling gives us a robust and formal way to define invariants related to the process flow of the application and the involved roles and responsibilities of each actor. For example, we can see that hotel reservation must be done after booking the flight. But we could have modeled a car rental step in parallel with hotel reservation.

Workflow modeling has another great point. It allows domain experts to participate in the high-level system design. BPMN diagram can serve as a map of business related activities to be implemented by the application.

All of the above points are not readily apparent, when considering a pure DDD approach. We may model flight, hotel reservation, and credit limit as domain entities and aggregates to the perfection. Yet, we won’t be able to clearly see the sequence of the required steps by just looking at the domain model. Of course, there is the “Application Services” layer (and to some extent the “Domain Services” layer) to help us with that. But this is, already, somewhat implementation specific and goes beyond the domain proper.

DDD and business rules

This being said, domain-driven design, still remains the best approach when it comes to analyse and model a problem space. We still need to develop a ubiquitous language which will solidify our understanding of the nature of the entities, value objects and relations involved. What is exactly a trip plan? What is involved in a hotel reservation? Is there a natural identifier which can uniquely identify a flight? How can we construct an immutable and reusable value object representing a price with a specific currency?

These all are important questions for a DDD practitioner. But there is more. Imagine that we have modeled the trip booking as an aggregate with the aggregate root entity — Trip and we have to confirm the trip. By confirming, I mean, we essentially have to set some boolean flag to true , like here:

A very legitimate question can be asked when it comes to the check of the invariants of Trip aggregate in confirm operation. Strictly speaking, we must first check that flight and hotel were booked before setting the “confirmed” flag. That is because, in theory, trip.confirm(); instruction may be called anywhere in our application (before a flight has been booked, for example).

Now, imagine a relatively complex application with dozens of these booleans and enums which must be set according a strict combination specified by the business rules. Enforcing aggregate invariants may become very tedious and repetitive. But what if we are using a workflow engine to impose a strict sequence of steps, such that a call to trip.confirm() will be guaranteed to never happen prior to the calls to methods for booking a flight and reserving a hotel. And that guarantee will, of course, come from a careful design of the process workflow.

Sequence of calls to mutator methods imposed by the workflow

If this is the case, our confirmation method on Trip aggregate is greatly simplified. We don’t need to worry about a hotel not being booked, for example, since by the very fact that we are in the step “confirm trip”, means that we must have passed by the step “reserve hotel”. And that step must have assigned a hotel (reservation) to our Trip instance. We are delegating enforcement of some aggregate invariants to the workflow engine instead of the aggregate itself. Yes, from the stand point of the aggregate, the rule become an implicit one. And, this is against a rule-of-thumb in DDD. But it is explicitly stated by the BPMN diagram and followed by the corresponding flow of control during execution.

Use case-driven workflow

As you see from the discussion above, workflow modeling with BPMN and domain modeling with DDD are different but complimentary aspects of design. They can nicely fit together and can even help in the overall application design. But that is only if Clean Architecture is chosen as the overall architectural pattern when designing our implementation. There are a lot of resources on the web specifically addressing the complementary relation between Clean Architecture and DDD. So I won’t dwell on that. We’ll look here at the way Clean Architecture and the use of a workflow engine can fit together nicely.

In my opinion, Clean Architecture (in contrast with simply Hexagonal Architecture) focuses on these specific important points:

• It centers around the Use Case layer.
• It implies a specific flow of control which most often starts from primary adapters (usually Controllers) going to use cases (through input ports), to secondary adapters (through output ports), going back to use cases, and, finally, to Presenters (secondary adapters of a special kind).

There is more, of course, but these are particularly relevant to our discussion. Indeed, conceptually speaking, a step in the workflow (modeled as a user task or a service task in BPMN) corresponds nicely to a notion of a use case or an interactor (to use the nomenclature by Robert C. Martin).

But we do have two distinct runtimes: the workflow engine executing our BPMN and the the client application executing our logic according to the rules of DDD and Clean Architecture. So what is missing here is the actual ways these two communicate. Diagram below will give an overview of how this communication takes place in TripFlow. I invite the reader to consult the source code if any of the points are not readily clear.

Side note: I am particularly thankful to the team at Camunda with whom I have had a chance to discuss some of these implementation points. They have cleared for me some technical aspects of Camunda Platform 8 architecture concerning the way JobClient can communicate with Zeebe efficiently. Their GitHub repository was especially helpful as an example.

Control flow from BPMN engine to the hexagon during execution of a user task

Primary adapter and Zeebe worker

Camunda 8 provides a way to declare a Spring component as a worker which will subscribe to the Zeebe gateway and listen to the events of the workflow execution. It also provides a client which can be used to communicate back to the workflow engine. First thing to notice in the diagram above is that I’ve separated the functionality of Zeebe workers and JobClient in two distinct adapters.

Side note: the different types of adapters is very well explained in this article by Herberto Graça.

Primary adapter using Zeebe worker is shown below.

Primary adapter with Zeebe worker

When the workflow engine activates a user task, see (1) on the diagram above, ZeebeJobHandlingAdapter will receive a notification containing an instance of ActivatedJob with some important information. We need to store this information somewhere. This is because the rest of the application may at some point later request this information in order to proceed with the usual control execution flow (UI view to MVC controllers, etc.). So (2) shows that the adapter will convert the job’s information into a special domain object and use the persistence gateway to store it in the application database.

Modeling workflow tasks

I use a special single entity aggregate, TripTask which models a user task or a service task exactly as any other aggregate in our domain model. You see how in this case we also respect the imperative of DDD: in our application there are different models one for a Trip and one for TripTask representing very different real-world concepts but still parts of the same domain model.

Side note: I am using here an the screenshots from Simple Monitor and DBeaver tools.

Modeling workflow task as an aggregate, TripTask

So, for a user task, Zeebe job handling adapter will limit itself by storing some information about the activated workflow job in the database. It is important to realize what kind of information we need from each instance of an ActivatedJob .

• taskId : unique job key. Will be used as the ID of TripTask .
• tripId : ID of the corresponding Trip aggregate root, corresponds to the process instance key. This is a link between TripTask and Trip .
• tripStartedBy : username of the user of the application who started the trip
• action : an input parameter from a user task which contains a path to the MVC controller responsible for servicing the task. This is the link between the execution of the workflow and the execution of the control logic in the rest of the application.
• name : user-friendly name of the task. This will be shown in the list of all tasks remaining to be completed for the user.

There could be some other useful information about the workflow task that we may need to model and store as well. For example, a custom header in the job which conveys the information about the assignee of the task or the candidate groups of the task may need to be persisted as well. But we need to be as parsimonious as we can. The information about other domain objects such as Flight and Hotel , as well as, some attributes of Trip should not be present in the variables of the workflow. This is where DDD design should be utilized to the utmost.

MVC controller and use case

Continuing with the flow in the diagram above, (3) shows that at some point in time and after the Zeebe worker has finish execution, the control of the application will arrive at some MVC controller servicing the UI for the user task. This will heavily depend on the nature of the client application. In our case, TripFlow leverages Spring MVC with Thymeleaf templates. The important point is that a specific controller will be called by the client application depending on the value of action attribute of the user task at hand. Another important point is that the MVC controller needs to have taskId parameter supplied to it. This parameter will reference the right instance of TripTask aggregate related to the user task.

From this point on, we are following Clean Architecture control flow: (4) shows that the use case calls the output port for the gateway to get the needed TripTask and, then, the related Trip that it needs to work on.

Secondary adapter and JobClient

Once the work of the use case is done, which can actually mean several screens on the client application later, the user task in the workflow has been completed as well. At this point we need to signal back to Zeebe the completion of the user task. This is the role of the secondary adapter: ZeebeClientOperationsAdapter , (5).

The secondary adapter will send an appropriate command to Zeebe signaling that the job (identified via taskId ) should be completed: (6). There is one more thing, which the adapter needs to take care of at this point and this is: the TripTask entry in the table of activated tasks needs to be removed. This is the reason behind a call to the gateway (7). Note that a use case can decide to set any variables (output) for any of the user tasks or service tasks when completing the job. In TripFlow, the use case checkCredit will use this to set the boolean sufficientCredit with the result of the check for the sufficient credit of the customer.

How to advance to the next task

At this point, we have finished our user task and the BPMN engine has happily advanced to the next user task (if any available) going through any intermediate service tasks (if any) in the process. At the client application we are about to finish our use case and are ready to present the next view to the user. But how do we know where to redirect the control next? If a next user task becomes available at some point, it will be registered by ZeebeJobHandlingAdapter and stored in the database. All this is happening asynchronously with respect to the request processing on the client application. So the client application needs to poll the database until some activated user task becomes available. In TripFlow this happens with the following method in the gateway.

It uses Spring’s RetryTemplate , configured with some sensible defaults, to retry a query for any TripTask with matching tripId and tripStartedBy which represent the next activated job in the workflow engine. Of course, at some point, the use case will abandon and simply signal to the presenter that the user should be redirected to the generic view listing all tasks to be performed by the user.

Discussion

I invite the reader to check out the source code for TripFlow application. It is runnable using simple Docker Compose command. It is interesting to see how the workflow advances for a particular trip booking process at the same time the user goes through the application.

Here is a couple of ideas for extending the example application:

• Add a new user task, like “rent a car”, try to put it after “reserve hotel” or parallel with it. You would, have to add the data in the database, create a new use case and a Thymeleaf view. Do not forget about action , name , and candidate groups attributes for the new task in the BPMN.
• Try to see what happens if you throw an exception in a particular part of the code: in Zeebe adapters, in a use case, in the gateway.

I leave you with with a look at the title image of this article which summarizes my main point. Yes, DDD, Clean Architecture, and a workflow can be put together in the same project. But we have to be careful to let DDD drive the domain modeling, to let BPMN model the process, and to use Clean Architecture to put it all together. Thank you for your attention.

Update

I have recently updated the version of Camunda Platform used by the example discussed in this article. Please, check the the full source code in the GitHub repository as it may differ with some of the code presented in the snippets above.

There is a wonderful article by Bernd Rücker: “Navigating Technical Transactions with Camunda 8 and Spring”. It discusses in details a very important aspect related to the dealing with business transactions and JobWorker in a client applications for Camunda. I strongly encourage the reader to check it out. In the example of TripFlow application, we are conforming to what Bernd is referring to as “Scenario C” in his article.

References

• TripFlow example
• “Camunda DDD and Clean Architecture”, Luc Weinbrecht, GitHub repository
• Lowcode UI template example, Camunda Community Hub
• “Ports & Adapters Architecture”, Herberto Graça
• “Clean Architecture”, Robert C. Martin
• “Navigating Technical Transactions with Camunda 8 and Spring”, article by Bernd Rücker