Camunda and Flowable — process and workflow automation platforms.
I don’t know about you but my life consists of chaotic “workflows” and “processes” that are full of surprises and unpredicted errors and just do not work as expected because I am a bit of a crazy person myself, forgetful, and unpredictable. However, in the business world this type of chaos is not acceptable and hence this is why businesses seek to automate their processes and ask software developers, like us, to provide a solution. Luckily, there are a lot of platforms out there to allow us to create robust, scalable, and manageable workflows that can also be represented visually so that any stakeholder (with or without technical knowledge) can have a clear representation of what is happening in the given workflow. This blog gives a brief introduction to automated workflow platforms: Camunda and Flowable, demonstrates how an automated workflow could be applied, identifies some best practices, and outlines challenges in utilising these platforms.
Camunda vs Flowable
As you can tell from the introduction to this blog, Camunda and Flowable are two open source platforms for workflow and business process automation that provide tools for the creation, management, and visualisation of workflows, business processes, and rules. To find out more about Camunda and Flowable you can visit their corresponding websites:
Both Camunda and Flowable are built on BPMN 2.0 standard (https://www.omg.org/spec/BPMN/2.0/PDF) which provides a graphical notation for workflows modelling, including business processes and rules, user interface, execution, monitoring, and optimization. Both Camunda and Flowable are forks of Activiti, which is a business process automation tool that is believed to be less versatile and powerful than Camunda. However, Flowable, in contrast to Activiti, has very similar features to Camunda. Camunda split from Activiti in 2013 and Flowable forked from Activiti in 2016 and is developed by the same engineers as Activiti.
User interface
Camunda provides a free modeller tool. It has web and desktop versions. Both versions enable you to model BPMN diagrams, DMN decisions, forms, and connectors to external systems without having to touch xml files. Should you wish to view or edit raw xml file, the tool allows you to do so too.
Camunda no longer supports Eclipse or IntelliJ plugins and encourages you to use Camunda modeller instead. Camunda modeller is a stand-alone tool and is not tied up to any IDE. If we set BPMN files to be associated with Camunda modeller in the operating system, then the BPMN files will be open in Camunda modeller from our chosen IDE. For example, if our preferred IDE is IntelliJ, we can right-click on the BPMN file and choose “Open With” to select Camunda Modeler as the default program to open that file. Then, every time we double-click on a BPMN file in IntelliJ IDEA, it will open in Camunda Modeler. For IntelliJ there is also a third party plugin that allows viewing and editing BPMN models. However, it is not possible to create a new Camunda model inside the IDE and we have to use Camunda modeller tool. Sometimes it can be inconvenient to jump from an IDE to Camunda modeller and back during development.
Flowable has user interfaces similar to Camunda. Flowable comes with Eclipse plugin – Flowable Eclipse Designer. So if our project is in Eclipse, it is very convenient and we don’t have to jump from Eclipse to an external tool to create a new model. There are other convenient features with the plugin, for example, it allows to autogenerate unit tests for Flowable diagrams. However, if we absolutely hate Eclipse with all fibres of our souls, then we are doomed! Well, maybe not so dramatically doomed, but we might be annoyed nevertheless. We could avoid Eclipse by using Flowable web application – Flowable modeller. In addition, Intellij IDEA has a Flowable BPMN visualiser plugin, which allows the user to view and edit Flowable diagrams in IntelliJ. But sadly, we still need Eclipse or Flowable modeller to create a new Flowable Diagram.
Both Camunda and Flowable user interfaces are highly customisable. For example, we can create custom elements, connectors, and shapes that represent things that are specific to our business domain.
In the case of Camunda modeller, it is possible to change the appearance by changing the css file (app.css) in the installation folder. In addition, Camunda allows adding plugins with different functionalities that can be developed or downloaded from GitHub. Custom plugins are developed with JavaScript and Camunda Modeller API. To load our custom plugin you need to specify the path to our plugin in the plugins section in the config.app.js file. We can also create custom element templates with predefined values. To create an element template in Camunda we need to create a json that implements Camunda Element Schema. To find out more about how to create an element template please refer to Camunda documentation about templates.
Similarly, to Camunda, Flowable allows extending the functionality of Flowable Designer. It is possible to extend the pallet and the output format. In contrast to Camunda which applies js plugins, Flowable Designer extensions are JAR files. To find out more about how to customise Flowable Designer please see Flowable documentation.
Flowable currently does not support element templates like Camunda.
Apart from modelling capabilities, both platforms provide UI for monitoring performances, task allocations, and administration. Camunda provides Camunda Cockpit for monitoring workflow and business processes performance, identifying and fixing the problem in business processes or decision tables, suspending processes, editing and deleting tasks, and migrating process instances to a new version.
Flowable doesn’t have a cockpit app like Camunda but it has different apps for single sign-on, role and access management (Flowable IDM), task management, including starting and stopping process instances (Flowable task), and administrative work that includes basic monitoring and changing business processes (Flowable admin). Compared to Flowable, Camunda allows better control and investigation capabilities for problems and their resolution within a single app.
Features
Both platforms provide support for modelling business processes (BPMN) and decision tables (DMN). And recently Camunda also included form creation and task allocation that Flowable also provides. We can create a form within our modeller and assign the form to a user task.
In addition, Camunda and Flowable support DRD – Decision Requirement Diagram. It allows modeling of multiple decision tables with a single DRD rather than using a process with multiple outcome rules to aggregate decision tables.
Flowable boasts of supporting and producing more features of the CMMN engine (Case Management Model and Notation) for unpredictable processes. Flowable claims that combining CMMN, BPMN and DMN in one tool allows developers to create problem models that are very human and event-driven. Camunda, on the other hand, stopped developing features based on CMMN engine. Switching CMMN on in Camunda modeller is possible, but by default, it is switched off. The reason behind it is that according to research made by Camunda team, only a few developers use CMMN. According to Camunda representatives, most processes are predictable and small deviations from predictable context can be modelled by subprocesses.
Flowable also allows dynamic process injection which allows process fragment injection on demand by AI or humans into a running instance. This is useful when we don’t want to rely on specific conditions to trigger a process but have an ad-hog decision to run a process to handle different circumstances. In addition, it allows simplifying the process model that is being injected because it no longer relies on conditions to be triggered. To find out more about dynamic process injection in Flowable please read their documentation.
At the time of writing of this blog, Camunda doesn’t support these features yet.
Integration with external systems
Camunda allows developers to create integrations with other systems with the help of an extensive range of Camunda REST APIs and connectors.
Camunda provides Cloud connectors which include Kafka streams, AWS, and Azure. For example, to connect BPMN service with AWS to invoke AWS lambda Camunda provides AWS lambda connector. Similarly, Camunda provides connectors for AWS Simple Queu Service for messages and AWS Simple notification service.
Other connectors include Business Connectors which include Salesforce and SWIFT, Productivity Application Connectors which include Microsoft teams and slack, Enterprise Content Connectors which allow to push content to or pull content from systems such as Box and Opentext, Data Connectors which allow to push data to or pull data from providers such as business intelligence systems, data lakes, and data warehouses.
In addition, Camunda allows the creation of custom connectors to integrate into an external service using Integration Framework. For more information about Camunda connectors including custom connectors please refer to their documentation .
Similarly to Camunda, Flowable provides ready-built integrations with external systems such as event stream integrations with AWS SQS, Kafka, Rabbit MQ, and Active MQ for event-driven business processes. In addition, Flowable provides third-party messaging integration including Whatsapp. It also has abstract data source integration and supports both relational and non-relational databases. However, Flowable seems to have fewer ready-made integrations to external systems like Camunda.
Deployment
Flowable is more flexible in terms of deployment than Camunda. Camunda can be deployed on the cloud and on-premises. While Flowable can also be deployed as a hybrid solution.
Language support
Both platforms provide support for different languages such as
- Java
- Groovy
- Kotlin
- JavaScript
- Python
In addition to these, Camunda also supports PHP and .NET.
Camunda or Flowable?
Ultimately the choice between the two platforms depends on the project and what the project needs. Typically, Flowable would suit a project that requires more flexibility in deployment, or/and has to address unpredictable contexts in business processes with dynamic process injections, and CMMN.
While Camunda would suit a bigger project with complex business processes and rules and integration with different external systems.
Use case: Dog behaviourist automated workflow
I have a very big 70kg puppy who recently started to be scared of certain dogs. To help my pup to cope with her fears I contacted a dog behaviourist. The dog behaviourist is expensive and popular and has a lot of new and existing clients contacting him about their dogs. Sometimes I wonder how he manages to reply to all these people and bill them accordingly. Maybe, just maybe, he has an automated process that does things for him…
As you can see from the diagram, the client asks for advice on their dog’s behaviour. This starts an “Advice Generation Process” workflow. The first step is to send the client’s question to AI. That service generates the answer. The answer is then customised accordingly and emailed to the client. Finally, an invoice for the services is created.
I should probably use ChatGPT directly instead of paying the dog behaviourist.
Some best practices
So we decided we have a use case for implementing an automated business process platform. Say we chose Camunda for it. We can use Camunda modeller and off we go creating our models. However, as we go along, our models might become difficult to understand, change, or maintain. This could be because we didn’t adhere to the best practices recommended by such platforms. Both Camunda and Flowable advise on best practices for the modelling and development of automated processes.
Readability
The model should be kept simple. Easy to understand and clearly labelled. Activities should be labelled in a descriptive way so it is clear to understand what they are about. Similarly, gateways should be named according to the naming conventions. For example, XOR gateways should be labelled as a question with mutually exclusive answers. Outgoing sequence flows or paths from the gateway should be labelled with corresponding answers.
All generated ids and labels should be replaced with coherent naming that includes ubiquitous language of the business process the model is representing, so that it is understood by all stakeholders (for example, developers and business analysts).
Models should be modelled systematically and from left to right. It is not recommended to use overlapping sequence flows where it is difficult to follow the directions of the flows. However, it is ok to use overlapping sequence flows going to one target in cases where it is clear to the reader where the flows are going.
Implicit BPMN constructs should be avoided as much as possible.
Single Responsibility
Did I mention that our process model should be as simple as possible? Ah, yes I did. Well, I will repeat myself – it should be as simple as possible. The process model should be responsible for solving only one problem. Activities within that process should be responsible for one thing and one thing only. Computing all data of the business process in one single step is an anti-pattern that should be avoided.
If a process is very complex it is possible to break it into subprocesses. This allows us to encapsulate certain process flows into meaningful chunks. Subprocess can be an embedded subprocess (a process that is embedded within a parent process and is accessible only by that process), an event subprocess (triggered by an event), or a global subprocess. Visually subprocesses can still be seen/expanded from the parent process that uses them.
Reusability
Some flows and steps in a process can be similar or the same as in other processes. As you probably guessed, for these cases global subprocesses can be created and reused by other processes. In our dog behaviour automated process system, invoicing clients or error handling can be subprocesses reused in different processes.
In addition, element templates are another way of reusing process steps by different processes. Element templates also ensure consistency among processes.
Graceful failures
There is always a possibility that something might go wrong during our process. For example, what if in our “Advice generation process,” there is a bug in the OpenAI integration system? We have to anticipate those failures and ensure our processes end in the way we want them. In the “Advice generation process” we can handle the error by emailing the client with some excuse, like “Sorry I am on holiday at the moment”, and aborting the process.
There are other steps that we could do in case of failures:
- roll back to previous transaction points,
- retry/wait a number of times,
- set global or local timers to the processes/ subprocesses/ activities,
- error boundary events, error event subrocesses
High Performance
Both Camunda and Flowable have built-in monitoring and reporting capabilities. It is highly recommended to use those regularly to monitor process instances and identify any bottlenecks and performance issues.
Challenges
BPMN graphical notations are xml files. They contain not only information on workflow, activities, and elements but also graphical representation such as the position of those elements. Therefore when changes are made by developers these are very difficult to code review. Especially if the developer didn’t adhere to best practices and didn’t give coherent labels and ids to activities, rules, and conditions.
Sometimes providing images of the diagram before the changes and after can help. However, it can also mislead if the image provided is not representing the actual changes made.
Similarly, it is difficult to merge conflicts as we need to identify which version of changes we need to leave, combine, or discard in an xml file that can be very complex and also contains information about position of different elements.
Camunda, Flowable, and other BPMN platforms are state-oriented and are very hard to maintain when flow and data for a particular state change. Typically, it is recommended to analyse the behaviour of the process in a new version with historical data to avoid significant bugs.
Changes can also affect performance. And therefore require monitoring, tuning configurations, and fine-tuning to ensure optimal performance.
Conclusion
This blog gave a brief introduction to workflow automation platforms Camunda and Flowable and attempted to demonstrate why we would use them. The platforms enable us to automate workflows and processes, task allocations, forms, and decision tables. Flowable is a more lightweight platform that can be used very efficiently with Eclipse. Camunda provides more enhanced monitoring and problem investigation and resolution capabilities than Flowable. The choice between the two platforms depends on the project and requirements.
Both platforms enable us to automate, execute, and monitor business processes. Camunda and Flowable also allow a better visual representation of what is going on in the process for all stakeholders.
However, in practice, implementing these platforms can be challenging. Especially when the processes require changes. Some challenges can be reduced or avoided by following recommended best practices provided by both platforms.
About the Author:
Tanya Maslova is a Java Developer here at Version 1.
