Mendix and Kafka — Using mTLS (Part 1)
Once in a while, as a Mendix developer, you come across something stimulating. Something you’re sure can be done, but at the same time, something you haven’t done before. In my previous project, I had such a moment myself and I’d like to take you through my discoveries and approach. It contains several components that will only be relevant to some, but I hope everyone can learn something from it.
This blog will be about Kafka. When the customer asked if we could start publishing events from our Mendix application to an existing Kafka cluster, the immediate answer was “Yes, of course we can!”. The Mendix Marketplace offers multiple modules that facilitate this need. As the Kafka module provides the most features, including full SSL support, I picked this module to start with.
https://marketplace.mendix.com/link/component/105878
For this implementation, there were two challenging requirements:
- The connection should be secured with mTLS.
- The messages should be serialized as Apache Avro.
Personally, I was already familiar with certificates in general (mostly client certificates) and event-driven architectures and services such as AWS SNS. As these services are primarily REST-based, sending Avro serialized messages was new for me. Also, the more ‘raw’ Kafka implementation was something I did not implement myself before.
This blog is primarily about the mTLS connection. Stay tuned for the follow-up on the Avro serialization and connector modification. Once it is available, I will add the reference here.
About Apache Kafka
Apache Kafka is an open-source distributed event streaming platform that LinkedIn initially developed. It is designed to handle high volumes of data in real time and has become a popular choice for building scalable, fault-tolerant, and highly available applications.
At its core, Kafka is a publish-subscribe messaging system that allows producers to publish messages to a topic, which one or more consumers can consume. It stores messages in a distributed, fault-tolerant way, making it highly reliable and resilient to failures. Kafka is highly scalable and can handle millions of events per second, making it ideal for use cases such as real-time analytics, log aggregation, and (IoT-) stream processing.
When using Mendix, your application can be either a consumer or a producer. For this customer, the Mendix application will only publish events.
Challenge 1 — Secure connection with mTLS
mTLS (Mutual Transport Layer Security) is a security protocol that is used to establish secure communication between two endpoints over a network. In mTLS, both the client and server are required to present a digital certificate to verify their identity. This mutual authentication ensures that both parties are who they claim to be and that the communication channel is secure.
mTLS is commonly used in microservices architectures and distributed systems, where multiple services must communicate securely. It provides an additional layer of security beyond simple username and password authentication and helps to prevent man-in-the-middle attacks, where an attacker intercepts and modifies communication between two endpoints.
Luckily, the mentioned Kafka module from the Mendix Marketplace already offers mTLS support out of the box. The main challenge here was converting the received PFX files to a keystore and a trust store. In this blog, I will give you the final steps I executed.
Converting PFX to keystore and trust store
To make the mTLS connection work, we must create a keystore and a trust store. In short, a trust store holds trusted certificates to verify the identity of others, while a keystore stores an entity’s private key and certificate to prove its own identity during secure communication.
To convert the PFX file, we will use OpenSSL and the Java Keytool. If you start with another file, the required commands might differ. To keep this blog clean, I won’t explain how to install OpenSSL. The Java Keytool can be found in %JAVA_HOME%\bin and is part of the JDK you also need to run Mendix.
If you are new to this topic and can’t find the folder; in Studio Pro, go to Edit and then Preferences. This page will also display the JDK being used by Mendix.
First, we convert the PFX file to PEM. The PEM file will contain the certificate chain we want to add to the new trust store.
A PFX (Personal Information Exchange) file is a type of digital certificate containing both a private and public key. It is commonly used to store and transfer certificates between different systems securely. PFX files are typically password-protected, which helps to prevent unauthorized access to the private key.
As mentioned before, we will use OpenSSL. OpenSSL is an open-source library for encryption and secure communication widely used to protect data on the internet. It’s widely used for encryption, digital signatures, and SSL/TLS protocol implementations. For our PFX file, we had to use the command below. So, let’s install OpenSSL, open CMD as an Administrator (if you’re using Windows, like me), and execute the command.
openssl pkcs12 -in cert.pfx -legacy -nokeys -out cert.pem -nodes
It’s worth mentioning that this actually might not work on your machine with your PFX file. For example, a colleague had to specify the path to the legacy provider manually. His command looked like the comment below.
openssl pkcs12 -in cert.pfx -nokeys -provider-path “C:\YOUR_PATH” -provider legacy -out cert.pem -nodes
You might encounter issues if you’re new to using command-line tools like OpenSSL. For example, OpenSSL should be declared properly in your PATH environment variable to make the command work. If not, Google can be your friend here. Specifying the path to both your OpenSSL executable and PFX file will also work.
As we now have the certificate chain in a PEM file, we could open it in any text editor to see what is in there. We will now add the certificates from the PEM file to the truststore. Make sure to match the file names with your situation. Replace the $PASSWORD placeholders with both the PFX- and key password. When you receive the PFX file from an external party, they can provide you with the corresponding passwords as well.
keytool -keystore kafka.client.truststore.jks -alias CARoot -import -file ca-cert.pem -storepass $PASSWORD -keypass $PASSWORD -noprompt
Please note that we only use the CARoot in this example. When doing this, we trust all requests signed by the same Root Certificate Authority. Depending on your use case, you might need to add the Intermediate Certificate Authority too. I used ‘kafka.client.truststore.jks’ as the file name. This is not fixed and can be changed freely.
After I executed the steps above, the connection was still not working. The PEM file was needed on the receiving end as well. As a developer, we had to upload the certificate to a self-service portal. Eventually, it turned out that we had to remove some attributes that were added in between the certificates. I strongly believe there are more sophisticated ways to do this (for example, by tweaking the OpenSSL command while exporting), but for now, this was the shortest route to success. Most likely, this issue was very specific to our setup. As many ‘weird’ challenges can occur while working with SSL, I wanted to share them with you.
As we now created the truststore, we will also create the keystore. It’s worth mentioning that the PFX file is a keystore on its own. As I had to manipulate the file to create the truststore, I generated two JKS (Java KeyStore) files to keep things consistent.
keytool -importkeystore -srckeystore cert.pfx -srcstoretype pkcs12 -destkeystore kafka.client.keystore.jks -deststoretype JKS
Now, we should be good to go. Add the Kafka module to your project and follow the documentation. When you add your server details, go to the SSL tab. Here, you can upload both the trust- and keystore. Afterward, move to the ‘Configuration details’ tab and configure the following attributes to match your mTLS setup.
- security protocol = SSL
- ssl keystore type = JKS
- ssl truststore type = JKS
When using the PFX file as a keystore directly, you must set the keystore type to PKCS12.
That’s it! If using mTLS is the only thing you’re here for, you should be good to go. Did you use the blog as a reference, but do you think something is missing? Feel free to reach out via the Mendix Community Slack. I would be happy to help.
In the upcoming blog, I will tell you more about the Avro serialized messages and how I made this work in Mendix. Make sure to watch my feed if you’re looking for this specifically or if you’re interested in using custom Java with Mendix in general.
Sources / Read more
How Certificate Chains Work — https://knowledge.digicert.com/solution/SO16297.html
Video SSL Setup for Kafka Clients Tutorial — https://www.youtube.com/watch?v=dtNmWiNlyl4
What Is Apache Kafka — https://developer.confluent.io/what-is-apache-kafka
From the Publisher -
Inspired by this article to bring your ideas to life with Mendix? Sign up for a free account! You’ll get instant access to the Mendix Academy, where you can start building your skills.
For more articles like this one, visit our Medium page. And you can find a wealth of instructional videos on our community YouTube page.
Speaking of our community, join us in our Slack community channel. We’d love to hear your ideas and insights!
