How to secure your Azkarra Streams application for production
--
One week ago we announced Azkarra Streams, a new open-source micro-framework for simplifying the development of Kafka Streams applications.
Azkarra Streams provides developer-friendly features such as an embedded HTTP server to expose metrics and states about the local Kafka Streams instances. It also provides the ability to query Kafka Streams state stores. Last but not least, it offers a Web UI to manage the application and to visualize the DAG of streams topologies.
If you are interested in learning more about Azkarra, check out our “Getting Started”.
Why Azkarra needs to be secured?
Security is one of the most important concerns (if not the most important) when it comes to deploying an application in production.
First, Azkarra allows you to access information about the running Kafka Streams instances (metrics, configuration, states, etc) via the REST APIs. Even if the secrets are not visible, some non-critical data may provide information, considered to be private, about your application. For example, it could be the Kafka topics that are consumed or used internally by your application.
In addition, the state stores, of your streams application, can be queried and, depending on your business nature, the data returned by the application may be sensitive.
Finally, the REST APIs enable you to perform some operations like stopping and restarting a Kafka Streams instance, or to dynamically deploy a new Kafka Streams instance for an existing topology. While some of these operations can be useful during development or for some production scenarios, it can be also necessary to not allow them.
For all the above reasons, we decided to implement the mechanisms required to protect the REST APIs, from the first release of Azkarra.
Azkarra is composed of three standard components for security :
Encryption on the wire: This allows data transmitted between an Azkarra application and a client application or between two Azkarra applications (i.e when using interactive queries) to be encrypted over the network.
Authentication: This allows client applications or an Azkarra application, accessing the REST APIs, to authenticate to the Azkarra application.
Authorization: This allows you to allow or deny your client applications to perform some operations or to access specific REST resources.
In the rest of this article, we will explore the different mechanisms that you can use to secure Azkarra.
Headless Mode
Azkarra has a headless mode that you can enable by using the server configurationazkarra.server.headless=true.
The headless mode doesn’t disable the REST APIs but instead will return a 401 Unauthorized HTTP response for POST, PUT and DELETE requests. The only exception to this rule is that you can still query the state stores of your application.
As of Azkarra 0.4, you can also disable the Web UI by using the server configuration server.enable.ui .
Basic Authentication
When it comes to secure access to an HTTP service, Basic Authentication is usually the simplest mechanism to implement.
You can add HTTP basic authentication to Azkarra by configuring the following properties :
azkarra.server.rest.authentication.mechanism="BASIC_AUTH"
azkarra.server.rest.authentication.realm="AzkarraServer"
azkarra.server.rest.authentication.roles="admin, alice"Then, you can configure the credentials of authorized users in two ways.
Simple configurations
The first approach is to set a UserIdentityManager using the configuration azkarra.server.user.identity.manager.class . Azkarra only provides a simple implementation namedInMemoryUserIdentityManager. This class accepts a single configuration property rest.authentication.users that must contain the list of users separated by a comma.
Below is a complete code example using a programmatic configuration style :
@AzkarraStreamsApplication
public class BasicAuthenticationExample {
public static void main(final String[] args) {
final Conf serverConfig = ServerConfBuilder.newBuilder()
.setAuthenticationMethod(
SecurityMechanism.BASIC_AUTH.name()
)
.setAuthenticationRealm("AzkarraServer")
.setAuthenticationRoles("admin, alice")
.setUserIdentityManager(
InMemoryUserIdentityManager.class
)
.setAuthenticationUsers(
"admin:admin-secret, alice:alice-secret"
)
.build();
new AzkarraApplication()
.setConfiguration(AzkarraConf.create("application"))
.enableHttpServer(true, serverConfig)
.run(args);
}
}JAAS configurations
Azkarra also supports the Java Authentication and Authorization Service (JAAS) and provides a custom LoginModule for loading users from a properties file.
You can create a JAAS file azkarra-server-jass.conf as follows :
AzkarraServer {
io.streamthoughts.azkarra.http.security.jaas.spi.PropertiesFileLoginModule required
file="/etc/azkarra/azkarra.password"
reloadInterval="60"
reload="true"
debug="true";
};Note : This is important that azkarra.server.rest.authentication.realm matches the section with the JAAS file.
Then, create a properties files containing users identification information :
admin:MD5:fa0deb5e70ca0c27c04b717a9c60d657,Administrator
alice:alice-secret,DeveloperNote, that password can be passes as a hash MD5 (e.g : echo -n “admin-secret” | md5sum).
Finally, the JAAS config file location must be passed as JVM parameter. For example:
-Djava.security.auth.login.config=/path/to/azkarra-server-jaas.confWeb UI
The Azkarra Web UI still works even with basic authentication. A login modal will open to ask you for your credentials.
Encryptions using TLS (or SSL)
Because Basic Authentication doesn’t protect credentials transmitted over the network, you will typically use it in conjunction with TLS (SSL)
SSL can be simply enable by configuring the following properties :
azkarra.server.ssl.enable=true
azkarra.server.ssl.keystore.location="/path/to/server.ks.pkcs12"
azkarra.server.ssl.keystore.type="PKCS12"
azkarra.server.ssl.keystore="password"
azkarra.server.ssl.key.password="password"azkarra.server.ssl.truststore.location="/path/to/server.ts.pkcs12"
azkarra.server.ssl.truststore.type="PKCS12"
azkarra.server.ssl.truststore="password"# only for development
azkarra.server.ssl.ignore.hostname.verification=true
Below is a complete code example using a programmatic configuration style :
@AzkarraStreamsApplication
public class SSLAuthenticationExample {
public static void main(final String[] args) {
final Conf serverConfig = ServerConfBuilder.newBuilder()
.enableSsl()
.setKeyStoreLocation("/path/to/server.keystore.pkcs12")
.setKeyStorePassword("password")
.setTrustStoreLocation("/path/to/server.truststore.pkcs12")
.setTrustStorePassword("password")
.setTrustStoreType("PKCS12")
.setIgnoreSslHostnameVerification(true)
.build();
new AzkarraApplication()
.setConfiguration(AzkarraConf.create("application"))
.enableHttpServer(true, serverConfig)
.run(args);
}
}Client Certificate Authentication
Azkarra also supports authentication using client certification (also called SSL Two-Way Authentication). This allows Azkarra to verify the identity of the client by requesting the client to issue a trusted certificate (i.e: a certificate signed by a certificate authority).
To be enable, you have to add the following properties :
azkarra.server.rest.authentication.mechanism="CLIENT_CERT_AUTH"
azkarra.server.rest.authentication.roles="*"or :
.setAuthenticationMethod(SecurityMechanism.CLIENT_CERT_AUTH.name())
.setAuthenticationRoles("*")Here we use a wildcard(*) to define any roles. This will authorize any authenticated user .
Authorization
Azkarra implements a very simple and pluggable authorization mechanism via the interface AuthorizationManager to ALLOW or DENY a user to perform an operation (GET, POST, PUT, etc) on a resource (e.g. /streams/:id).
You can configure the implementation to be used by the security chain using the server property server.authorization.manager.class. Azkarra provides a default implementation class so- calledSimpleAuthorizationManager.
The SimpleAuthorizationManager allows any user whose principal or role is present in the list of roles defined by the server propertyazkarra.server.rest.authentication.roles to perform any operation.
In addition, you can also restrict a user using the server configuration auth.restricted.roles. A restricted user will only be allowed to perform GET HTTP requests.
You can provide your implementation, for example, to allow a user to only query some state stores.
Conclusion
Although Azkarra is a recent open-source framework, it already provides the necessary functionalities to secure your Kafka Streams application before deploying it in production. In addition, its pluggable authorization mechanism allows you to easily implement custom policies.
About Us :
StreamThoughts is an open-source technology consulting company. Our mission is to inspire companies to create ever more innovative services that make the most of the opportunities offered by real-time data streaming.
We deliver high-quality professional services and training, in France, on the Apache Kafka ecosystem and Confluent.Inc Streaming platform.
