UI for Kafka part 2 (with LDAP/AD)
This is a continuation of an earlier blog on writing a UI for Kafka. In the part 1 blog, i described the Kafka Web UI we created which can use SASL accounts as login to connect to Kafka. This allows your organization to define ACLs, to allow users to view messages only inside a certain subset of topics. But most organizations want their users to login with LDAP/AD accounts and don’t want a SASL account to be shared by multiple users. In this part we will discuss how the Kafka Web UI handles this.
As mentioned in our earlier blog we are using SASL accounts as our logins by having them stored as a file or environment variables and then the user logins with these SASL accounts and they are verified against these store accounts. Once verified these are used to connect to the downstream Kafka server. The UI makes calls to the server using JWT tokens from the SASL account userId. You can then create ACLs to restrict the view permissions of a particular SASL userid. Now if we want to login through LDAP, first of all Kafka Web UI needs to allow LDAP logins. This can be done by enabling the LDAP settings mentioned here.
Now the user will be able to login using his organization’s LDAP account instead of SASL account. This allows Kafka admins to run a more secure Kafka, since they don’t have to share sensitive SASL accounts(username/passwords) with multiple users. But these LDAP accounts can not connect to Kafka on their own, as Kafka is running on a SASL mechanism (PLAIN/SCRAM). So the Kafka Web UI needs to translate the LDAP account into a SASL account.
It’s a bad idea to translate each LDAP user to a SASL account. LDAP users are under things called LDAP groups. We need to use the LDAP groups to do this translation. So the flow should look something like this.
The above approach should work if you name your LDAP groups exactly the same as the SASL Userids. One thing to mention is that LDAP account retrieval is case insensitive. So you should have your SASL accounts all lower case.
This can work fine if a single user is always mapped to a single LDAP group. But problems will arise if a user is mapped to more than one group.
In this case Kafka UI will think user will belongs to Group A and show him only topic A, instead of all three topics.
To overcome this we have to follow following approach to list topics.
When the message service is called to list messages inside topic, we will again have to try to hit Kafka in loop for every SASL account (continue on TopicAuthorizationException) and break out of the loop if we get back messages.
With this Kafka Web UI can now allow employees in your organization to use their AD/LDAP logins to view messages inside Kafka. Kafka admins can restrict who can see what using ACLs set to SASL accounts, which exactly match these Group names(all lower case). The environment variables you need to do this are specified here.
