Log4j Vulnerability in Tableau — How to Fix / Workaround [unofficial]

We’ve all heard the bad news about the log4j vulnerability, as even services like Amazon S3, Minecraft and Apple iCloud have been impacted in the last few days. Tableau — as it some of its parts were written in Java — has been impacted too.

Update 12/16: Tableau released an official Knowledge Base article with patched versions and a supported workaround: https://kb.tableau.com/articles/issue/Apache-Log4j2-vulnerability-Log4shell

In short:

If you do not patch your Tableau Server or apply a workaround, everyone who has network access to your Server can gain admin access to the application and tableau service user access to the underlying operating system.

While Salesforce is busy working on a patch, with a simple config change, you can disable some of log4j’s problematic behavior. I will show how.

I learned this method from Merlijn yesterday, and it is as useful as it is scary. The easiest way to test if you are vulnerable is to use huntress.com’s free service that gives you a string token that you can pass on in your application’s input forms for testing purposes. If your app is vulnerable, you will see a new connection in huntress.com’s connection panel:

How to test your Tableau Server with Huntress.com (method by Merlijn Buit)

Here, my Tableau Server is shown to be vulnerable, because when I pass this magic string as login user (with a random password), the website registers a new connection. Please note that if your Server is behind a firewall you need to open up that port for testing.

To fix (or work around) the issue until Salesforce releases a patch the two most common procedures are:

  1. Set up a special environment variable that disables this functionally. This only works with newer tableau server versions (the oldest version I tested successfully was 2021.1.3 but it was reportedly not working on 2020.2).
  2. Remove the problematic piece of code from the installation. This is more complex (and potentially dangerous) solution but it works on all Server versions.

Workaround 1: Environment variable based

The easiest way to disable this behavior is to set LOG4J_FORMAT_MSG_NO_LOOKUPS environment variable to true. This does not protect from all attacks, since:

However, CVE-2021-45046 is only a Denial of Service attack — while it can crash your services, at least your infrastructure cannot be hacked remotely. (CVE-2021–45046 issue applies only for specific custom logging configurations and does not allow remote code execution).

Update 12/15: it seems this workaround does not work on older Tableau Servers such as 2020.2. I tested it on version 2021.1.3 and it worked.

Open a terminal and execute the following commands with root permission:

The first command adds the LOG4J_FORMAT_MSG_NO_LOOKUPS to interactive shells (like protecting your tsm command) while the second line creates a new environment file 20-log4j.conf in Tableau’s systemd configuration folder. This systemd config file will be used by all Server components.

After executing this on all worker nodes, reboot the servers to make sure all services are picking up the new configuration.

Similar to Linux, you should add this environment variable (LOG4J_FORMAT_MSG_NO_LOOKUPS) as a system-wide setting, then reboot all nodes in your cluster.

Set new system variable LOG4J_FORMAT_MSG_NO_LOOKUPS to True in System Properties/Environment Variables

Seeing is believing, please test it with huntress.com or with other ways to make sure the fix works on your servers too.

Workaround 2: Delete all JndiLookup class

This is a more complex but comprehensive solution that works for older Tableau Servers as well.

To be on the safe side, we should delete all JndiLookup class files from all jar files. For this we can use a handy tool called CVE-2021-44228-Scanner. You can download it from here, it has binaries for Linux and Windows too.

On Linux, you can download it by:

To use it, simply execute:

With root/administrator user. Then, restart your server and validate if you don’t have any vulnerable files on the system.

How to test if you have been compromised

Just search Tableau Server log files for a jndi string.

Zak Geis’s thread on Twitter

For Windows, you can run this to find logs that contain the jndi: text (you might need to change the folder to your actual Tableau Server logs directory):

On Linux, simply use:

If you see anything like this, then you’ve been compromised:

However, a simple search might not reveal all attempts as log4j allows other ways to specify jndi locations, thus the string could be:

In other words, you should check your logs really carefully and look for different permutations.

Next steps

First and foremost, it is always highly recommended that you apply Tableau’s patches when they become available. You should continuously monitor Salesforce communications here: https://status.salesforce.com/generalmessages/826

There are other ways to mitigate this issue: some of them are temporary but do not require restarting Tableau Server, while others (patching the log4j packages on the file system) will protect you from CVE-2021–45046 too. A good summary of potential actions are collected here: https://www.lunasec.io/docs/blog/log4j-zero-day-mitigation-guide/

Hey, I am just trying to help on a best effort basis! So, whatever happens with your system, I take absolutely no responsibility whatsoever. These methods are working properly, but you know, your Server, your choice.

Credits

Information from this post came from Lorant Bellus, Zak Geis, Merlijn Buit Mark Edwards and Ken Flerlage.

Tamas is co-founder and CTO of data services firm Starschema where he leads the Starschema technical team to deliver results for the most innovative enterprises