Unit testing of logs: sensitive data shall not pass!
As promised in the previous blog post, in this article I would describe techniques that could be used to ensure data, that should not be visible to everyone in the company, is not leaked into application logs.
In mid to large companies exist such data policies, that protect customer’s data to not be seen by non-authorized users. In that case, we as developers need to guarantee that such data is not part of the logs, that every developer could get access to. As an example of such information, I could think of: credit card numbers, phone numbers, first & last names, etc.
One approach could be to not print this data at all, but if we want to use it for troubleshooting, we could decide to go with the masking technique, which I would describe below.
Let's use as an example bank transaction service:
- we perform certain operations with a customer bank card
- in case the customer has several cards we need to know with which we do what, so we trace this information in logs
- that led to the leakage of sensitive data, as we described above
Let me show you a couple of approaches, from simple to more advanced, you could use in your application to prevent breaking data policies in your company.
Note: for the sake of this example, the technologies used were: Kotlin, Log4J2, Junit. But it could be transformed into any other language/stack of your choice.
Service that potentially could log sensitive data:
We would use log4j2 functionality to mask data, here it's card numbers.
As the first iteration, which suitable for simple cases, we could use standard PatternLayout replace functionality:
As we see it's working pretty well:
2020–04–27 11:09:36,851 INFO c.a.b.s.BankService [main] Started transaction
2020–04–27 11:09:36,855 INFO c.a.b.s.BankService [main] Performing transaction on card: 4012************
2020–04–27 11:09:36,855 INFO c.a.b.s.BankService [main] Finished transaction
In case we will need to add several different patterns to mask other customer-related data we could use our own Layout implementation.
It also supports embedded PatternLayout implementation.
Finally testing phase
To test our approaches really work we would use custom log4j appender, which would save for us all log events and for the sake of convenience print to console.
Unit test itself is pretty simple and short, but it could grow to a set of tests, depends on needs.
After integrating this testing into our application we ensure to comply with company policies and could sleep well at night.
Full repo could be found here