Protect Against Accidental Logging of Sensitive Information

This article introduces a novel way of handling sensitive data in the structural part of a Perl web application. Quite clearly, it does not cover the required infrastructure and protocol to store, retrieve, monitor and otherwise secure its storage. Such mechanisms are outside the scope of the article.

booking.developmentBlockedUnblockFollowFollowing

Jan 2, 2013

Written by Steffen Müller

It won’t come as a surprise to anybody that an e-commerce web site such as Booking.com has to deal with sensitive data such as Personally Identifiable Information, PII. At Booking.com we treat such information with the utmost care. For our taste, it is too easy for programmers to make mistakes that accidentally cause some bit of information to be written to an application log that he didn’t intend. Specifically web server error logs may contain stack traces that may in turn contain some sensitive bit of information. The following technique can help prevent such mistakes[1] from happening in the first place.

Consider that you have code that handles data that should never end up in your logs.

use Carp;
foo("This does not belong in logs");
sub foo {
    my $sensitive = shift;
    this_can_die();
}
sub this_can_die {
    Carp::confess("Gotcha"); # stack trace
}

If you’re logging errors, you now get this in your logs:

Gotcha at /path/to/yourapp.pl line 8
    main::this_can_die() called at /path/to/yourapp.pl line 5
    main::foo('This does not belong in logs') called at /path/to/yourapp.pl line 2

Yikes! In particular, if you aggregate your error log using some concentrator, then the wrong bit of information was just leaked to another (potentially less closely monitored) system. Not cool. And it doesn’t even take a huge mistake. It’s a fairly subtle flaw that can easily bite you. Various techniques can be used to fix this. The easiest one is simply using hash-based objects to store this info and pass it around. In general, you’d pass it around as some sort of reference to prevent this:

use Carp;
my $secret = 'This does not belong in logs';
foo(\$secret);
sub foo {
    my $sensitive = shift;
    this_can_die();
}
sub this_can_die {
    Carp::confess("Gotcha"); # stack trace
}

This now produces the following log output:

Gotcha at /path/to/yourapp.pl line 9
    main::this_can_die() called at /path/to/yourapp.pl line 6
    main::foo('SCALAR(0xa7e9c0)') called at /path/to/yourapp.pl line 3

Much better. If your developers, however, make an actual, not-so-subtle mistake of leaving debugging statements in as in the following snippet, then you’re back in the land of trouble.

sub foo {
    warn Data::Dumper->Dump(\@_); # FIXME just for debugging
    my $sensitive = shift;
    this_can_die();
}

That bit of code will log $VAR1 = \'This does not belong in logs\';. We can continue this game. Instead of using a "normal" data reference, you could use inside-out objects, generate a closure that refers to your sensitive data and so on. At that point, I'll bring up the more powerful Data::Dump::Streamer[2] (in conjunction with PadWalker and B::Deparse) next. Even inside-out objects can accidentally be dumped if you're using Data::Dump::Streamer to dump their methods. Admittedly it's becoming progressively less easy to make a mistake such as the above, but why bother with the arms race against developers' creativity in debugging mistakes?

The CPAN module Store::Opaque was written to take care of this kind of accident once and for all. This module implements an opaque object representation that does not suffer from these issues. Let me repeat the all-important warning: This isn't about hiding anything from an attacker who can run code. It's about preventing mistakes by people who have legitimate access. Here's how you use Store::Opaque:

package MyClass;
use Moose;
extends 'Store::Opaque';

sub get_sensitive_info {
    $_[0]->_get("sensitive_info")
}

sub set_sensitive_info {
    $_[0]->_set("sensitive_info", $_[1])
}

__PACKAGE__->meta->make_immutable;
no Moose;

And then simply use it like any other object:

use MyClass;
my $info = MyClass->new;

$info->set_sensitive_info(1234567234567);
my $number = $info->get_sensitive_info;

# Not even DDS will compromise the data now:
use Data::Dump::Streamer;
Dump($info); # FIXME debug

The Store::Opaque implementation is a really rather trivial XS module that hides a hash behind an opaque (to Perl) C pointer. You can store any Perl data structure in this hash and it will resist any accidental dumping!

[1] Note that we explicitly refer to mistakes here, not malicious activity. This is not about security, it’s about preventing accidental leaking of information from a secure to a less secure system, not about active security measures.

[2] See the Data::Dump::Streamer documentation for information about how Data::Dump::Streamer can dump closures, for example.