When Your Problem Isn’t Your Problem

In a recent post, I wrote about my experience crafting the PJL Industries mission statement. I’d like to follow up that post with an example of how we put that mission statement into practice. This blog illustrates what we mean when we say that “most businesses already have the technical capital necessary to meet their needs,” and that PJL Industries helps clients achieve “lasting implementation” by working with them to ensure the root cause of their problem is being addressed.

The Situation

As part of the PJL Industries consultation process, we require at least one free initial consult to discuss a client’s needs and expectations. This ensures that PJL is a good fit and that we are in a position to deliver. It’s helpful for both PJL and the client as it gives us an opportunity to work together to clarify the problem and identify what needs to be solved.

During one particular consult, a potential client had called to discuss a data warehousing problem his analytics team was having. The client knew that product delivery was slower than it needed to be, and his team was unhappy about the performance and storage limitations of their existing cloud data warehouse. The client was being pitched on a promising new cloud data warehouse that would ostensibly solve all the problems his team was facing. He wanted PJL to help determine what he should do: stay with his current solution or pay for the new vendor and migrate.

PJL had had experience with both products, enabling us to make an objective comparison based on performance and storage costs. So it should have been a straightforward ask, right? Not quite. The problem of whether or not to switch products is certainly “a” problem, but in our experience it’s generally not “the” problem — especially when it comes to data. Therefore, we continued the conversation in order to dive deeper into what was going on and ask “why” performance and storage were bottlenecks to delivery. As it turned out, the problems were much more nuanced than they appeared.

The Power of Asking “Why”

At PJL Industries, we love to ask “why.” In this instance, we continued the conversation by asking “why” the team was running out of storage, and “why” performance was lagging. This, of course, led to other questions, and I’ve summarized the conclusions below:

Storage

• New data are automatically dropped into the data warehouse from outside sources on a monthly basis.
• Older data are not automatically pushed out, however, because the new data aren’t always an adequate replacement.
• The default behavior is to keep all data and evaluate usefulness on an as-needed basis.

Performance

• The inconsistencies and incorrectness of the data (see above) meant that client deliverables often had to be reworked to fix errors and inconsistencies.
• Data fixes were made at the deliverable level, not at the data warehouse level. So the same mistakes, and subsequent fixes, were made repeatedly.
• There was no system in place to optimize access to the most important and most frequently accessed data.
• There was no documentation describing the data. As a result, analysts had to ask somebody or spend time looking for what they needed.

Whoa! Things had definitely gotten more interesting. The first thing that should stand out is that a new data warehouse alone would not address any of these underlying problems. And paradoxically, increasing storage would only make client delivery slower because analysts would have to spend time sifting through even more data. What’s worse, there is no guarantee deliverables would get any better with the addition of more data. Uh, oh.

What Next

Understandably, the client was somewhat taken aback. After all, this conversation had started with what seemed like a simple vendor evaluation question, but had since uncovered deeper issues that would require more complicated solutions than a new data warehouse. PJL explained that in this instance, “complicated” wouldn’t necessarily mean more expensive (data migrations themselves are not cheap and they are often fraught with hidden costs), but “complicated” would mean more work because it would require the team to change their current behavior.

Of course, we also discussed ways in which PJL could help. For example PJL could build automated processes to clean and validate incoming data. Or we could look at the access logs to define which data were accessed most frequently. Or we could implement a more logical storage structure. We warned the client, however, that our solutions could only do so much because they require consistent effort on the part of the team to maintain. Software can certainly help facilitate a mindset shift, but it is ultimately up to the individual (or manager) to commit.

In the end, the client was appreciative, but wasn’t prepared to formally start making changes to address the issues identified. It’s possible the client never will, or will implement internally, or even use another contractor. But that’s okay. The purpose of the consult is to clarify the problem and identify what needs to be solved, and one of the purposes of PJL Industries is to help clients avoid wasteful expenditures. As far as we’re concerned, mission accomplished on both counts!