Smart over BIG in the data battle

We are a culture consumed with two S words — size and smart. We love big things — big houses, big cars, big televisions, big experiences, and big vacations. We are also obsessed with smart. We have smart phones, smart cars, smart appliances, and smart devices.
Size and smart are not mutually exclusive. You can have both, but when it comes to data you have to be very careful to decide upon which to focus. In the world of data analytics I often see size and smart secretly fighting for priority. As I have consulted with companies you hear C-suite leaders talk about needing “more data” on that topic. They say things like, “if we just had an idea about X, we would be in a much better place.” It is this need to have as much information as possible that takes over and creates a false sense of security. Organizations think the more they know, the safer they are.
The big data revolution and the consumption of huge data sets are the equivalent of Thanksgiving dinner. Organizations gorge themselves on data — asking for more, knowing all along they should stop. They want structured data, unstructured data, and then they want to correlate the date to look for trends among subsets of data. Often times organizations find themselves curled up on the proverbial couch in a tryptophan induced coma. They spend so much effort on the collection of the data, that there is nothing left in the tank to do the real work of taking the operational steps to change the culture.
It is estimated that by 2020 1.7 megabytes of new information will be created each second for every person. At least Thanksgiving is just one day a year, but sticking with the metaphor, soon it could be Thanksgiving everyday for organizations as the cornucopia of available data continues to grow. Organizations are stuck in the perfect storm. With the emergence of behavioral economics reminding us what poor decisions makers we are, and the technology revolution providing us huge amounts of data, it is no wonder that we are simply looking for more and more data. We are hoping that with just that one more piece of information we can ultimately make the best possible decision. We all need to stop letting “size” drive our efforts and give “smart” a chance at the wheel.
In our company, here are some of the processes we use to try to get to smarter data.
We are very clear as to what we are trying to “do” with the data? What is the goal of digging around for the data? I don’t even put this as a step — it is more of a non-starter. Many organizations go wrong here, they believe they want more data but they can’t articulate why. It has been my experience in these situations there is already data available, but the organization does not like what the data is saying — so they are looking to find the data to fit their own pre-conceived story.
Step 1 — Search — determine what you are looking for. This is where you look through all sources looking for the one that you think makes the most sense.
Step 2 — Filter — Narrow down the options so you can real hone in on a specific data set.
Step 3 — Analyze — Spend some time going through the data, start crunching the numbers. Work to develop a good understanding of what the narrowed data set is telling you.
Step 4 — Use the data — Do some hypothetical exercises, go through and think about what decisions you might have made if you had previously had the data before making that decision. When you hypothetically “re-make” your decision — does it feel better? Could you create a limited test to see if this data drove the correct decision?
Step 5 — Verify Data — Look to have another data set and then use that data set to see if you come to the same conclusion as the first data set.

