How to make Big Data yield big insights

Whether you’re a Big fan or a Big skeptic, there’s no avoiding Big Data.

Some of the buzz has subsided, but Big Data is growing faster than ever, in volume, challenges and unprecedented opportunities. It’s estimated that 90% of the world’s data was created in the last two years, and that amount will double in the next two years.

If you prefer the view from the bottom line, check out worldwide spending growth on Big Data and business analytics — $122 billion in 2015 to $187 billion in 2019, according to International Data Corporation.

Yet, with this explosion of investment, not everyone is seeing the benefits of Big Data.

A key issue is that the amount of data is growing much faster than our ability to process it. While top-performing organizations are able to collect, organize and analyze data for insights and competitive advantage, others continue to struggle under Big Data’s crushing weight.

What’s the difference?

For many companies, the difference is in the DNA. Many of today’s most admired companies, like Apple, Google, Amazon, Facebook and Uber are “digital natives’ that never had to transition away from legacy technology and business models. Things are tougher for traditional companies struggling with digital transformation and smaller organizations without the resources to deal with Big Data.

Another challenge — not only is Data Big, it’s unorganized. Experts estimate that 80 to 90 percent of existing data is unstructured. Your internal unstructured data comes from internal systems including CRM, financials, company email and and countless spreadsheets. External data includes customer emails, text messaging, social media, and data from the Internet of Things.

Challenging, but there are lessons to be learned from organizations who have made Big Data a big success.

5 tips to help make Big Data a big success:

1. Establish budget and buy-in

Without executive approval and funding, there’s no such thing as a data-driven organization. You could have the world’s greatest data scientists performing amazing feats of analysis, but their insights won’t see the light of day without executive sponsorship. With top-down sponsorship, organizations are more likely enjoy the financial benefits of a company-wide data strategy. But Big Data leadership doesn’t come cheap. Staffing, training and maintaining modern data analysis tools is a hefty ongoing expense that must be approved and accounted for.

Surprisingly, many organizations successfully clear the budgeting hurdle and implement analytics programs, then largely ignore the reports. This despite strong evidence that data strategy is a key factor for today’s high-performing companies. Even high-performing companies make subjective, rather than data-informed decisions, from time to time. A robust analytics program requires leadership not only to promote a data culture, but to use data for their own decision support.

2. Adopt an agile approach

What happens when Big Data initiatives are paired with traditional project management?

A whole lot of nothin’.

Traditionally managed projects typically have long, rigidly defined development periods with less frequent team communications. An agile approach that employs shorter, more frequent work periods (sprints) that facilitate ongoing testing and improvement. Agile methodologies enable data-driven teams to make constant improvements and course-corrections throughout the development process to avoid ending up with a product that’s gone off track before it’s even launched. Bottom line? A culture of testing, optimization and improvement depends on an agile approach. It’s practically impossible to have one without the other.

3. Data centralization

As data volumes growing exponentially, the data sources contributing to that volume continue to multiply.

This rich diversity of data sources makes it difficult to collect data into a single “source of truth.” Any database designer will tell you that that redundant data from multiple sources can ruin management and reporting capabilities in a hurry.

Still,data from a company’s internal systems (CRM, HR, Financials) must be combined with third-party data to add valuable context and guide better business decisions. How much better? Investments in analytics tools are now paying back better than a 13-to-1 return on investment with increased returns when these tools integrate with three or more data sources, according to O’Reilly Media. Data centralization may be difficult and expensive, but it appear that the resources invested are justified by huge potential returns.

4. Data strategy and measurement model

For every popular tech phenomenon, there’s a stodgy underlying framework that doesn’t get much love and attention. It’s a shame, because there’s really no point in having data without data a strategy. A great example of a great framework is Avinash Kaushik’s Digital Marketing & Measurement model. This model is often used to implement Google Analytics, a super-powerful and free application that’s used by 29,616,540 live websites, according to How many of those websites support their implementation with a data strategy and use analytic rigor to improve business results? Far fewer.

The reason this model has likely been so successful is that encourages a top-down approach that syncs sales and marketing with the organization’s most important goals. Models such as this can be used to encourage cross-departmental efforts to measure and improve marketing, sales, and customer success initiatives in an objective way.

The model with documenting high-level business objectives, then putting goals and Key Performance Indicators in place to support each goal. Target goals are then agreed upon and documented to ensure a clear definition of success or failure. This is followed by identifying key audience segments, because aggregated data isn’t actionable. The model comes with the admonition that “We not only wanted focus, we wanted hyper-focus” to ensure we understand our success or failure.

5. A story of data storytelling

Once upon a time, in the Valley of Silicon, there lived a small team of data analysts. The team was happy, as they had largely achieved their goals of gathering data and reporting key analytics to senior management. They reported meaningful metrics such as conversion rates, customer acquisition costs and retention.

But nobody read their reports.

Nearly everyone in the company received weekly analytics reports and they liked the colorful tables and charts. But, using a magical tracking link, the analytics team learned that few people actually viewed their reports.

And so the team realized their job did not end with reporting — they also had to make sure that their management and peers understood the benefits of being data-driven. So, they removed detail and presented executives with streamlined, high-level reports. They pointed out significant differences from past results and progress toward future goals. They highlighted key findings and potential causes they deemed important for decision support.

As the executives came to understand the insights contained in the reports, they made better decisions, which led to more revenue. The data analysts were awarded large bonuses and they all bought new hoodies, Apple Watches and expensive German sports cars. They all lived happily until the next reporting period.

The end.

Have your own data story to tell? Share it in the comments!