In the UK alone, we spend £24bn on data projects every year. According to recent studies, however, organisational leadership has been dissatisfied with the value they get from data. In fact, they consider 80% of all data projects a failure. That equates to £19bn of waste. And why? Because so many don’t do the basics well. They never stood a chance.
But before we dive into the specific reasons for data project failure, let’s look at project success.
What is a successful data project?
The fundamental purpose of any data team is to provide successful data projects. However, a commonly quoted statistic is that around 80% of all data projects fail. This is a remarkable level of failure and, frankly, concerning. We had to investigate.
We’ve recently released our first report titled ‘Emotional Intelligence Report 2022: Data & Insight job satisfaction and its impact on successful data projects’, for which we have surveyed over 120 data professionals across many sectors. This project was an opportunity to find out from data professionals themselves if the statistic mirrored their experience.
When asked ‘how often are your data & insight projects a success?’, the majority (66%) of respondents indicated that their projects were a success at least most of the time, with 11% saying their projects were successful always. From this we could conclude that, from the perspective of data professionals, 80% failure rate might have been a gross overestimate.
So, where does the statistic come from? Well, data teams aren’t the only ones defining a project’s success. It’s a matter of perspective. Business customers also have a defining opinion and, as it turns out, organisations struggle to convert data into actionable insights, creating a huge gap between access to data and the ability to extract meaningful business insights from it.
A business customer is the person asking the initial question which kicks off the data project. 9 out of 10 times the person asking the question wants to achieve a business goal. If they do, by their definition, the project’s a success. If they don’t, it’s a failure. Makes sense, right? This is where the disconnection between data creators and consumers often comes about. Data creators work hard to answer the question or questions asked. That’s their definition of success. But their business customers aren’t satisfied.
Without actionable insights, data projects are not a success
It’s a harsh truth, but without actionable insights and business outcomes, data is nothing more than numbers, words, and charts, with no real value. Data creators’ effort doesn’t matter if data they extract doesn’t make impact in the real world.
Keeping all this in mind, we can conclude that a successful data project is one that results in actionable insights for business growth. Actionable data insights are, therefore, a missing link between data and business value, and they are essential for making informed data-driven business decisions. If you’d like to find out more about actionable insights, you can find a separate post on it here.
While extracting insights of increased actionability from data doesn’t guarantee their adoption, harvesting those should strengthen organisational data culture and encourage business leaders to consistently explore data and act on it.
To extract insights of increased actionability from data, outcomes must be reinforced at every step of the data project so data teams can focus on delivering real value. If they’re not, chances of project failure are increasing.
5 reasons why 80% of data and insight projects fail
1. Wrong questions get asked
This is by far the most common cause for data project failure and the reason is simple: organisational leaders aren’t data experts. They do not understand the highly technical language of analytics, they often don’t know what data they need to answer their business questions. Data experts are data experts. So, it is their job to ensure their business stakeholders ask the right questions.
Requirements gathering is a vital part of any data and insight project, helping to really define the scope of it. Mess up this initial phase and the whole project could be a huge waste of time and financial resources, with many frustrations along the way. Therefore, it is important for data people to keep the following principle in mind when starting a new project: forget the initial question. You don’t know if it’s the right one. Instead, do everything you can to get more context — find out why the person wants to know that and what they’re planning to do with the information.
2. Collaboration between data analysts and business customers is low
There is a gap in communication between data professionals and their business customers, which leads to disconnect on every stage of a data project. While the first group is in fluent in the language of analytics, the other is much more comfortable communicating in the language of project management and business. This can lead to challenges in translating your analysis into actionable insights.
What’s more, many data teams use systems like Jira to manage parts of the insight process. Platforms like these often work as a barrier to business customers because they tend to be less “technical” than data teams. Reluctance to use systems is why 90% of data teams engage in messy email chains for requirements gathering. As a result, data creators and consumers don’t engage properly, leading to a disconnect. And what happens when the most important project stakeholder isn’t involved in the process all the way through? They don’t extract valuable insights or encourage positive business outcomes. Data projects can’t succeed without project stakeholders working closely together. They should all work from one common platform, always knowing the status of their work and where and how to engage.
3. Data project’s insight isn’t clearly communicated to the right stakeholders
Another reason, another gap between technical understanding and expertise of data professionals and pressed-for-time domain leaders. Clear communication is vital for any data analysis success. Organisational leaders may not be as knowledgeable about extracting the correct takeaways that influence their business decisions as data teams. Therefore, data should be presented and explained in ways that are easily digestible by target audiences. Data visualisations are often a very memorable way of communicating insights to ensure its actionability and drive urgency and business action.
Only 14% of Data, Analytics & Insight consumers identify that a business decision or action is always taken because of insights they receive. Without an outcome, data projects are nothing more than numbers. If insight isn’t communicated to the right stakeholders clearly, despite data professionals’ hard work, there is a high chance it will not be implemented but left behind and forgotten.
4. Data projects aren’t discoverable
I cannot overstate the importance of organised and easily searchable database for organisational data insights and knowledge. The problem is that they often don’t fit into places or tools we already have in place because they’re not built for that purpose. This results in difficulty in finding and reusing past data insights. If insights are not discoverable, they’re useless.
To prevent potential project failure, organisational data should be highly discoverable, that is easily accessible, consumable, and usable. Having a centralised management for your organisation’s knowledge, instead of a maze of system folders, helps increase efficiency and security, improves workflow, and simplifies regulatory compliance to name a few.
5. There are no systems and standardisations in place
Data projects are challenging. Requests gathering often involves a lot of email back and forth between data professionals and their business customers. The scope of stakeholders’ needs changes frequently throughout the development process. Updates get missed and email chains get lost, prolonging the turnaround times. Non-technical data users are reluctant to use complex tools like Jira, creating further barriers. Data gets ignored or lost. Labour, time, and money get wasted.
These are common frustrations which can, unfortunately, contribute to eventual project failure. The good news is, they’re easy to eliminate by implementing systems and standardisations which work for both data professionals and their customers. Having the right tools ensures the project outcome meets your business customer’s needs, driving action and business value.
Data creators and their business customers need to work closely together
The bottom line is that both data creators and their business customers need to be involved in the data & insight project from the initial question through to the outcome and work closely together for it to provide actionable insights and urge action.
Currently, there are many gaps between the two groups, resulting in disconnect, frustrations, time and financial losses, and no real-world outcomes. Organisations need to close these to truly harness the power of data and maximise its value. Businesses must promote organisational data-culture if they want to maintain their competitive edge. This means their data teams and all other departments must be aligned, processes must be standardised and systemised, and technology must be used effectively and efficiently.
Data should be at the heart of all business decisions. Organisations around the world understand that. Next step is for technical talent and organisational leadership to understand they must implement tools and processes to close the gaps between them and harness the true value data offers.
— — — — — —
Author: Adrian Mitchell, Founder and CEO at Brijj
Brijj is a work management platform by and for data & insight teams, designed to give organisations of any size the means to help data creators and business users collaborate on projects in the best way possible.
The full report can be downloaded here: www.brijj.io/emotional-intelligence-report-2022
The Emotional Intelligence report is a part of Brijj’s broader #Brijjthegap campaign aimed at closing gaps in understanding and collaboration between data teams and their business customers, as well as within data teams themselves. Its purpose is to elevate jobs across the data & insight industry and eliminate challenges standing on organisations’ way to truly harness the value of data, for happier teams and thriving businesses. To help us in doing so, follow the hashtag of my company’s page!
