Analytics promises to provide organizations with critical data that drives better decision making, yet the landscape is littered with analytics projects canceled or deferred due to changing requirements, slipped schedules, disappointed users, inflexibility, poor quality, low ROI, and irrelevant features. The data analytics team is under tremendous pressure and subject to constraints, but that won’t appease unhappy users.
This post discusses three behaviors that will shorten your career in Data Analytics and point to a solution called DataOps.
According to the research firm Gartner, half of all chief data officers (CDO) in large organizations will not be deemed a success in their role. Per Forrester Research, 60% of the data and analytics decision-makers surveyed said they are not very confident in their analytics insights. Only ten percent responded that their organizations sufficiently manage the quality of data and analytics. Just 16 percent believe they perform well in producing accurate models. What is going on in the world of analytics?
As the Gartner and Forrester surveys indicate, analytics professionals are falling short. Businesses exist in a fast-changing environment and analytics can’t seem to keep up. Unfortunately, many CDO’s and data-analytics professionals respond to these challenges in ways that will shorten their once promising careers.
Heroism — Data-analytics teams work long hours to compensate for the gap between performance and expectations. When a deliverable is met, the data-analytics team is considered heroes. However, heroism is a trap. Heroes give up work-life balance. Yesterday’s heroes are quickly forgotten when there is a new deliverable to meet. The long hours eventually lead to burn-out, anxiety and even depression. Heroism is difficult to sustain over a long period of time, and it ultimately just resets expectations at a higher level without providing additional resources.
Hope — When a deadline must be met, it is tempting to just quickly produce a solution with minimal testing, push it out to the users and hope it does not break. This approach has inherent risks. Eventually, a deliverable will contain data errors, upsetting the users and harming the hard-won credibility of the data-analytics team. Many data analytics professionals will recognize the problem that hits Saturday morning. You work late on Friday night pushing changes out the users. After a heroic effort, you get it done and go home. Saturday morning you wake up, startled. I forgot to check X. Did I just break the analytics? You can either rush back to work and spend the weekend testing or let it go and hope everything is OK.
Reliance on heroism and hope is all-too-common in the data analytics profession. However, these strategies are risky and unsustainable and can lead to disappointment. When an analytics professional has given up on heroism and hope he or she commonly resorts to the third career-limiting behavior.
Caution — The team decides to give each data-analytics project a longer development and test schedule. Effectively, this is a decision to deliver higher quality but fewer features to users. One difficulty with this approach is that users often don’t know what they want until they see it, so a detailed specification might change considerably by the end of a project. The slow and methodical approach might also make the users unhappy because the analytics are delivered more slowly than their stated delivery requirements and as requests pile up, the data-analytics team risks being viewed as bureaucratic and inefficient.
None of these approaches adequately serve the needs of users. Heroism can’t be sustained and the analytics professional quickly reaches the limits of his/her productivity. Hope is a tacit acknowledgment that quality control is insufficient and manual testing is extremely time consuming. Caution is akin to giving up. None of these behaviors provide the company with the analytics they need and when users are unsatisfied, career trajectories of data analytics professionals are altered for the worse.
DataKitchen was founded by three analytics professionals each with decades of experience. We have lived the futility of heroism, hope and caution and wish to help others out of this no-win situation. Fortunately, we found a solution. We call it DataOps.
We’ll explain DataOps in a future blog, but to fully appreciate DataOps it helps to understand a little about Agile Development, DevOps and the statistical process controls used in lean manufacturing.