How to Become a Rising Star with Data Analytics
A Guide to DataOps
Why drive to the store to buy a stapler, when you can order it from Amazon who will deliver it to your doorstep in two days for around the same price? The list of goods and services that consumers purchase online seems to grow each year: movies, travel, software, taxi rides, lodging, food, IT resources and more. Mike Jaconi calls this the on-demand economy. In his Business Insider article, he defines the on-demand economy as the “economic activity created by technology companies that fulfill consumer demand via the immediate provisioning of goods and services.” Immediate provisioning is not always instantaneous, but it is fast and it will keep getting faster. Companies will be under tremendous pressure to be agile and lean. Those who can adapt will become the pacesetters and the market leaders. Those who stick to outdated methods will fade away. Key employees who help their companies succeed under these challenging market conditions will become nothing short of rising stars.
The market leaders will succeed because they have better information about products, customers, and trends. This comes down to data analytics. If you examine the most successful companies, data analytics is at the heart of their success. Companies will sink and soar with the quality and flexibility of their data analytics.
Imagine working at a company that has the industry’s best data analytics. The data analytics team would respond to requests for changes with previously unimaginable speed. The automated data analytics pipeline would incorporate changes rapidly and continuously. A robust test suite would ensure system quality. No more enterprise-critical IT emergencies at odd hours. Instead of fighting fires and being consumed with system maintenance, the data analytics team is freed to focus on higher-value activities that provide the company more insight into the marketplace. The data analytics team isn’t relegated to some dark corner, they are in the center of the company’s strategy advancing its most strategic and important objectives.
We call this new approach to data analytics, DataOps. As we explained in our previous blogs, DataOps is built upon a foundation that includes Agile Software Development, DevOps and statistical process controls (SPC). Agile development and DevOps have enabled IT and software development organizations to advance from performing one release about every twelve months, in the 1980’s, to releasing code many times per hour today. SPC guards against failures, controls quality and provides real-time alerts when data metrics drift outside defined limits. DataOps is a rapid-response, flexible and robust data-analytics capability, which is able to keep up with the creativity of internal stakeholders and users.
DataOps is an analytic development method that emphasizes communication, collaboration, integration, automation, measurement and cooperation between data scientists, analysts, data/ETL (extract, transform, load) engineers, information technology (IT), and quality assurance/governance. The method acknowledges the interdependence of the entire end-to-end analytic process. It aims to help organizations rapidly produce insight, turn that insight into operational tools, and continuously improve analytic operations and performance. It enables the whole analytic team involved in the analytic process to follow the values laid out in the Agile Manifesto.
We discussed in previous blogs the high rate of failure of data analytics projects due to changing requirements, slipped schedules, disappointed users, inflexibility, poor quality, low ROI and irrelevant features. When DataOps is implemented correctly, it addresses the issues that have plagued data-analytics teams.
Key Benefits of DataOps
DataOps doesn’t require you to throw away your existing tools and start from scratch. With DataOps, you can keep the tools you currently use and love. DataOps views the data-analytics pipeline as a process and as such focuses on how to make the entire process run more rapidly and with higher quality, rather than optimizing the productivity of any single individual or tool by itself.
DataOps can accelerate the ability of data-analytics teams to create and publish new analytics to users. It requires an Agile mindset and must also be supported by an automated platform which incorporates existing tools into a DataOps development pipeline. DataOps spans the entire analytic process, from data acquisition to insight delivery. Its goal is to achieve more insight and better analysis, while still being faster, cheaper and higher quality.
The key business benefits of adopting DataOps are:
• Reduce time to insight
• Improve analytic quality
• Lower the marginal cost to ask the next business question
• Improve analytic team morale by going beyond hope, heroism and going slowly
• Promote team efficiency through agile process, reuse and refactoring
Implementing DataOps
Imagine the next time that the Vice President of Marketing requests a new customer segmentation, by tomorrow. With DataOps, the data-analytics team can respond ‘yes’ with complete confidence that the changes can be accomplished quickly, efficiently and robustly. How then does an organization implement DataOps? You do it by applying Agile development, DevOps and statistical process controls to your data analytic pipeline. On a practical level, we’ve boiled the implementation of DataOps down to seven simple steps. These seven steps will be covered in a series in upcoming editions of our blog.
Like this story? Download the DataOps Cookbook!
