A lengthy cycle time is the primary reason that analytics fail to deliver on the promise of improving data-driven decision making.

Minimizing Analytics Cycle Time with DataOps

When someone walks into your office asking for new analytics, the clock starts ticking. The time that elapses from that moment to the delivery of the new analytics is your cycle time. In many enterprises, a lengthy cycle time is the primary reason that analytics fail to deliver on the promise of improving data-driven decision making.

The internal and external customers for your analytics are mostly business people, not technologists. They have a job to do, and they want insights in order to make better decisions. They need answers quickly — usually much faster than the data-analytics team is capable of executing.

The Cycle-Time Challenge

In a typical scenario, a request comes in, and the data analytics team assembles to plan out a solution. Engineers, data scientists, data visualization consultants, and others put together a project to design new analytics to meet the customer’s requirements. After formulating a solution and finding numerous clever optimizations, they might conclude that a project will take, for example, two weeks to complete. Unfortunately, the customer expects it in two hours. The mismatch between expectations and reality can be a cause of great frustration on both sides.

The Heroism Challenge

When faced with the problem of cycle time that is too long, many managers and individual contributors make the mistake of trying to solve the problem through heroism. They work nights, weekends, holidays and cancel vacations in order to meet deliverables. There are many problems with this approach. It isn’t sustainable, doesn’t scale and doesn’t provide the elasticity necessary to accommodate unplanned work.

The hero attempts to solve the cycle-time problem by working hard and doing everything themselves. The hero is your star performer, and they are incredibly talented and sought after. They tend to love to work on new analytics, but become bored by analytic operations when they are faced with having to do the same things over and over again, day in and day out.

If you run a team of heroes, and you are struggling with cycle time, you need to ensure that you don’t get stuck with technical debt stemming from an expedient move made in the heat of meeting a deadline. It can be painful to be locked into a sub-optimal system architecture due to a decision that someone quietly made six months ago.

To use a manufacturing analogy, the hero is a bottleneck. On the factory floor, the bottleneck is what limits the overall throughput of the system. For example, the team can become dependent on a piece of code that no one understands, except for the hero who wrote it. When the hero has a finger in every pie, he or she is in high demand. This ends up limiting the productivity of the entire team.

The Challenge of Unplanned Work

With personnel spread thin and heroes working at their full capacity to meet internal and external customer requirements, the team is vulnerable to crises. A service disruption can happen at any time. Something is wrong in the data — reports are breaking or showing anomalous data. Sometimes if this is not fixed immediately, there are consequences. The team has to drop what it is doing and solve the problem. This is called unplanned work, and it can be a silent drag on a team’s productivity.

Unplanned work stems from a lack of checks and balances in the data pipeline (the Value Pipeline) and the pipeline for new analytics (the Innovation Pipeline). It is essentially an outcome of the technical debt incurred by relying on heroism and inefficient and unreliable manual processes.

Rising Above Heroism with DataOps

Many enterprises face these and similar problems. Data-analytics professionals are tasked with delivering insights. Their job is to take large flows of data and visualize them and model them so that the company can make better decisions. However, lengthy cycle times, heroism and technical debt prevent most data-analytics teams from satisfying their customers.

When things go wrong, the hero often gets the blame, but it is actually the manager’s fault. Good people achieve mixed results when they are working in a broken process. The manager needs to figure out a way to meet the cycle time requirements of the business without relying on heroism or incurring technical debt. Fortunately, there is an answer.

It is possible to deliver analytics rapidly, flexibly and robustly with a new approach called DataOps. DataOps replaces heroism with automated orchestration and highly-optimized processes. It reduces the cycle time of new analytics by an order of magnitude and helps reduce the time to insight. This helps lower the marginal cost of asking the next business question, which helps unlock creativity. DataOps rests on three pillars:

• Agile Development — An iterative software development methodology, which can help analytic teams remain focused, and eliminates wasted effort.
• DevOps — Automation of the Value and Innovation Pipelines based on the continuous deployment methodology pioneered in software development.
• Statistical Process Control — Monitoring and filtering of data flowing to users ensuring that quality remains high.

If you oversee data analytics for your company and are struggling with cycle time, DataOps offers you an efficient and reliable way forward. Let’s talk.