Seven Challenges of Customer Analytics

Companies increasingly look to customer and market analytics to drive growth strategies. As the leader of the data-analytics team, you manage a group responsible for supplying sales, marketing and other business partners with the analytic insights that can create a competitive edge. Customer and market opportunities evolve quickly and drive a relentless series of questions. Analytics, by contrast, move slowly, constrained by development cycles, limited resources and brittle IT systems. The gap between what sales needs and what IT can provide can be a source of conflict and frustration. Inevitably this mismatch between expectations and capabilities can cause dissatisfaction, leaving the data-analytics team in an unfortunate position and preventing a company from fully realizing the strategic benefit of its data.

As a manager overseeing analytics, it’s your job to understand and address the factors that prevent the data-analytics team from achieving peak levels of performance. If you talk to your team, they will tell you exactly what is slowing them down. You’ll likely hear variations of the following seven challenges:

1 — The Goal Keeps Moving

Sales and marketing are demanding customers for a data-analytics team. Their requirements change constantly. They require immediate responses, and no matter how much the analytics team delivers, sales and marketing keep generating new requests. It’s enough to overwhelm any data-analytics team.

They don’t know what they want. Sales and marketing professionals are not data experts. They don’t know what insights are possible until someone from your team shows them. Sometimes they don’t know what they want until after they see it in production (and maybe not even then). Often, sales and marketing contributors do not know what they will need next week, let alone next quarter or next year. It’s not their fault. It’s the nature of pursuing customers in a fast-paced marketplace.

They need everything ASAP. Sales and marketing are competitive endeavors. When a customer opportunity opens, they need to move on it faster than the competition. When sales and marketing bring a question to the data-analytics team, they expect an immediate response. They can’t wait weeks or months — the opportunity will close as customers seek alternative solutions.

The questions never end. Sometimes providing business stakeholders with analytics generates more questions than answers. Analytic insights enable sales and marketing to understand customers in new ways. This spurs creativity, which leads to requests for more analytics. A healthy relationship between the analytics and sales teams will foster a continuous series of questions that drive demand for new analytics. However, this relationship can sour quickly if the delivery of new analytics can’t meet time frames required by sales and marketing.

2 — Data Lives in Silos

In pursuit of business objectives, companies interact with customers across a multitude of channels and collect an enormous amount of data: orders, deliveries, returns, website page views, mobile app navigations, downloads, clicks, metrics, audio logs, social media and more. Further, this data can be combined with demographic, psychographic or other third party market data. All of this data is collected in separate ERP, MRP, CRM, marketing automation, web analytics, call center platforms and other systems. Typically, none of these systems talk to each other. They are implemented in a variety of databases and on different software platforms. They utilize numerous APIs and technologies. Accessing all of this data is a daunting task requiring such a wide range of skills that it is rare to find a single person that can do it all. Integrating data from these myriad sources becomes a major undertaking.

Sales and marketing want fast answers. Meanwhile, the data-analytics team has to work with IT to gain access to operational systems, plan and implement architectural changes, and develop/test/deploy new analytics. This process is complex, lengthy and subject to numerous bottlenecks and blockages.

3 — Data Formats are not Optimized

Data in operational systems is usually not structured in a way that lends itself to the efficient creation of analytics. For example, an ERP system might have a schema that is optimized for inserts, updates, and for display in a web user interface. For operational systems, these are the actions that need to happen in real time.

A database optimized for data analytics is structured to optimize reads and aggregations. It’s also important for the schema of an analytics database to be easily understood by humans. For example, the field names would be descriptive of their contents and data tables would be linked in ways that make intuitive sense.

4 — Data Errors

Whether your data sources are internal or from external third parties, data will eventually contain errors. Data errors can prevent your data pipeline from flowing correctly. Errors may also be more subtle such as duplicate records or individual fields that contain erroneous data. Data errors could be caused by a new algorithm that doesn’t work as expected, a database schema change that broke one of your feeds, an IT failure or many other possibilities. Data errors can be difficult to trace and resolve quickly.

5 — Bad Data Ruins Good Reports

When data errors work their way through the data pipeline into published analytics, internal stakeholders can become dissatisfied. This causes unplanned work, which diverts your key contributors from the highest priority projects. Bad data also harms the hard-won credibility of the data-analytics team. If business colleagues repeatedly see bad data in analytics reports, they might learn not to trust or value the work product of the data-analytics team.

6 — Data Pipeline Maintenance Never Ends

Data-analytics is a pipeline process that executes a set of operations and attempts to produce a consistent output at a high level of quality. Every new or updated data source, schema enhancement, analytics improvement or other change triggers an update to the pipeline. The data-analytics team is continuously making changes and improvements to the data pipeline. Each one of these changes must be made carefully so that it doesn’t break operational analytics. The effort required to validate and verify changes often takes longer than the time required to create the changes in the first place. You may not realize it, but your analysts, data scientists and engineers may be spending 80% of their time updating, maintaining and assuring the quality of the data pipeline. This is necessary work, but much of it is behind the scenes and unappreciated when viewed against the growing backlog of new requests from sales and marketing.

7 — Manual Process Fatigue

Data integration, cleansing, transformation, quality assurance and deployment of new analytics must be performed flawlessly day in and day out. The data-analytics team may have automated a portion of these tasks, but some teams perform numerous manual processes on a regular basis. These rote procedures are error-prone, time-consuming and tedious.

Further, manual processes can also lead to high employee turnover. Many managers have watched high-performing data-analytics team members burn out due to having to repeatedly execute manual data procedures. Manual processes strain the productivity of the data team in numerous ways.

Overcoming the Challenges

Some say that an analytics team can overcome these challenges by buying a new tool. While it is true that new tools are helpful, they are not enough by themselves. You cannot truly transform your staff into a high-performance team without an overhaul of the methodologies and processes that guide your workflows. In our next blog, we will discuss how to combine tools and new processes in a way that improves the productivity of your data analytics team by orders of magnitude.

DataKitchen offers a DataOps Platform and managed services for data and analytics-group leaders whose team struggles to keep up with customer requests and lets errors slip into production. Please join the DataOps movement by signing the DataOps Manifesto.