Are you looking to better leverage your organization’s data resources to facilitate modeling and analytics? No matter what stage you’re in currently, there’s always room for improvement. The typical data analysis processes consist of five general steps:
- Collecting data. Identifying the right sources both internal and external.
- Preparing data by cleaning, transforming and blending together data sets.
- Storing data by designing the appropriate data model, as well as planning for data archival and business continuity strategies.
- Analyzing and Reporting on data to understand business drivers and risks.
- Deciding what action is appropriate to take as a result of the analysis.
Studies show that as much as 80% of time spent performing data analysis is spent on these first three steps: getting data ready and structured for use. Why duplicate efforts in this phase if you can avoid it? What if each functional area could get at the data they need, know how it was derived and perform their own data exploration and analysis with confidence that they’re using the right information to make decisions.
In the early stages of data analysis, each functional area is responsible for their end-to-end processes. Operations collects and manages its own data, handles infrastructure and data cleaning separately and uses what’s available to them, as does Finance, Compliance, and other groups. Similar to the structure below.
Let’s assume each of these groups averaged just one full time resource dedicated to data analysis. At 80% of their time spent in data preparation. That would be almost two and a half people of potentially duplicated efforts cleaning and blending the same data for different end goals. With this siloed approach to data analysis, each functional area may have different quality standards and implement different data blending methodologies leading to different results. Having numerous processes and approaches running in parallel can also create a lack of transparency and result in limited governance.
By moving to a centralized data management approach, such as that represented in Figure 3, an organization can limit the time spent on data preparation and give business areas ready-access to the data they need. In my previous example of having 3 employees largely dedicated to preparing data, those efforts can now be significantly scaled down. Assuming some level of duplicated efforts in the siloed scenario, you may be able to restructure processes to have one dedicated data preparation and management resource. This structure would free up functional area team members to perform valuable analysis and focus on driving strategic business initiatives forward.
In this new, centralized framework, organizations can apply the same quality standards and methodologies across data sets. This will increase transparency and facilitate the implementation of proper data governance, ensuring traceability and auditability of analytics results.
As organizations collect more and more data, the ability to perform analytics becomes an ever more critical capability. Leveraging this resource can create new opportunities, drive higher profits, and help meet mission goals more efficiently. As more organizations seek to decentralize decision-making and increase responsiveness, we see an increasing trend towards centralized data management. With new analytics and data visualization tools, users across the organization can easily access data and derive their own insights.
Without thoughtful data preparation and governance, transitioning to centralized data management can introduce its own set of challenges for business owners and analytics teams.
Challenges
- Business users are now further removed from the data source and may not be as familiar with necessary nuances of data preparation that may affect their analysis.
- Functional experts in the data management team are closer to the data but may be less familiar with the end use cases.
- Centralized data has the potential to be less flexible, too complex, and harder to manage in an effort to create a structure that works for everyone.
- If users don’t find their centralized resources useful, they will still attempt to add layers of data wrangling to meet their needs and create additional duplication of both efforts and data sets.
Without information on how data was derived and potential nuances of each data set, an analyst may inadvertently chose the wrong data source or use a field that doesn’t accurately represent the business metric they were original after, leading to inaccurate or inconsistent results. This can cause management to question results and teams to spend hours tracking down the source of discrepancies.
There are several key steps you can take to address all of these challenges and ensure a successful transition to a centralized data management model.
Keys to Success
- Implement strong data governance — policies, procedures, and controls around how data is managed to ensure quality.
- Ensure sufficient transparency into data sets and methodologies.
- Continue ongoing, two-way communication between data management teams and business users.
How much data preparation do you need before opening up access to business users? How do you get the right data? What are the right tools, and where do you get started?
