DataOps: Enabling Agile in the Insurance Data World — Saama Analytics
Being data-driven has become a trend since Data and Data Analytics technologies showed how to leverage data to gain business insights that can help a company stay ahead of the competition. However, the lack of collaboration leads to challenges in productivity and quality across the data functions. Data quality gets further compromised by siloed and complex data pipelines. Additionally, data analytics teams are often restricted by traditional approaches while trying to meet the evolving requirements of business users.
Manual, ungoverned processes used for data delivery have created compromised analytics processes and generated data friction. The traditional approach requires several weeks to produce business ideas, whereas the industry is looking for ways to quickly innovate and stay ahead of the next looming disruption.
Understanding DataOps
DataOps is a practice, which has brought together the responsiveness of DevOps and Agile methodology to unite the data and business teams toward improved and faster deployment of data analytics use cases.
Gartner defines DataOps as “a collaborative data management practice focused on improving the communication, integration, and automation of data flows between data managers and data consumers across an organization. The goal of DataOps is to deliver value faster by creating predictable delivery and change management of data, data models, and related artifacts. DataOps uses technology to automate the design, deployment, and management of data delivery with appropriate levels of governance, and it uses metadata to improve the usability and value of data in a dynamic environment.”
DataOps, with the addition of ongoing scaling, monitoring, and verification of data pipelines, smoothens the operational bottlenecks of Continuous Integration and Continuous Deployment (CI/CD).
Tenets of DataOps
It’s is all about implementing an iterative lifecycle for the provisioning of data, databases, and data integration pipelines to support self-service agile analytics and reduce data friction that exists in a data organization. Here are some of its most important aspects:
- Team-based Agile Development (automated, incremental, and collaborative)
- Seamless Promotion of Data Pipelines from Dev to Prod (CI/CD)
- Enablement of Data Governance and Security Processes
- Technology: Cloud First with Lambda Architecture (Both Batch and Streaming Mode)
- Modern Data Operations
Applying DataOps Approach to Insurance Claims Data
P&C Insurers have started their DataOps journey and are realizing the benefits of the approach.
We have seen the benefits first hand when we worked with an insurance organization to streamline and modernize their subrogation opportunities identification process with the DataOps approach. The DataOps initiative was delivered with a quick turnaround.
The Agile team included the scrum master, development team members, and the product owner, who worked in sync with the business team. The team was able to quickly prototype and iteratively deliver the features in Cloud by using CI/CD pipelines. CI/CD enabled the operations team to not only support the application with minimal effort but also to add new features with a small resource footprint.
Author: Sandeep Kumtakar
Professional with progressive IT experience in development and support of near-real-time applications, Data Analytics with near real-time ingestion, Data migration, and Data Visualization projects
References:
Gartner Research “Introducing DataOps Into Your Data Management Discipline” https://www.gartner.com/en/documents/3970916/introducing-dataops-into-your-data-management-discipline
Originally published at https://saamaanalytics.com
