Do You Need a DataOps Dojo?

As DataOps activity takes root within an enterprise, managers face the question of whether to build centralized or decentralized DataOps capabilities. Centralizing analytics brings it under control but granting analysts free reign is necessary to foster innovation and stay competitive. The beauty of DataOps is that you don’t have to choose between centralization and freedom . You can choose to do one or the other — or both. Below we’ll discuss some standard DataOps technical services that could be developed and supported by a centralized team. We’ll also discuss building DataOps expertise around the data organization, in a decentralized fashion, using DataOps centers of excellence (COE) or DataOps Dojos.

DataOps Technical Services

A centralized team can promote DataOps adoption by building a common technical infrastructure and tools to be leveraged by other groups. Centralizing analytics helps the organization standardize enterprise-wide measurements and metrics. For example, some teams may recognize services revenue in the quarter booked, and others may amortize the revenue over the contract period. With a standard metric supported by a centralized technical team, the organization maintains consistency in analytics.

A centralized team can publish a set of software services that support the rollout of Agile/DataOps. The DataOps Technical Services (DTS) group provides a set of central services leveraged by other groups. DTS services bring the benefits of DataOps to groups that aren’t ready to implement DataOps themselves. Examples of technologies that can be delivered ‘as a service’ include:

• Source code control repository
• Agile ticketing/Kanban tools
• Deploy to production
• Product monitoring
• Develop/execute regression testing
• Development sandboxes
• Collaboration and training portals/wikis
• Test data management and other functions provided ‘as a service’

The DTS group can also act as a services organization, offering services to other teams. Below are some examples of services that a DTS group can provide:

• Reusable deployment services that integrate, deliver, and deploy end-to-end analytic pipelines to production.
• Central code repository where all data engineering/science/analytics work can be tracked, reviewed and shared.
• Central DataOps process measurement function with reports
• ‘Mission Control’ for data production metrics and data team development metrics to demonstrate progress on the DataOps transformation

DTS creates robust DataOps services and capabilities, but if an organization wishes to seed DataOps practices throughout the organization, it should plan methods to transfer DataOps solutions and “know-how” to data scientists and engineers in the periphery of the organization.

DataOps Center of Excellence

The center of excellence (COE) model leverages the DataOps team to solve real-world challenges. The goal of a COE is to take a large, widespread, deep-rooted organizational problem and solve it in a smaller scope, proof-of-concept project, using an open-minded approach. The COE then attempts to leverage small wins across the larger organization at scale. A COE typically has a full-time staff that focuses on delivering value for customers in an experimentation-driven, iterative, result-oriented, customer-focused way. COE teams try to show what “good” looks like by establishing common technical standards and best practices. They also can provide education and training enterprise-wide. The COE approach is used in many enterprises, but the DevOps industry has more often standardized on Dojos as a best practice.

DataOps Dojo

Demand for skilled DataOps engineers is skyrocketing, and like DevOps engineers, they are hard to find and harder to hire. Enterprises moving towards DataOps transformation may find it worthwhile to build DataOps expertise organically in each team within the data organization.

A DataOps Dojo is a place where DataOps beginners go for a short period of intense, hands-on training. In Japan, a dojo is a safe environment where someone can practice new skills, such as martial arts. Companies like , Delta Airlines, and John Deere employ the Dojo concept effectively to build lean, Agile, and DevOps muscles. The Dojo offers a separate workspace where teams learn new skills while working on actual projects that deliver customer value.

Dojos provide an environment where teams gain practical experience without worrying about introducing errors into the production environment. The staff rotates in for weeks or months at a time to learn new skills by working on real-world projects. They then bring those skills ideas back to their original teams.

DataOps Transformation

Each of the approaches described above can deliver DataOps benefits to the enterprise. Nevertheless, it can be challenging to grow DataOps expertise in-house without the benefit of mentorship. DataKitchen offers DataOps Transformation Advisory Services that address DataOps methodologies, strategy, tools automation, and cultural change. Learn more about our DataOps Transformation Advisory Services here, or feel free to if you just want to chat.

Originally published at https://datakitchen.io on January 20, 2021.