DevOps and Compliance — a match made in heaven

Everybody working as software engineer in a regulated environment like the financial industry knows that traditionally there is a huge tension between delivering fast and taking care of all the regulatory requirements.

The reason for this is simple. The processes involved to cover regulatory requirements usually are not automated and involve a lot of manual work. Think about the creation of documentation like release records, proof of successful testing, security testing or tasks like deployment approvals. Many engineers consider these activities as cumbersome and wasteful, reducing the speed of experimentation and innovation.

This tension is real, no doubt about it.

But what is also clear is that we must resolve this tension because the market demands a continuous flow of customer value. There is just no place for companies that are not able to deliver the good stuff quickly and at the same time cover all regulatory requirements.

Or, to put it into a different perspective, being able to pull this off can be an extraordinary competitive advantage in a business area that typically still works rather traditionally.

The Good News

The good thing is that the DevOps movement has given us powerful tools that we can also apply in the case of compliance.

Automation: Automated build and delivery processes generate a lot of data that can be used as proof of compliance if collected and aggregated in the right way. In the best case, all required proof of compliance is generated automatically and completely transparent for the engineers.

Shift left: Very similar to quality and security we must add compliance as concern for the whole team that needs to be considered from the start.

Breaking down of silos: Eliminating the barriers in organisation, language and skills between development, quality assurance and operations definitely was one of the key achievements of the DevOps movement. Later, also security was introduced into the team and DevSecOps emerged. Now the same needs to be done with risk, audit and compliance functions. Regulatory requirements always leave room for interpretation and can be applied context specific. A joint effort including all stakeholders will guarantee the effective design and implementation of these requirements.

Business involvement: Compliance is not a technical topic only, it obviously also is a business concern and it must be budgeted accordingly. If products and processes are not managed according to the regulatory requirements, severe fines will have an impact on the bottom line and in the worst case companies can go out of business.

Product Owners must consider compliance related efforts as part of the product backlog (as a side note, this is true for all other non-functional requirements as well). Compliance, like performance, scalability or observability does not emerge by itself but must be implemented and this requires effort.

Implementing Compliance as an automated part of the software delivery is the answer to allow to deliver fast and effectively while still stay compliant. Another quite important benefit is that it takes tedious work off the shoulder of the teams. I know from personal experience that most software engineers hate process/audit related stuff. They just don’t want to be bothered. Automation will remove that part of the work completely with a very positive effect on people’s work satisfaction.

But how?

Making this work is not simple and implementing all the necessary data collection, aggregation, attestations and report generations into a tool chain is quite a lot of effort. The highest leverage can be achieved if there is a centralised toolchain that is used by all or most of the teams.

Centralising tools needs to be considered very carefully in the context of autonomous delivery teams but if done well it can be an important contribution to reduce the cognitive load of product teams. In the case of compliance automation, the centralised tool chain is an asset that really should be leveraged.

(DevOps and compliance automation is important for me at the moment so I expect to write some more about it in the coming months.)