Merging Build Engineering with DevOps

In previous Medium posts I’ve spoken about the challenges with the software development lifecycle, polyglot environments and the build engineering landscape. I’ve also provided best practices for navigating the challenges encountered.

The focus for my post today is considering how to connect Build Engineering with DevOps. Today DevOps is responsible for bringing together everything from coding and building to deploying and monitoring applications. However, underlying all of these processes is build engineering. And as of yet, there’s no tying together of build engineering with DevOps processes.

Recently we hosted a Summit for Customers and Advisors where I presented some of the macro trends in tech that we’ve witnessed over the last 20 years, one being the shift towards DevOps. We’ve moved from developers and engineers being solely responsible for coding and leaving deployment to Ops to the realization that development and operations cannot be so easily separated if we want a productive process. The result of which is DevOps.

The shift to DevOps has resulted in a number of trade-offs: speed for security; agility for certainty and proprietary code for open source:

  • Speed wins over security because time-to-market takes greater priority over fixing vulnerabilities, which we can always address in the next release (hopefully). And retrofitting for security and vulnerabilities after the fact becomes a big blocker for Development and Engineering teams. At the end of the day, the risk of lost profits is greater than the risk of potential unknown threats.
  • Agility trumps certainty in support of speed. For example, iterative pushes to production do away with fixed, known roadmaps in favor of implementing whatever this week’s most important feature/functionality is. The trend toward immutability is also important here, as it’s generally simpler and quicker to throw away something that’s broken/outdated rather than trying to fix/update it.
  • Open source has won over proprietary code because of the lower costs, greater innovation, and transparency it delivers. Rather than focusing on creating proprietary code, developers can quickly iterate on a solution by assembling open source code, accelerating time-to-market. Plus, the ability to take from what has already been created can free up your Dev teams to innovate and build IP for your enterprise efforts.

Organizations that have mature DevOps processes tend to come out on the plus side of these trade offs. Yet the tension between fixing something and knowing if it will break something upstream still remains. You may be able to quickly update and patch a single solution, but how will this impact you once you push to your CI stream? Even for mature DevOps organizations there’s an opportunity cost associated with:

  • Programming Languages — retrofitting your open source language of choice with new versions, dependencies, security patches, etc. adds overhead to your DevOps teams who should be focused on the things that matter: creating and delivering the latest release.
  • Build Engineering — every time the underlying open source language is retrofitted, you need to rebuild all your developer environments, CI/CD systems and production, slowing down time to market. Plus, you’re not certain your rebuild and update will provide performance improvements that warrant the update.

We believe there’s a better way. The key to DevOps is automation, so why not automate the retrofitting and building of your open source languages and seamlessly connect to the rest of your DevOps cycle? You could ensure build reproducibility, dependency management and compliance with your corporate security and license criteria, for example:

  • Continuous Builds — To keep up with your continuous deployments, what if every time a new version of a library or patch for a vulnerability was found, a new build was kicked off and you were notified when it was done?
  • Packaging — what if each continuous build could be packaged in multiple ways — any form factor for any operating system?
  • Build Validation — what if every continuous build could be vetted against your smoke test, inclusive of compliance with your security and license criteria, so you knew whether it was ready to deploy?
  • Build Impact — what if you could understand the performance improvements (if any) that could be gained by taking the latest build?
  • System Update — what if you could update all of your impacted systems across dev, test and even production environments with the latest build using a single command?

All these build engineering capabilities will be possible using the ActiveState Platform, which can be integrated into any DevOps workflow.

Certainty wins over updates because updates may work with one component of your application or one piece of code, but downstream it’s uncertain if it will cause something else to break. The dependencies are multi-fold and a change in one place may have unknown ripple effects. By considering the automation of your build pipeline for open source languages you eradicate the ripple effect downstream. You could ensure build reproducibility, dependency management and compliance with your corporate security and license criteria.

I went into more depth about how we’re working to help companies ship faster and developers code faster in a recent Q&A with SourceForge. You can find that article here.