Build Engineering: Tools can Address a Myriad of Daily Challenges
Build engineering is critical to building code. Yet while it’s critical, it’s complex and varied. Each programming language typically has its own set of tools. And there’s also variance in the tool set for each operating system platform on which you’re building.
As critical as build tools are to the software development process, it’s surprisingly difficult to find information about the topic online. And while you’ll regularly hear “I hate <insert name of build tool>, but all the rest are worse.”, there hasn’t been much done in terms of improving or evolving build engineering.
At ActiveState we ourselves have wrestled with build engineering for open source languages for the past 20 years. From dependency conflicts to compiler errors to library vulnerabilities to system incompatibilities, we’ve seen all the things that make builds fail and releases get delayed across numerous operating systems and architectures.
As a result, we’re passionate about helping organizations eliminate the pain associated with building, updating, patching, and managing the dependencies of their open source languages. Consistent, reproducible, standardized builds are our goal to create the de facto standard for build engineering.
Typically, the four challenges in the build pipeline stem from:
- dependencies;
- lack of reproducibility: stakeholders across the software development lifecycle (SDLC) are constantly facing issues with getting a consistent language version;
- keeping code ‘to spec’: licenses, patches, upgrades, security vulnerabilities, etc.;
- technical debt: if updates aren’t regularly done you will eventually end up with a system you don’t dare upgrade.
Part of addressing the challenges faced by enterprises in their build pipeline was a deeper understanding of the pain points. So we ran a survey of 1,400 developers regarding the challenges faced with their open source run times this year to set our baseline — click here if you’d like to get the published report.
One of the key findings of the survey was the load on developers. Over 75% of developers surveyed said they spend at least part of their time managing dependencies and their development tools. Combine this number with over 50% of respondents stating they start a new project at least once per quarter.
The net result is that developers need to dedicate a lot of time to each the building a new version of an open source language per project started and the subsequent maintaining of the language distribution for each team in the organization. This time produces a significant cost to an enterprise.
But what if you could automate the build of your open source languages? If you could select only the packages you need, the platform you want to deploy on, and then have that language distribution automatically updated for you when new versions come out or vulnerabilities are found?
You’d get a consistent build that you can continuously and rapidly distribute to all team members across any of your SDLC environments: development, test, and production.
And if you have a consistent version of your language everyone on your team has all the necessary pieces. Because when you build the language, you’re also building in the dependencies and creating the environment by which you drop the code.
The outcome is a consistent build environment from development through CI/CD and into production where you no longer need to hear “well, it works on MY machine.” Moreover, you’d accelerate build times.
Other build systems are meant to just compile the code and assume you have all the dependencies installed already. What we believe, is you can take the language, you tell us what your dependencies are and over time we’ll manage it and track it. You could have your checklist, inventory and wiki all-in-one. Your environment is ready but better yet, you’ll have the list of your assets so you can audit and “inventorize” as you want.
The value is exponentially greater for polyglot enterprises. Every language has its own way of doing things and the 4 key challenges of your build pipeline (from above) mushroom in impact with each new language added.
And it’s not enough to just automate build engineering. There needs to be a systematic, standardized and unified approach that takes into account differences in the languages you use, but employs a consistent workflow or process to output the type of build you need specific to your project or use case.
That’s how we’d like build engineering to evolve. Continuous, reproducible builds to enable continuous deployment and free up developer time and give enterprises control of the risks associated with their open source code.
You’d reduce build overhead and remove maintenance of dedicated tooling and infrastructure so the team can better spend their time on high-value work.
ActiveState will be examining these challenges in detail in an upcoming webinar with DevOps.com. Save your spot for our live discussion on the evolving role of build engineering in managing open source.
