SDM 1.0 Platform for Software Delivery

Published in

The Composition

6 min readDec 5, 2018

Originally posted on blog.atomist.com

We are excited to announce the GA release 1.0 of our Software Delivery Machine (SDM) framework and associated open source projects.

Over the past several months, through numerous milestones and release candidates, we’ve been working hard to finalize this important release so that you can automate your software delivery in modern and powerful way — in code.

At Atomist we strongly believe in the principles outlined in the Software Defined Delivery Manifesto: our software delivery challenges should be addressed using the same tools, expertise, and processes we apply to the applications and services we develop. Everything in our delivery process should be implemented using versioned and tested code and not mountains of shell script hacks and hundreds of lines of copy/pasted YAML.

With an Atomist SDM you and your team can automate every task in your daily life as software developers such as applying consistent formatting to your source code, managing your changelog, labelling issues when fixes get deployed into production, auto-merging your PRs when reviews and builds succeed, or updating license notice files in every repository across your entire organisation.

More importantly, an SDM enables you to program your delivery process, like running builds and integration tests, managing deployments and rollbacks, and publishing artifacts, using the same tools and approaches you are using for writing you application code. An SDM enables you to treat your delivery process as a first-class citizen — an engineered approach to delivery.

Not only can an SDM apply the same delivery process to your entire organization and therefore dramatically increase the level of consistency, it can also help build up a shared understanding of the entire delivery process by bringing all relevant activity into chat and making it actionable. An SDM connected with the Atomist service provides a sophisticated, programmable delivery flow with native chatops that works with existing Git services and CI tools.

For more background on SDM, I recommend watching this recent talk by Rod Johnson.

Noteworthy Features of SDM 1.0

I want to highlight some of the more technical aspects of our framework that we previously didn’t discuss much.

Custom Goals

An SDM is an open and extensible framework that lets you customize almost every aspect of its out-of-box behavior. This extensibility lets you code your own goal implementations that can operate alongside the standard goals like Autofix, Build or KubernetesDeploy. Those custom goals can integrate any pre-existing tool or API you might be using into your delivery process.

Below is an example of a custom goal that sends a text message after a successful production deployment using Twilio’s Node.JS package.

const textMessageGoal = new GoalWithFulfillment({
    uniqueName: "send-text-message",
    environment: ProductionEnvironment,
}).with({
    name: "send-text-message",
    goalExecutor: async gi => {
        const { configuration, sdmGoal } = gi;        const client = require("twilio")(
            configuration.sdm.twilio.accountSid,
            configuration.sdm.twilio.authToken);        await client.messages
            .create({
                body: `Deployment of ${sdmGoal.sha.slice(0, 7)} succesful`,
                from: "+15017122661",
                to: "+15558675310",
            });
    },
});const productionGoals = goals("production deploy")
    .plan(productionDeployment).after(stagingDeployment)
    .plan(textMessageGoal).after(productionDeployment);

Testing and Debugging

Using code to define your delivery lets you use unit tests and debugging capabiltites of your IDE during devlopment. This takes out a lot of the guesswork usually connected with updating CI configuration: Let’s make that change and see what happens when I push to our CI server?!? A couple of examples of unit tests for autofixes etc are available in addHeader.test.ts of our atomist-sdm.

Extension Packs

SDM Extension Packs are a way to extend the functionality of your SDM by adding a new NPM dependency to your package.json and—depending on the extension pack—add it to your SDM by calling sdm.addExtensionPack(). You can browse the NPM registry to get an overview of the constantly growing list of SDM Extension Packs.

Goal Approval

Making decisions about who gets to deploy to production or who can overwrite a certain failing code review are an important aspects of every organization’s delivery process. The SDM provides extensions points for you to plug in strategies to implement your authorization requirements in code rather than configuration. These strategies are then invoked every time a user or automation is trying to start or approve a goal. We are using a custom goal approval mechanism in our own atomist-sdm to collect additional confirmation via a Slack direct message. See machine.ts for an example.

Consistent Audit Trail

Many of our customers require consistent audit trails for their software delivery process: they want to know exactly when, who, or what triggered a certain action in the process of moving software into production. They want to keep track of this across all systems they use in the process; across issue tracking, SCM, CI, and deployment tools. The event-driven nature of an SDM allowed us to build recording audit information directly into the framework and expose the information to reporting tools via GraphQL.

Runtime Awareness

One of the many advantages of using a higher-level framework to implement your software delivery is the ability of the SDM framework to abstract away the underlying runtime environment and transparently optimize the execution of goals to what the runtime supports. When you develop your SDM or use an SDM on your laptop in local mode, it limits resource consumption and runs goals in the same process. But when you deploy your SDM into a Kubernetes cluster it automatically forks goals into their own Kubernetes jobs utilizing the advanced scheduling capabilities of Kubernetes. This works without changing the code of your SDM.

Configuration and Secrets

Running an SDM and enabling it to access your systems to execute goals often requires access to secrets like API keys and tokens. To make configuring an SDM easy and flexible, it can be configured using various sources like JSON files, environment variables, TypeScript code or lookup of secrets in secret stores like Hashicorp Vault. This rich configuration support is built into the core framework.

Monitoring

Following the Observable principle of the Software Defined Delivery Manifesto, an SDM exposes relevant information like execution times and counts, failure rates, heap and GC statistics to StatsD and Prometheus and can easily be integrated with other monitoring tools.

Conclusion

It has been amazing to see what our customers are able to achieve with an SDM, from automating the creation of OpenShift clusters to orchestrating the deployment of applications to AWS using Terraform to keeping project dependency versions across hundreds of repositories in sync. We at Atomist are excited to be on this journey with you, transforming software delivery through the power of code!

To get started with SDM, head over to the developer quick start to start realizing the advantages of software defined delivery. We’re interested to see what you make your SDM do.