Continuous Integration & Continuous Delivery

Continuous integration and delivery are standard practices for a lot of projects, helping developers prevent defects in their software, and reliably and quickly release software to users. At Reconfigure.io, we’re no different: all our projects have automated testing and releases, using common services and tools, like Travis CI. You can see this in our Goblin project, where every commit is tested, and tags trigger automatic releases.

Our compiler and build tools for FPGA environments have a few complications that make workflows designed for command line tools and web apps unsuitable. Most of these tools are designed with relatively short build and test times, and are able to install all of these dependencies. Our compiler relies on an extensive integration test suite, which caused us to run into timeouts on Travis CI. Even worse, our build tools and bundled libraries for F1 instances require proprietary vendor tooling, and total end-to-end test times can run for several hours.

Even using our own hosted CI tool, these build times would be a huge maintenance burden on our developers. Waiting for hours at a time to verify a pull request would require developers to keep a close eye on test statuses and it would be easy for a test result to be invalidated by another merge. These times make it tempting for a developer to merge without a full test run, which could result in a build that doesn’t pass tests. To prevent our developers from needing to diligently maintain the merge queue, we use a bot to do this for us: homu.

Whenever a reviewer deems a pull request ready to merge, they tell our bot it’s approved, and it takes it from there. This allows us to have queues of pull requests, while ensuring that we never have our master branch in a state we can’t release.

Instead of delivering our compiler and tooling directly to users on every commit, we bundle these changes up into weekly releases. We deliver the latest compiler internally on every commit, and deploy a staging version of our build infrastructure on every commit. This is where our devs can test out performance improvements and new features in their own code before releasing it to end users.

To give us the benefits of the reliable release process associated with continuous deployment, we automate our release process as much as possible. Releases for our compiler and build tools happen through Slack commands. Commands in our Slack trigger automated jobs to do a release. These jobs will do repository maintenance such as update changelogs, package release notes, and tag the commit. For our compiler, it will upload artifacts. For our build tooling, it will publish Docker images, and update our build cluster.

Everyone at Reconfigure.io can do a release, at any point. By pinning ourselves to weekly releases, we hope to balance the safety of a continuous deployment process while lessening the burden of constant changes on our users.

Building a great developer experience is core to Reconfigure.io, and we’re always looking to improve. If you have any comments or similar experiences, we’d love to talk about it on our forums.