Our Trip from Jenkins v1 to v2

By Max Mautner

Max Mautner
Mar 28, 2019 · 3 min read

Continuous integration is at the heart of Personal Capital’s process for building software, and Jenkins has been our chosen CI tool since the days of old.

However, several big problems faced the team as our engineering headcount grew:

  • our builds would fail due to un-reviewed changes to our build configuration
  • our number of projects grew as we pursued new avenues for the business
Image for post
Image for post
Back in the Day at Personal Capital

Here are some of the benefits we gained from upgrading our Jenkins version from 1 to 2:

Managing Build Configuration as Code

In Jenkins 1 you must declare your project’s build configuration through the web UI — giving little granular control over reviewing changes and verifying changes work before they are made(!).

Jenkins 2 allows us to track our projects’ build configuration in code, using a Jenkinsfile

This gives us critical visibility and control into:

  • an audit log of changes made to build configurations
  • visibility into what our build pipelines do: “My build failed” => “Look in the Jenkinsfile to reproduce the build failure”
  • verify that configuration changes work before merging to master (i.e. via merge request)
  • reduce the # of Jenkins jobs from an ad-hoc collection of Jenkins projects to a 1–1 Github repository<>project mapping — giving the team more transparency into what builds are for
  • get to reuse a custom, tested DSL for build pipeline definition across all our projects1 . This was not possible with the UI-based build configuration of Jenkins 1

Build stability has improved since the v2 upgrade, hand-in-hand with morale :)

Polling vs. Real-time Push

This might seem small, but engineers are expensive and need feedback fast.

Jenkins 1 required us to poll Github for new pull requests once a minute.

Jenkins 2 allows us to adopt webhooks so that builds are initiated as soon as you push your commit to Github — no delays.

Horizontally Scaling our Build Compute

Personal Capital currently operates under a 1-week sprint process.

This means in practice that the end of sprint there is a noticeable spike in build activity as team-members push revisions to their code and integrate feedback from code reviews.

This resource usage spiking was difficult to handle with Jenkins 1, but we now support auto-scaling of our build cluster, allowing us to manage our server costs in off hours and support our large and growing team of engineers when it’s crunch-time.

Image for post
Image for post
Source: https://thumbs.gfycat.com/KindlyCharmingKrill-size_restricted.gif

Secure Build Infrastructure

At Personal Capital, we take security seriously.

We manage all employee accounts and access via single sign-on using Okta, which allows employee access to be managed globally across all services we use.

This includes enforcing multi-factor authentication and allowing permissions revocation if necessary.

Additionally, Jenkins 1 long-term support (LTS) guarantees are obviously weaker that Jenkins 2. Jenkins 2 will make it easier to stay current with the latest vulnerability patches and features.

Ease of Managing Permissions

One final benefit we realized from making the Jenkins 2 move is that user access management with our single-sign on (SSO) provider allows us to onboard employees faster, eliminating a one-off credential that managers had to go fishing for on a team-member’s first day!

Going forward, we’re looking forward to the big, bright future!

If you’re curious about the software that the Personal Capital team build’s then checkout our our award-winning iOS and Android apps.

  1. https://github.com/jenkinsci/JenkinsPipelineUnit

Thank you to Rohit Iyer, Priyesh Kannan, Ehsan Lavassani for their input & feedback on this article.

Personal Capital Tech Blog

We are Personal Capital's Engineering team.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store