Indexing multiple Jenkins masters

It’s probably fair to say we are “all in” on using Jenkins as our ci server. There’s varying opinions about Jenkins around here and it’s not perfect, but overall its’ excellent and highly extensible. There’s really not much better if you need to build on multiple platforms and integrate the release process into the build pipeline as we’ve done. However….

We ran Jenkins here using a single master and a Docker swarm cluster with dynamic containers to handle the slave nodes (clone army article). This provided (and continues to provide) great scalability for build slaves since they are virtually infinite. However, as we moved to more of a microservices architecture, the Jenkins master itself became our bottleneck. So we embarked on a project to split the master into multiple masters (which we’ll feature in a future post).

We went from 1 master to around 7 currently (all running in Docker). This then led to the inevitable problem of finding anything! So we created…

Super Jenkins

Super Jenkins is a Jenkins job indexing app to index jobs running across multiple Jenkins masters.

For configuration, you provide a list of the master servers and an optional image to display per server (ours are named after celebrity butlers!). Super Jenkins then indexes on-the-fly jobs across all servers. It’s pretty fast, searches are fuzzy and the UI is really simple. Users can also “star” their favourite jobs across servers and see a smaller view if they wish.

I think the lesson learned here is while breaking the single master up into multiple masters was clearly the way to go, we didn’t quite anticipate how hard it would be to find things across multiple masters. We had to re-vector and create Super Jenkins to solve this problem as we went along with the project.

Super Jenkins is on GitHub