DevOps: Jenkins as a service
1. Introduction
1.1 What is Jenkin
Jenkins is an open source continuous integration (CI) server written in Java that can be self-hosted to automate the build cycle of any project. Jenkins provides CI services for a software project, which can be started via command line or web application server.
Before going into details about how Jenkins provides CI, I believe that it is important for you to understand what is CI. CI is a development practice in which developers integrate code into a shared repository several times a day. Tests run on a non-developer (tester) machine automatically when someone pushes new code into the shared repository.
The workflow of CI provides you fast feedback. If a change breaks a build you will know shortly after committing the code to a shared repository. The console also provides detailed log messages so you can identify what changes lead to the broken build. In addition, by using an artifact repository in the CI server you can deploy builds to other developers and testers.
Jenkins is heavily used in CI to allow code to build, deployed, and tested automatically. Let’s consider the situation where the complete source code of an application was built and then deployed on the test server for testing. It is an ideal way to develop software, but in practice, it is not practical. The developers will have to pause their work until they get the test results, which can take hours. The entire process is manual so there is a chance for human error(s), which can lead to time wasted.
Naturally, with the above scenario, it leads to slow software delivery and unhappy customers. This leads to a huge demand for a system to exist where developers can trigger the above process to happen automatically. This is where Jenkins comes in to fulfill the role in CI.
With Jenkins, you can hook it up to repositories such as Mercurial, SVN, Git, etc. So, all developers have to do is configure Jenkins to check the repositories periodically and when there is something new it will attempt to build, test, and deploy the new code. This help saves a massive amount of the developers’ time so they can focus on their work.
1.2 Pros of Using Jenkins
- Jenkins is open-source (free) and has a large community
- There are many plugins available that make Jenkins easier to use
- Jenkins is written in Java, which means it is portable
- Developers can spend more time elsewhere: Most of the integration and testing is managed by an automated build and testing system.
- Speeds up development: Most of the integration work is done automatically, which results in fewer integration issues. In the long run, this saves both time and money over the lifespan of the project.
- Higher quality software: Issues are detected early and resolved almost right away. This keeps the software in a state where it can be released at any time safely.
1.3 Cons of Using Jenkins
- Cost and Time: Jenkins must be run on a server (cost) and often needs the attention of a system admin (time). You can’t expect to set up Jenkins and then have it run itself. The system will require frequent updates and maintenance.
- Requires some investment to start using: Some main barriers that most teams encounter is the initial setup, procrastination, or failed previous attempts to set up Jenkins. Many teams understand that it is best practice, but will neglect it for more urgent coding work. If Jenkins was implemented in the past but was not successfully maintained then all you get is wasted effort as well.
- Tests can break: Tests can break and that requires updating. If they are not updated then those tests stop delivering value.
Heavily workload Jenkin which regularly causes issues with plugins, user management, EC2 instances types, etc. As we are now heavily relying on Jenkins it is important to create a more flexible, scalable and stable Jenkins setup.
To achieve this we need separate Jenkins instances for specific purposes to optimize the Jenkins configuration fit-for-purpose and allow its owners/users to quickly make adjustments while they don’t need to worry about the stability of the Jenkins instance, named “Jenkins as a Service”. For example, a development team should have its own Jenkins instance for the specific applications they’re building and should be able to manage plugins, users, global configuration, etc., while they can rely on a stable and up-to-date Jenkins instance.
2. Use cases
As a developer I want to be able to…
· Have a maintained and up-to-date Jenkins environment per application
· Have Slaves (Linux EC2, Windows EC2 or physical Mac Mini’s in office) attached
· Have SSH access to my own Slaves to install specific build/deploy dependencies
· Have the Slave’s OS and system dependencies managed by Operations (FPT)
· Deploy artifacts via Jenkins to a Test environment in VPC or RWE DC (via S3 upload or SSH)
· Let Jenkins have access to development tools like SVN, Bitbucket and Sonar
· Have Jenkins admin rights myself to manage plugins and global configurations
· Access any Jenkins dashboard via DNS on a Smart Client
· Start jobs and access logs from Mobile Access network for ‘non-sensitive’ jobs (would have)
· Have a temporary on-demand Jenkins environment to test new plugins or jobs (nice to have)
· Deploy artifacts via Jenkins to an Acceptance or Production environment (via S3 upload or SSH)
· Have a stable running Jenkins instance for which technical issues are quickly resolved
3. Automated Orchestration Deployment
3.1 Jenkin Server
3.2 Jenkins Linux Slave
4. Chef recepies
