DevOps: CI/CD explanation
CI/CD enables the tech companies to improve their products many times per day. Here’s what you need to know to do the same.
What is CI/CD?
CI/CD is a way of developing software in which you’re able to release updates at any time in a sustainable way. When changing code is routine, development cycles are more frequent, meaningful and faster.
“CI/CD” stands for the combined practices of Continuous Integration (CI) and Continuous Delivery (CD)/continuous Deployment (CD).
What is Continuous Integration?
Continuous integration (CI) is a software development practice in which developers merge their changes to the main/master branch many times per day. Each merge triggers an automated code build and test sequence, which ideally runs in less than 10 minutes (based on the time resource which is utilizing). A successful CI build may lead to further stages of continuous delivery.
If a build fails, the CI system blocks it from progressing to further stages. The team receives a report and repairs the build quickly, typically within minutes.
All competitive technology companies today practice continuous integration. By working in small iterations, the software development process becomes predictable and reliable. Developers can iteratively build new features. Product managers can bring the right products to market, faster. Developers can fix bugs quickly and usually discover them before they even reach users.
What is build stage?
In continuous integration (CI), this is where we build the application for the first time. The build stage is the first stretch of a CI/CD pipeline, and it automates steps like downloading dependencies, installing tools, and compiling.
Besides building code, build automation includes using tools to check that the code is safe and follows best practices. The build stage usually ends in the artifact generation step, where we create a production-ready package. Once this is done, the testing stage can begin.
The build stage starts from code commit/code download at CI tools and runs from the beginning up to the test stage
Build automation verifies that the application, at a given code commit, can qualify for further testing. We can divide it into three parts:
- Compilation: the first step builds the application.
- Linting: checks the code for programmatic and stylistic errors.
- Code analysis: using automated source-checking tools, we control the code’s quality.
- Artifact generation: the last step packages the application for release or deployment.
Step 1: Compilation of your code
Step one of the build stages compiles the application. On compiled languages, it means that we can generate a working binary, whereas on interpreted languages, we confirm that we have the required dependencies and tools to successfully build the application.
The result can be a binary file, an installable package, a website, a container image — the main thing is that we have something that we can run and test.
Step 2: Analyzing Your Code
Programming requires discipline. We must solve hard problems while following good practices and abiding by a common coding style. Automated analysis tools help you keep out gnarly code at bay. In general terms, there are three classes of code analysis tools:
- Linting: linters improve code quality by pointing problematic and hard-to-maintain bits.
- Quality: this class of tools uses metrics to find places where code can be improved. The collected metrics include the number of lines, documentation coverage, and complexity level.
- Security: security analysis tools scan the code, flag parts that may cause vulnerabilities, and check that dependencies don’t have known security issues.
Step 3: Preparing your Application for Release
The released package will include everything needed to run or install the application, including:
- Application code.
- Installation scripts.
- Dependencies and libraries.
- Application metadata.
- License information.
Final package will be published at the server and configure the database and other connectivity as per the requirement.
Software product needs to be tested like other product before it’s delivery. The obvious way to test something is to use it for a while to make sure it behaves as expected. This is the way most developers in the past used to check their work before shipping it.
After all, you have written it, you have it on your computer, you know how to run it, and you know how to quickly test it as you make changes.
The flip side is that you as the creator are biased. You might forget to test some parts or might not realize the ramifications of a change. Also, you can’t be 100% sure that your code will work in an environment that isn’t your development machine.
Even if you do everything right, testing is a boring task — a time-consuming activity where skilled, creative professionals are forced to do mindless drudgery. Historically, the countermeasure was hiring people to test, making developers happy and removing biases.
This seemingly win-win solution makes things worse often. Developers lose the overall picture and awareness of all the moving parts when they aren’t involved in testing. Testers, on the other hand, spend an enormous amount of time just making things work and must bother developers whenever they find something they don’t understand.
The role of a QA team when testing is automated.
Testers are part of what is known as the Quality Assurance (QA) team. They are paramount for shipping high-quality products, so it’s important to let them do their job efficiently. They do not have to test everything every time, nor catch unhandled exceptions or 404 errors. They are there to help developers and companies to deliver good quality software.
Requirement of the Continuous Integration?
In business, especially in new product development, we often don’t have time or ability to figure everything upfront. Taking smaller steps helps us estimate more accurately and validate more frequently. A shorter feedback loop means having more iterations. And it’s the number of iterations, not the number of hours invested, that drives learning.
For software development teams, working in long feedback loops is risky, as it increases the likelihood of errors and the amount of work needed to integrate changes into a working version software.
Small, controlled changes are safe to happen often. And by automating all integration steps, developers avoid repetitive work and human error. Instead of having people decide when and how to run tests, a CI tool monitors the central code repository and runs all automated test on every commit. Based on the total result of tests, it either accepts or rejects the code commit.
Continuous Delivery is the ability to get changes of all types — including new features, configuration changes, bug fixes and experiments — into production, or into the hands of users, safely and quickly in a sustainable way.
Our goal is to make deployments — whether of a large-scale distributed system, a complex production environment, an embedded system, or an app — predictable, routine affairs that can be performed on demand.
We achieve all this by ensuring our code is always in a deployable state, even in the face of teams of thousands of developers making changes on a daily basis. We thus completely eliminate the integration, testing and hardening phases that traditionally followed “dev complete”, as well as code freezes.
Continuous deployment goes one step further than continuous delivery. With this practice, every change that passes all stages of your production pipeline is released to your customers. There’s no human intervention, and only a failed test will prevent a new change to be deployed to production.
Continuous deployment is an excellent way to accelerate the feedback loop with your customers and take pressure off the team as there isn’t a “release day” anymore. Developers can focus on building software, and they see their work go live minutes after they’ve finished working on it.
CI and CD pipeline
CI and CD are often represented as a pipeline, where new code enters on one end, flows through a series of stages (build, test, staging, production), and published as a new production release to end users on the other end.
Each stage of the CI/CD pipeline is a logical unit in the delivery process. Developers usually divide each unit into a series of subunits that run sequentially or in parallel.
For example, we can split testing into low-level unit tests, integration tests of system components working together, and high-level tests of the user interface.
Additionally, each stage in the pipeline acts as a gate that evaluates a certain aspect of the code. Problems detected in an early stage stop the code from progressing further through the pipeline. It doesn’t make sense to run the entire pipeline if we have fundamental bugs in code to fix first. Detailed results and logs about the failure are immediately sent to the team to fix.
Because CI/CD pipelines are so integral to the development process, high performance and high availability are paramount for developer productivity.
Prerequisites for doing Continuous Integration
The basic prerequisites for implementing continuous integration include:
- Automating builds.
- Automating testing.
- More frequent commits to a single source code repository, and
- Providing visibility of the process and real-time access to CI status to the team.
Development team need to follow the full CI/CD process so that productivity will be rise as well as it will increase velocity of the entire business.
A typical development workflow
You can apply continuous integration in most software projects, including web applications, cloud-native microservices, mobile apps, system software, IoT / embedded systems and more.
Here’s a typical continuous integration workflow developers practice on a daily basis:
- A developer creates a new feature branch of code in GitHub, makes changes in the code, and commits them.
- When the developer pushes her work to GitHub, then GitHub will build the code via Github action/other CI tools can be used where build specific build action has been configured.
- If any error detected in the CI pipeline, then developer gets a notification via email (which email id has been configured at the time of GitHub login)
- If the developer has raised a pull request and at the time of build if any issue occurs and the build is failed, then requester will get the email notification as per the build fail status.
- Otherwise, the user gets a notification that CI has passed (status green) and next stage will execute like artifacts creation along with test cases execution and artifacts will be ready for the deployment at the lower environment (dev, stag, qa)
- Once another developer has verified the changes in a peer review, the author can merge the new branch of code into the master branch and again build will be published for production deployment and also test cases need to be passed for that one.
Benefits of continuous integration
CI provides numerous benefits for your software development team including improving developer productivity, automating the development process, improving code quality and software reliability.
Improved developer productivity
The biggest benefit of practicing continuous integration is increased developer productivity. Continuous integration frees developers from manual tasks and the pains of integrating their code with other parts of the system. Instead, they can focus on programming the logic that delivers the features the business needs. Or they can use the time CI has saved them to invest in professional growth.
Teams that work in a fast CI feedback loop can deliver more value to customers than teams that struggle to integrate each other’s work or suffer from a slow pipeline that doesn’t scale.
Deliver working software more often
Continuous integration is a way for your team to automatically build and test every single change in the source code. This is the foundation that enables your broader software delivery process to be efficient, resilient, fast, and secure.
Find bugs earlier, fix them faster
The automated testing process can include many different types of checks:
- Verify code correctness.
- Validate application behavior from a customer’s perspective.
- Compare coding style with industry-standard conventions.
- Detect security updates in third-party dependencies.
Building these tests into your CI pipeline, measuring and improving the score in each is a guaranteed way to maintain a high quality of your software.
Example CI/CD workflows with the GitHub tools
Here’s a simple example of fully automated CI/CD (Continuous Deployment) pipeline:
CI/CD shouldn’t be more complicated than necessary
So here all the test cases will be passed and after approval of the pull request it will go for build and deploy stage based on the environment (dev, stg, prod)
Benefits of CI/CD
CI/CD is much more which can reduce the manual work and via CI/CD automation of application build, test and deployment can be handled.
Deliver software with less risk. CI/CD pipelines standardize release processes across projects. By testing every change in source code, we reduce the chances of introducing bugs.
Release new features more frequently. A CI/CD pipeline can visualize your entire path from commit to production in a single screen. You can navigate across stages, spot inefficiencies, and optimize your process. By removing the roadblocks to productivity, you enable your company to succeed.
Deliver the product that users need. Delivering updates often leads to more user feedback. You can take advantage of that by A/B testing features or testing early versions of products with real customers. This way you avoid investing too much in features that your customers don’t need and focus on those that matter.
Improve developer productivity. Engineering teams that don’t practice CI/CD often work under stress. There are constant fires for bad deploys and hard-to-fix outages. Developers write a lot of code that never gets used also at the time of build process via linting those can be identified and removed. Long-lived feature branches are too big to get proper peer review, so code degrades in quality. On the other hand, CI/CD guides product management to optimize for user impact. Developers deploy code while it’s fresh in their minds. The result is a happy engineering team.
When is CI/CD not feasible?
“Continuous Delivery is great but some places it is not applicable” e.g:
- Your customers don’t want continuous updates to their systems.
- Regulations restrict how software can be updated. For example, continuously updating software used in aerospace, telecom, and medical industries is not an option.
- Proper monitoring will be required after the deployment along with the skill resources
Even in a CD-averse environment, teams can enjoy the benefits of easy deployment and keeping the system in a deployable state. Via CI development team can create the package or artifacts and have to wait for the deployment once in a blue moon as per the requirement.