Enabling Cloud Native CI/CD Workflows in AWS

By: Biswajit Mohapatra

Continuous Integration (CI) and Continuous Delivery (CD) are a set of next generation IT operating practices and principles that enable application development and migration teams to deliver code changes more frequently. When these principles are implemented in the software development lifecycle to track, build, monitor, test and deploy a continual stream of changes to the product, it is referred to as CI/CD pipeline. CI/CD pipeline enables change implementation in the quickest possible time leveraging Agile and DevSecOps.

AWS provides several differentiated services to facilitate implementation of a robust CI/CD pipeline. AWS CI/CD capabilities facilitate effective version control, building, packaging and testing the components. This enables consistency in a frequently changing code environment and leads to optimized delivery of software quality.

Implementing CI/CD Pipeline with AWS

Deploying code changes is a seamless process in AWS Cloud with AWS CodePipeline. AWS CodePipeline is comprised of AWS CodeCommit, AWS CodeBuild and AWS CodeDeploy. CodeCommit is a source control repository where all code changes are committed. CodePipeline creates an end-to-end Continuous Delivery pipeline that detects the code changes and pulls the change to rebuild the components. This facilitates Agile delivery of features automating build, test and release processes. CodePipeline can utilise a parameterized AWS CloudFormation template to rapidly create the development environment. Then, it requests AWS CodeDeploy to install and configure applications in the development environment. CodePipeline helps to run the test scripts before deployment. The process steps are repeated for the test and production environment. The components produced by CodeBuild are promoted to the test and production environment through the Continuous Delivery process. CodePipeline can be configured for the blue-green deployment pattern. With AWS CloudFormation, it can be leveraged to automatically build and test changes to AWS CloudFormation templates before promoting to production. This release process enables rapid and reliable changes to AWS infrastructure.

Benefits of AWS Cloud Native CI/CD Workflows

The business benefits that can be delivered through implementation of AWS Cloud Native CI/CD pipeline are provided below.

• Seamless Integration — Easy integration with GitHub Code repository, On-premises Jenkins build server, custom dashboards to stream deployment logs and Third party Test tools.
• Quick Start — There is no overhead of provisioning or managing the servers. The CodePipeline can be configured and integrated with existing services.
• Configurable Pipeline for Workloads — The pipeline can be configured for Non-containerized, Containerized or Serverless workloads that can be deployed on EC2, ECS,EKS and Lambda.
• High Speed Delivery — Automated build, test and deploy enables testing the code change and detecting defects when impact is minimal and resolution is faster. This ensures quality, productivity and time to market advantage.
• Low Cost — Since there is no server to be provisioned or software to be procured, this leads to cost savings for enterprises.

Use Cases of AWS CI/CD Pipeline

There are 4 implementation use cases that are driving enterprise adoption of AWS Cloud Native CI/CD pipeline.

• Non-Containerized Workloads — Applications running on AWS EC2 instances
• Containerized Workloads — Applications running as Docker Containers on AWS ECS, EKS and Fargate
• Serverless Workloads — Applications running as AWS Lambda Functions
• Static Website Hosting on AWS S3

CI/CD for Non-Containerized workloads:

• Source repository is AWS CodeCommit, AWS S3.
• AWS CodeBuild pulls the changes from the CodeCommit and builds the components. Resulted components are dependent on the build scripts and the configured technology such as Python, NodeJS, Java and Ruby on Rails.
• Target Deployment Environments are Dev, Test and Prod. Target resources are AWS EC2 instances, AWS EC2 Autoscaling Groups and AWS Elastic Beanstalk.
• During Continuous Delivery, promotion of already built components to the higher environments can be done. Build stage can be skipped and components can be deployed on Test and Prod.

CI/CD for Containerized workloads:

• Source repository is AWS CodeCommit, AWS S3, Elastic Container Registry (AWS ECR).
• AWS CodeBuild pulls the changes from the CodeCommit and builds the Docker image.
• The Docker image is deployed to a specific ECS/EKS cluster by specifying the cluster resource naming.
• During Continuous Delivery, promotion of already built Docker images to the higher environments can be done. Build stage can be skipped and Docker images can be deployed on Test and Prod.

CI/CD for Serverless workloads:

• Source repository is AWS CodeCommit, AWS S3.
• The source code for Serverless Lambda applications can be in Java, Java Script, Python and Ruby on Rails.
• The source package consists of AWS SAM template (template.yml) providing the mapping of handler to the exposed URI resource. Source package also consists of AWS CodeBuild Build Specification (buildspec.yml) mentioning the runtime environment for the function. An AWS CodeBuild Build Specification installs required packages and uploads the deployment package to Amazon S3.
• The deployment stage has an action that creates a change set for the AWS CloudFormation stack that manages the Lambda application. A change set specifies the changes that are made to the stack such as adding new resources and updating existing resources. Change sets allows preview of the changes.

CI/CD for Static Website Hosting:

• Source repository is AWS CodeCommit, AWS S3.
• AWS CodePipeline is used to enable deployment of web pages for static website hosting where the version control for the web pages is done with CodeCommit.
• Build stage is skipped. AWS CodeDeploy is used to deploy updated or new web pages in the respective S3 bucket for website hosting.
• This ensures a standard process for updating static website contents.

CI/CD Cloud Native Best Practices on AWS

Based on several complex transformation projects, following best practices is recommended.

• AWS CodeCommit should be configured with the required access controls. In case of cross account, IAM user and AWS service access requirement, access provisioning must follow the standard IAM practices such as role based access, least-privileged access and multi-factor authentication.
• Server-side encryption should be enabled by AWS KMS managed keys (SSE-KMS) where AWS S3 is used as a source repository.
• Explicit authentication should be provisioned if GitHub is utilised as a source repository. This authentication mechanism can be based on AWS managed OAuth Token or a customer managed Identity provider access token.
• CloudTrail logging should be enabled to view the operational logs over CodePipeline. This could be audit logging for the respective components or the changes made to the environments.
• Amazon CloudWatch Events should be configured to detect and respond to the changes in pipeline.

Cloud Native CI/CD Workflows in AWS helps in collaborative software development, software delivery process automation, accelerated innovation, reduced time to customer feedback and continuous learning. To meet the growing needs of DevOps CI/CD implementation, AWS CodeStar which is a cloud-based service brings in CodeCommit, CodeBuild, CodeDeploy and CodePipeline under a single roof together to develop, build, test and deploy applications in AWS. CodeStar Project Dashboard tracks progress of the end-to-end process and manages software delivery from one place. This capability is an enabler for enterprises to seize market opportunities. It’s time now to Go Cloud Native from here !!!

Link to original source

DevOps Institute is dedicated to advancing the human elements of DevOps success through the SKIL Framework: Skills, Knowledge, Ideas, and Learning. Learn more.

Originally published at https://devopsinstitute.com on July 13, 2020.