
AWS DevOps — Part 1 CICD
Continuous Integration, Continuous Deployment (CICD) is an essential concept in modern Software Development, it is a cornerstone to the Development Operations (DevOps) methodology. DevOps automates much of the process of developing, testing, and deploying software from source code to executable files. Software matures quicker than traditional methodologies such as waterfall.
Benefits of DevOps
Software development is an iterative process, meaning there are multiple versions of the same product, think of the Windows Operating system there are multiple versions — Windows XP, Windows 10, etc. DevOps speeds up the development cycle by integrating development, testing, and deployment/operations as one continuous process. This allows code to “fail fast” by automating the code maturation process providing constant feedback.
While not an entirely new concept, tools such as Docker, Jenkins, and Chef have facilitated the expansion of DevOps practice. These tools have made continuous development and deployment possible, moving away from methodologies such as waterfall where development is slower. AWS provides analogs to these tools, making it a highly scalable platform for developing and hosting applications.
Traditional Software Development, development, testing, and operations are seen as separate practices
DevOps allows for continuous integration between software development cycles allowing for less time between iterations.
This series of tutorials will explore how to implement DevOps with AWS:
- Source Code Management — CodeCommit, CodePipeline, CodeBuild, CodeDeploy
- Monitoring, Auditing, and Messaging — CloudWatch, X-Ray, and CloudTrail, SQS, SNS & Kinesis
- Microservices and Containers — ECS, ECR, & Fargate
- Security & Encryption — KMS
At the end of this series you will see how AWS facilitates DevOps.
CodeCommit
CodeCommit utilizes Git to create a source code repository that is managed by AWS. Version control is a pivotal tool, allowing developers to revert files to previous versions based on the hash (unique id) of a commit (code added to a repository). Developers can collaborate on the same code simultaneously by creating branches allowing individual developers to change code without affecting other developers. Branching also facilitates code maturity by allowing developers to review individual changes before merging it with a “master” branch that contains the accepted changes. Git is an important tool all developers should learn to use, below is a general outline of how to use Git.
CodeCommit has no storage limit and is fully managed by AWS, which comes with many benefits. Security is handled through IAM policies, data is encrypted by KMS at rest and SSH or HTTPS when in transit. Simple Notification Service (SNS), a service you will learn about in the next tutorial along with Lambda trigger notifications developers receive when actions such as a deletion of a branch or pushes to the master branch, allowing developers to reaction in real time. CodeCommit can be integrated with other tools such as CloudWatch to create Event Rules or CodePipeline.
CodePipeline
Continuous Delivery, a key principle in DevOps that develops code to maturity. CodePipeline allows for a visualization of the processes that help mature code, such as building, testing, or deploying. Pipelines allows sections of code to move to a fully mature state, while allowing developers continue to development.
As code matures various checkpoints known as stages that define code maturity based on actions such as unit tests, integration or peer review. Stages perform pipeline actions sequentially or in parallel. Artifacts created from each stage are collection in S3 and passed to the next stage.
SNS can be integrated to notify developers when changes happen in CloudWatch. The CodePipeline console will notify users when a pipeline fails. CloudTrail can be used to audit API Calls. IAM service roles need to be attached to allow CodePipeline to execute each stage.
CodeBuild
CodeBuild works in conjunction with CodePipeline to facilitate continuous integration by creating software packages. CodeBuild compiles source code and runs tests based on the stage of maturity of the code. The mature software packages are ready to deploy with no servers that need to managed or provisioned and no build queue.
Docker is used to create CodeBuild images that can be reproduced and modified. Builds are based on source code and defined by a Buildspec.yml file, which must be stored at the root of a software package along with environment variables such as credentials. There are 4 phases of a Buildspec
- Install — install dependencies
- Prebuild — preparation for build
- Build — execute build
- Post Build — processes artifacts such as zip files
CodeDeploy
CodeDeploy is the final stage by deploying code to a non Elastic Beanstalk EC2 instance. Each EC2 instance must run a CodeDeploy agent that continuously polls for updates. Instances are grouped by deployment types such as dev, test, and prod. Artifacts from S3 or CodePipelines as well as source code repositories such as Github are deployed from CodeDeploy. CodeDeploy also supports Lambda functions and Docker containers.
Deployments are defined by a variety of attributes. The app name and computing platform (EC2 or Lambda must be defined) as well as deployment configurations that define success and failure including the groups that are being deployed. The deployment types include in-place or blue/green. IAM permissions need to be defined to give EC2 or Lambda permissions to pull from S3 or Github, a service role allows CodeDeploy to perform the deployment. The application revision is defined by the code and appspec.yml, while the target revision defines the target app version that is being deployed.
The appspec.yml for EC2 or appspec.yaml for Lambda or ECS must be placed in the root of a software package. The file section determines where and how to copy the source code to the filesystem. Hooks are deployment instructions that define the order in which the code must be deployed
- AppStop — stops the running app
- DownloadBundle — install dependencies
- BeforeInstall — prepares install
- AfterInstall — post install
- ApplicationStart — starts application
- ValidateService — verifies installation
Deployment configurations determine how the application should be deployed. Developers can determine if the application should be deployed to an EC2 instance one instance at a time, half the instances and the last half of the instances if the first half deploys, all at once good for development, or custom configurations. If a deployment fails, the EC2 instance will stay in a failed state, new deployments will be deployed to failed instances first, you can redeploy a failed instance or rollback to a previous successful deployment. The instances that get deployed to are determined by EC2 tags or directly to autoscaling groups.
Conclusion
AWS Code services allow for a seamless integration of code from development to deployment with CodeCommit, CodePipeline, CodeBuild, and CodeDeploy. A tool called CodeStar can be used to combine all these services and integrate with CloudFormation and CloudWatch to create a CICD project. This suite of tools allows for an automated way to mature code.
The next tutorial will discuss how AWS services are audited and monitored. This allows for application to coordinate and inform developers of events taking place within the VPC.
