Continuous Integration and Delivery using AWS and Github
When developing with a team you may want to establish rules for when code is able to merge to your main branch and also control when and how new code gets deployed to a certain environment.
For example, when a team member is ready to merge new code into the main branch:
- We want to first make sure it has at least one other approval from one other team member
- The new changes do not break any current tests
Once a pull request meets the above criteria, we want the merge to our main branch to automatically deploy the changes to a staging environment.
For demonstration purposes, I’ve created a small Express application that you can either recreate yourself or clone.
- The application has one base route that returns “hello” on success.
- You can run npm run test to see the test example
📔Elastic Beanstalk will by default listen on port 8080, so that’s the port this application will listen on.
Create an Elastic Beanstalk application
First, let's create a new Elastic Beanstalk application. For the purposes of this article, we will keep things simple and use the default configuration.
Navigate to the Elastic Beanstalk portal and click Create Application.
Enter an Application name.
Select Node.js as the Platform and use the most up-to-date or recommended Platform branch and Platform version. Then Select Sample application. Finally, click the Create application button.
After the infrastructure is done configuring, the sample application will be available at the URL generated by Elastic Beanstalk.
Create an S3 bucket
We will use S3 to store a zip file containing a new application code. When we deploy a new application, Elastic Beanstalk will look inside the S3 folder for the source code.
- Navigate to the S3 portal and click Create bucket.
- Enter in a unique bucket name.
- Click Create bucket at the bottom of the screen.
Create an IAM user
In CodeBuild, we will use the command line to deploy zip files to S3 and update the Elastic Beanstalk environment. Let’s create a CLI user that has permissions to both AWS service offerings.
⚠️ For demonstration purposes, the new IAM will have full access to S3 and EBS. This is not best practice as you should limit access to only what’s needed.
Navigate to the IAM portal. On the left select Users, then click the Add user button.
Enter a User name and give the user Programmatic access. Then, click the Next: permissions button.
Select Attach existing policies directly.
In the policy selection list, select the checkbox for AmazonS3FullAccess and AdministratorAccess-AWSElasticBeanstalk. Then click the Next: tags button.
On the tags page, click the Next: Review button as we aren’t adding tags for this article.
On the review page, you should see the following:
If the permissions look correct, click the Create user button.
⚠️ On the next screen, make sure to copy the Access key ID and the Secret access key temporarily to a safe location. We will use those values later in the CodeBuild steps.
CodeBuild project for pull requests
First, we want to create a CodeBuild project to install our Node.js dependencies and run any tests. If the tests pass, CodeBuild will notify Github that the tests passed and update a status check for that pull request.
Navigate to the CodeBuild portal and click Create project.
In the Project configuration section, enter in a Project name.
In the Source section, select GitHub as the Source provider. If you haven’t already Connect your Github account to give CodeBuild access. Then, Select Repository in my GitHub account. In the Github repository drop down menu, select the repository that contains the code for this article.
Click the checkbox for Report build statuses to source provider when your builds start.
In the Primary source webhook events section, click the check box for Rebuild every time a code change is pushed to this repository. Then in the Event type drop-down, select PULL_REQUEST_CREATED and PULL_REQUEST_UPDATED.
In the Environment section, select Managed Image. Then choose Amazon Linux 2 for the Operation system. For the Runtime(s) select Standard, then select the most recent Image. Choose Linux as the Environment type. Finally, allow CodeBuild to create a new service role.
In the Buildspec section, select Use a buildspec file. Set the Buildspec name to pipeline/build.yml. In our repository, this is the folder path and file where we will store the steps for this CodeBuild.
Finally click Create build project at the bottom of the page.
CodeBuild project for merges
We also want to create a CodeBuild project that will deploy our application to Elastic Beanstalk when something merges to the main branch. The steps are exactly the same as before, except few small changes:
- In the Source section, set the Source version to main.
- Give this CodeBuild project another name that indicates its for a deploy. For example, I named mine aws-ci-cd-staging-deploy.
- In the Primary source webhook events, select PULL_REQUEST_MERGED as the Event type.
- In the Buildspec section, make the Buildspec name pipeline/deploy.yml.
Additionally, in the Environment section, expand the Additional configuration section and add the following environment variables.
For DEPLOY_BUCKET, Use the bucket name that you created earlier. For EBS_APP_NAME and EBS_ENV_NAME, use the application and environment name from the Elastic Beanstalk application. For the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, use the access ID and the access key from the IAM steps earlier.
These values will be used in the pipeline/deploy.yml file in the repository.
Once you are finished, click Create build project at the bottom of the page.
Create a new branch and pull request
Let’s create a new pull request before we test out the new functionality. I made a new branch with a small change and created a pull request.
As of right now, you should be able to merge to new code without any issues. Lets make some changes to the repository settings so we can make sure the tests are passing and also we have a review from another team member before merging.
Protect the main branch in Github
In the repository for this article, go the Settings tab on the repository. On the left select Branches, then click Add rule.
In the Protect matching branches section, give the branch rule a name, then we need to select a couple values.
- Require pull request reviews before merging, set to 1
- Require status check to pass before merging and Require branches to be up to date before merging
- In the status checks list, select the option that has your CodeBuild project name for pull requests (I have multiple from testing for this article).
- Select Include administrators so you cannot push directly to the main branch.
⚠️ If you are a solo developer or just want to see the updates immediately, you can remove the requirement for number of reviews so the PR can merge only with the tests passing.
Once you’re finished, clicked Save changes at the bottom of the page.
If we navigate back to the pull request, we can see one of the new requirements:
Let’s update the test to fail. In the app.test.js file, change the expected response text to hell from hello, then push up the changes. After a little while, you should see the pull request update with the results from the CodeBuild execution.
If you click on the Details link, it will take you directly to the logs of the CodeBuild project to see why the test failed.
Change the test back so it will pass and push the changes back up. The pull request will now show to tests passing.
Now, either remove the branch requirement of one other review, or have another user approve the pull request.
Finally, merge the pull request into the main branch.
Now, if you visit the URL in your Elastic Beanstalk application portal, you should now see the new code, which returns hello.
How it works
To learn more about what’s going on, check out the build.yml and deploy.yml in the pipeline folder of the repository. The code is very simple and it gets the job done.
For the deploy steps, we basically:
- install the dependencies
- create a zip file for everything we need for Elastic Beanstalk
- Push the new zip file to our S3 folder
- Create a new application version for our Elastic Beanstalk application, and set the source to the S3 file, using the build number as the version
- Update the environment to get the new source code and deploy it to our instances
If anything goes wrong with a deployment, in the Elastic Beanstalk portal, you can select an application version and use select Actions > Deploy to redeploy that version.
Thanks for reading!