Secure and Speed up CI/CD with AWS CodeArtifacts
Making accelerated software delivery your top priority
A common thing that unites DevOps professionals is the term CI/CD. For the sake of the wider audience reading that would read this article, this term is called Continuous Integration and Continuous Delivery. I aimed at keeping this article short so I will save you some time not to talk about anything else other than a newfound favorite AWS service which is CodeArtifact. Some DevOps maestro eagerly seek ways to always improve what’s already working while others believe in the quote that says “if it ain’t broke, don’t fix it”. The top on the list of what I eagerly seek to improve in a CICD pipeline has always been speed. We are not all going to be Usain Bolt on the track and field but his fame inspires me to always deliver at his near pace in my own domain.
During the build phase of an automated pipeline, software needs packages to build a usable service. This is where most pipelines spend so much time. You can use AWS CodeArtifacts to save your pipelines the time it takes to go over the internet to download software packages.
Two key services in focus
- AWS CodeBuild
- AWS CodeArtifact
About AWS CodeBuild
AWS CodeBuild is a fully managed build service on Amazon Web Services that compiles your source code, runs unit tests, and produces artifacts that are ready to deploy. It eliminates the need to provision, manage, and scale your own build servers.
What is AWS CodeArtifact?
It is a fully managed software artifact repository service for DevOps, Developers, Infrastructure Engineers, and organizations to help securely store and share the software packages, used in their development, build, and deployment processes. It works with common package managers like Gradle, Maven, npm, yarn, pip, and twine.
Essential Terminologies
- Domain: higher-level entity where you can organize your repositories
- Repository: contain packages of any supported type
- Package: is a bundle of software and the metadata that is required to resolve dependencies and install the software.
- Asset: An individual file stored in CodeArtifact that is associated with a package version.
As soon as a package version is successfully published, it can be requested using a package manager from within a pipeline.
How it works
- Application dependencies are pulled from Public Artifact repositories
- The dependencies are stored and can be published on AWS CodeArtifact securely.
- Apply policies on the domain and share it with multiple accounts.
- Developers can pull application dependencies for development.
- CI/CD pipeline tools can fetch application dependencies at build time and publish build artifacts back to the repositories.
Benefits from AWS CodeArtifact
1. Create approved centralized repositories for software packages
2. Securely store, publish, and share packages
3. Use (IAM) to control who has access to the packages
4. Use AWS CloudTrail to track packages that need to be updated.
5. Encrypt packages using KMS
6. Share repositories with other AWS accounts
Quick Walkthrough — CodeArtifact for NPM CodeBuild Pipeline
Pre-requisites: install the following
- AWS CLI
- NPM CLI
Next: Let’s create a domain, repo, and package.
# create-domainaws codeartifact create-domain --domain $DOMAIN --region $AWSREGION
# create-repositoryaws codeartifact create-repository --domain $DOMAIN --domain-owner $AWSACCOUNTID --repository $REPO --region $AWSREGION
# create an upstream repository for your $repo repositoryaws codeartifact create-repository --domain $DOMAIN --domain-owner $AWSACCOUNTID --repository npm-store --region $AWSREGION# add an external connection to the npm public repository to your npm-store repositoryaws codeartifact associate-external-connection --domain $DOMAIN --domain-owner $AWSACCOUNTID --repository npm-store --external-connection "public:npmjs" --region $AWSREGION#associate the npm-store repository as an upstream repository to the $REPO repositoryaws codeartifact update-repository --repository $REPO --domain $DOMAIN --domain-owner $AWSACCOUNTID --upstreams repositoryName=npm-store --region $AWSREGION#configure your npm package manager with your $REPO repositoryaws codeartifact login --tool npm --repository $REPO --domain $DOMAIN --domain-owner $AWSACCOUNTID --region $AWSREGION
Example:
awsguru@aws-works:~# aws codeartifact login --tool npm --repository $REPO --domain $DOMAIN --domain-owner $AWSACCOUNTID --region $AWSREGIONYou should see the output below;
Successfully logged in to codeartifact for npm
npm install <library-name>npm install -g forever# to view the package you just installed in your $REPO repositoryaws codeartifact list-packages --domain $DOMAIN --repository $REPO --region $AWSREGION
Using npm packages in CodeBuild
Run the login command from the pre-build section of buildspec.yaml . This will configure npm to fetch packages from CodeArtifact
Cache your Dependencies
Now, let’s speed up the pipeline by enabling local caching in CodeBuild to reduce the number of dependencies that need to be fetched from CodeArtifact for each build.
Ensure that the IAM role attached to your Build project has the necessary permission to access CodeArtifact. Construct your buildspec.yml file with this line.
cache:
paths:
- '/root/.npm/**/*'
Some results
From this sample project, I was able to improve my pipeline speed by 72 seconds of build time.
Congratulations. You’ve successfully secured and saved your pipeline the time it takes to travel to the internet to fetch packages and additionally, you’ve also saved more time by reducing the number of dependencies that needed to be fetched from CodeArtifact.
Stick around, I will be exposing more AWS services you can explore to speed up your software delivery pipelines in my coming articles.
Stay safe!
MyCloudSeries is a training and consulting firm with expertise in Cloud Computing and DevOps. We assist organizations in their DevOps strategies, transformation, and implementation. We also provide Cloud Computing Support contact us at www.mycloudseries.com
