.NET Core serverless CI/CD on AWS using CodePipeline and CloudFormation
We are constantly creating new microservices in my team at carsales and we have chosen to deploy these using a serverless architecture on AWS.
To reduce the time and effort of staying truly serverless, I have implemented a serverless CI/CD pipeline and .NET Core solution template that we use to create all our new services.
In this article, I’ll provide you with an ASP.NET Core template project which includes a SAM template to deploy an API Gateway and the Lambdas necessary to host your APIs. I’ll also include a CloudFormation template which will create a CI/CD pipeline for your application.
ASP.NET Core Template
No matter how many microservices you have running on your serverless architecture, you’ll need a way to automate deployments. You’ll also need to generate client libraries for each microservices to share with other services.
I have created a .NET Core template solution that I use to clone when I’m creating a new microservice. You can find the template here: https://github.com/kavehfa/serverless-templates/tree/master/Templates/NetCoreProject
Solution Structure
API Project: This is the ASP.NET Core project with a default controller. I’ve added automatic Swagger generation to the project using nswag library. Go to /swagger path to view the Swagger page.
I’ve also added error handling and validation attributes to make it easy to handle and log errors. AWS X-Ray is also enabled to allow monitoring of your service interactions with other service.
Client Project: This project has one file which implements a client library for your API. You could use the DLL or publish to Nuget to allow other projects to make API calls to this service. ApiClient.cs is auto-generated by ClientGenerator project.
ClientGenerator Project: You can run this project to generate a client library for the API.
Right-click API project and select “view in browser” and then run this project to generate the client library. You could configure code generation settings in Program.cs file.
Test Project: An XUnit test project with some useful libraries installed to make your life easier when writing tests.
Build Files
buildspec.yml: Is used by the build stage of the pipeline. It will install some dependencies such as Node.js to execute transform-template.js and will test, build and publish the API project.
template-params.json: It includes parameters that need to be passed to the CloudFormation template. I only pass the environment name to the service template as a parameter and read any other parameters directly from AWS SecretsManager.
transform-template.js: Handles transformation of any files before packaging the build files. This script changes the CodeUri value to set it to the publish folder and also sets a random value on UpdateDate to force all environment variables to be updated every time CloudFormation deploys Lambda functions.
CI/CD pipeline template
The CI/CD pipeline template provides a very simple process which does not consider QA and manual approval processes but you could easily modify the template to add more stages to the pipeline.
This template will create a three-stage pipeline to:
- Fetch the code from GitHub
- Test and build the project
- Deploy the CloudFormation template
To create the pipeline go to your AWS console, navigate to CloudFormation and click Create Stack.
Upload the provided template and click next. Select a name for your template. I prefer to prefix my stacks with the environment name. For example if this is a pipeline for dev environment then I’ll prefix it with dev-CICD-{service-name}.
You’ll need to provide three parameter values to deploy this stack:
- ArtifactBucket: This is the bucket name where your artifacts will be stored. Artifacts are files generated during build and packing that are necessary for your deployment.
- EnvironmentName: Such as dev, stage, prod etc… This value is used to name the service deployed using this Pipeline. In real-world scenarios, you’ll be deploying more than one service and you probably want each service deployed in a certain environment to have a similar naming schema.
- ServiceStackName: The name of the stack deployed using this pipeline. The final name of the stack is generated by combining {EnvironmentName}-{ServiceStackName}.
For example, if I have a service called Payments Service and I’m deploying this service in dev environment then I’ll set the value of this parameter to payments-service and my final stack name will be dev-payments-service.
Click next on the rest of the screens and set any settings specific to your environment and hit “Create Stack”. Pipeline will start to be deployed. You can monitor the progress in the events section.
After deployment is done you’ll need to set your GitHub account settings so the Pipeline can pull your changes and start the build process.
From AWS Console, navigate to CodePipeline, click on the newly created pipeline and click “Edit”.
Then click “Edit Stage”
And then click edit again!
Click Connect to GitHub and select your repository and branch that you want to deploy. When you are ready, click “Release Changes” to start the process.
Conclusion
Using similar processes and project structures my team and I at carsales are able to quickly and easily create new microservices to react to rapidly changing requirements. We are also able to ensure best practices are followed by creating CI/CD pipelines to deploy our services, allowing faster and safer collaboration and quality assurance.
Thanks for reading and please get in touch if you have any feedback/questions.
