Continuous Integration Using AWS CodeBuild
Continuous integration is a DevOps software development practice where developers regularly merge their code changes into a central repository, after which automated builds and tests are run.
In this article, I will explain the process of Continuous Integration using AWS CodeBuild and store the built jar file into S3 bucket and copied that jar file into EC2 using a Lambda function and send the build status to developer E-mail using AWS SNS Service.
AWS CodeCommit:- AWS CodeCommit is a version control service hosted by Amazon Web Services that you can use to privately store and manage assets (such as documents, source code, and binary files) in the cloud.
2.AWS CodeCommit is a secure, highly scalable, managed source control service that hosts private Git repositories. AWS CodeCommit eliminates the need for you to manage your own source control system or worry about scaling its infrastructure. You can use AWS CodeCommit to store anything from code to binaries. It supports the standard functionality of Git, so it works seamlessly with your existing Git-based tools.
- Open AWS CodeCommit console and choose Create project.
2. Create a secure repository to store and share your code.
3. After completion of the repository creation push your source code into your AWS CodeCommit repository.
4.Project structure in AWS CodeCommit.
5.Buildspec.yml file
6. Create a lambda function for auto start the code-build for your project when there is a new commit on your source code and send the build status notification to developer E-mail using AWS SNS Service.
7. Configure your CodeCommit details in your lambda function as shown below.
8. We can find the Lambda trigger configuration details in your code-commit repository settings as shown below.
AWS CodeBuild:- AWS CodeBuild is a fully managed build service in the cloud. AWS CodeBuild compiles your source code, runs unit tests, and produces artifacts that are ready to deploy. AWS CodeBuild eliminates the need to provision, manage, and scale your own build servers.
- Open AWS CodeBuild service console and choose to Create project.
2. Create your build project.
3.Select source provider and Repo name. Here I have selected source provider as GitHub.
4. Select your project environment.
5.Select buildspec.yml file.
6. Select artifacts for storing your build file.
7. After completion of the code-build project click on start build
8. After completion of build check the status of the build and if you want to show the build details click on view details
9. JUnit test cases result in the console log.
10. Build status and upload artifacts details in log…
11. After build success, the artifact is stored in S3 bucket and set the event in S3 for lambda function.
12. Open your bucket, go to properties ->Events->select All object to create events and select your lambda function.
13. The us-west-2_dev_CICD_PROCESS lambda function copies the built jar file into EC2 from S3 bucket and sets the CLASSPATH for the jar in EC2.
14. If you want to connect to EC2 using Lambda function you must install boto3, paramiko, pycrypto packages into your local machine.
15. Copy all packages, EC2 keypair file(.pem) and lambda code into one folder.
16. Folder name and lambda name must be the same and the folder should be compressed and uploaded into a lambda.
17. us-west-2_dev_CICD_PROCESS lambda code as shown below.
18. After completion of download file from S3 to EC2, connect your EC2 instance and type ls command then it shows your build jar file.
NOTE:-
1. In AWS CodeBuild project configuration I selected source provider as GitHub but if you want to select source provider as a code-commit, the building process can be automated using lambda function I already mentioned how to configure a code-commit repository in a lambda function.
2. If any new commit on your code-commit repository then immediately the lambda trigger and start the build process and send the build status to developer E-mail using AWS SNS service.
Lambda Source code:-
import json
import boto3
import datetime
import time
from dateutil.tz import tzlocal
codecommit = boto3.client(‘codecommit’)
client = boto3.client(‘codebuild’)
client1 = boto3.client(‘sns’)
def snsNotify(devSnsTopicArn, snsSub, snsMessage):
response = client1.publish(
TopicArn=devSnsTopicArn,
Message=’CodeBuild Status is::: ‘ + snsMessage,
Subject=snsSub,
)
return response
def lambda_handler(event, context):
ram = event[‘Records’][0][‘codecommit’][‘references’][0][‘commit’]
ts = time.time()
sysDateStr = datetime.datetime.fromtimestamp(ts).strftime(‘%Y%m%d’)
sysTimeStr = datetime.datetime.fromtimestamp(ts).strftime(‘%H%M%S’)
finalpath = sysDateStr + ‘/’ + sysTimeStr + ‘/’
devSnsTopicArn = ‘arn:aws:sns:us-west-2:XXXXXXXXXXX’
snsSub = ‘[alert]build status Project’
print(sysDateStr)
print(sysTimeStr)
response = client.start_build(
projectName=’code build project name’,
# sourceVersion=’NONE’,
artifactsOverride={
‘type’: ‘S3’,
‘location’: ‘S3 bucket name’,
‘path’: finalpath,
‘namespaceType’: ‘NONE’,
‘name’: ‘NONE’,
‘packaging’: ‘NONE’
},
timeoutInMinutesOverride=123
)
codebuildRes = response[‘build’][‘id’]
print(codebuildRes)
print(event)
print(response)
time.sleep(180)
response2 = client.batch_get_builds(
ids=[codebuildRes, ]
)
snsMessage = response2[‘builds’][0][‘buildStatus’]
finalOutPut = snsNotify(devSnsTopicArn, snsSub, snsMessage)
return “hello lambda”
Download Source code by using this URL ….. https://github.com/prabhakarreddyn/ContinuousIntegration.git
That’s all for Continuous Integration using AWS CodeBuild and send the build status to developers E-mails and finally copy the latest build jar file into EC2 server.