Build a CICD pipeline for WSO2 EI with Jenkins
WSO2 Enterprise Integrator is an open-source, light-weight, battle-tested, hybrid integration platform which comes with Apache 2.0 license. In this post, I’m going to discuss how you can build a CICD pipeline with WSO2 EI and Jenkins to automate the development and deployment process.
Installing Jenkins locally
I will be using a locally running Jenkins server to demonstrate the workflow. You can install Jenkins locally by referring to the below documentation.
I followed the simple steps for installing on Mac OS.
$brew install jenkins-lts
After the installation, it won’t start automatically. You should start it by running “jenkins-lts” in your terminal.
Then you need to run the “post-installation setup wizard” by accessing the Jenkins URL which is http://localhost:8080. Make sure you install the required plugins (I installed all the recommended plugins).
Developing Integration with WSO2 Integration Studio
WSO2 Integration Studio is the much improved development IDE which is built on top of eclipse. You can download that from the following URL.
Once you have downloaded the Integration Studio, you can start building an integration project by following the documentation. In this article, I won’t be talking about building integrations with Integration Studio, rather I will use an already built simple Integration project which is residing in my github repository.
Cloning the source code into the local repository
I will be using an already developed integration project which you can download from the following URL.
Make sure you fork the above repository into your GitHub account and then clone that into your laptop. This repository contains 2 sample integration projects which have an ESB configuration project and a composite application project (CAR) which is used to deploy the artifacts into the ESB server.
Start the WSO2 EI server locally
Now we need to start a local WSO2 EI instance locally. You can download the latest version of WSO2 EI from the below link.
I will be using the WSO2 EI 6.5.0 for this tutorial. You can start the WSO2 EI with the below command.
$sh <EI_HOME>/bin/integrator.sh
Building the CICD process for WSO2 EI with Jenkins
In this tutorial, I’m going to explain how to build a pipeline so that when you make a change to the Integration project, it will automatically trigger a build and deploy the latest code to the WSO2 EI server.
As depicted in the above figure,
- Integration Developer will make a change to the source code
- That change will be committed to the source repository (Git)
- Once the change is committed, the Git hook will get triggered and start a Jenkins job
- Jenkins job will build the artifacts and deploy that to running WSO2 EI server
Configure Jenkins for CICD process
You have to log into the Jenkins server with the password you have set during the post setup step above. Once you log in, you can click on “New Item” button on the left side. It will take you to the next page where you need to select a project type. Give a name (“wso2ei-jenkins-tutorial”) and select the “Freestyle project” type and click “OK”.
Once you create the item, it will take you to the next page where you can configure the job with instructions. In that page, add a sample description to the “Description” box and go to the “Source Code Management” section. There, you have to select the “Git” option and provide the local file location where you cloned the github repository which was mentioned above. Keep the rest of the parameters as it is and go to the “build” section and click on “add build step” button and select “Execute Shell” option. This will allow us to select a build step by writing a shell script to build the integration project. In the text box, copy and paste the following shell script.
cd JenkinsCICDSample
mvn clean install
cd HelloCompositeApp
mvn clean deploy -Dmaven.deploy.skip=true -Dmaven.car.deploy.skip=false
The below screenshot captures the configurations mentioned above.
Now we have configured the jenkins to build and deploy the integration artifacts in to the WSO2 EI server with a single click.
Run Jenkins job
Now if you go to the jenkins dashboard, you will see the job which is just created. Click on the job name and execute the “Build Now” button on the left side of the UI.
Once you run the above job, it will create the CAR file and deploy into the running server. You can send a curl request and validate the functionality.
curl -X POST -d “<test/>” -H “Content-Type: application/xml” localhost:8280/services/HelloProxy
The above command will send back a response with the following message.
{“message”:”Hello world!”}
That’s pretty cool!. But wait a sec. We have to click a button to enable this build job right? That’s not exactly what we wanted. We don’t want to click buttons when there is a change. Let’s see how we can configure a git hook to automate the process.
Configure git hook to automate the build process with every commit
Now let’s go to the directory where you cloned the github repository. There, go into the following directory
cd <REPO_CLONED_LOCATION>/.git/hooks
In this directory, you can find several executable files with “.sample” extension. These are sample hooks come with git by default. You create a file with name “post-commit” without any extension and place the below content into that file.
#!/bin/sh
Once you save the above file, make sure to make it executable by running the below command.
$chmod a+x post-commit
Now we have configured a git hook at the source control side. Then we have to listen to this hook from the Jenkins job. To do that, go to the jenkins dashboard and click on the above created job name and click “Configure” option. Now you will go back to the original wizard where you configured the source location and the shell command. In the same interface, go to the “Build triggers” section and click on the “Poll SCM” option and “Save” the configuration.
Now we are all set. Let’s go and make a modification to the source code. You can open the “HelloProxy” service and modify the message within the “PayloadFactory” mediator to something different.
<?xml version=”1.0" encoding=”UTF-8"?>
<proxy name=”HelloProxy” startOnLoad=”true” transports=”http https” xmlns=”http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<payloadFactory description=”Set the payload” media-type=”json”>
<format>{“message”:”Hello Jenkins!”}</format>
<args/>
</payloadFactory>
<respond description=”Send the response”/>
</inSequence>
<outSequence/>
<faultSequence/>
</target>
</proxy>
After you make the above change, make sure you commit this change to the repository.
git add JenkinsCICDSample/HelloESBProject/src/main/synapse-config/proxy-services/HelloProxy.xml
git commit -m “Updated the message
All set. Now if you go to the Jenkins dashboard, you will see that there is a job automatically triggered and deployed the new CAR file to the server.
If you see the above image, you can observe that it has been started by and SCM change. Now let’s send a message to the server and verify our change.
curl -X POST -d “<test/>” -H “Content-Type: application/xml” localhost:8280/services/HelloProxy
You will get the updated response as below.
{“message”:”Hello Jenkins!”}
That’s it. We made it!
Extending the project into multiple environments and automated testing
If you want to extend this to a real world scenario where you have multiple environments and automated testing, you can refer the below github repository which contains a reference implementation for such a requirement.
