Setup Continuous Integration based on SalesforceDX

Using Jenkins and GitHub

In Summer’17 Salesforce released SalesforceDX (Beta) — a brand new developer experience approach to building on Force.com platform. Using SalesforceDX (SFDX) capabilities you can in real-time create a new environment (Scratch Org) and deploy the sources where it will be continuously tested.

This blog post is meant to walk you through a setup process of a popular continuous integration server called Jenkins and sync it with GitHub. The goal is to have an integration between Jenkins and Github repository using Github Webhooks. The build will automatically run on every check-in to the repository.

“Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily — leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible”
— Martin Fowler

Before you start with actual configurations and setup process you should be ready with next requirements:

  1. Create Salesforce Org with enabled DevHub to manipulate Scratch Orgs. You also can create trial org: sign-up page
  2. Install SFDX CLI on the server/machine where Jenkins will be hosted. Here is the official tutorial how to do that
  3. Salesforce Dreamhouse App Github repository has used as an example project. You should open the repository and fork it to use as you own test repo. But you also may use your personal project Github repo
  4. The build script uses jq. It is a bash utility to work with JSON strings. The build script will use this utility. You should also install it along with CLI

Note: you can use Jenkins Custom Tools Plugin to install CLI and jq on the fly. Also, if you plan to use another integration server instead of Jenkins (Travis), you should install CLI and jq using the service strategy. For example on “before script” build stage.

Install Jenkins

To run builds right after changes were pushed to the repository, you should host Jenkins on the server and configure Github Webhooks. Basically, in most cases your Jenkins environment will be hosted somewhere on a client server or a cloud.

However, you can start from free/cheap hosting to experiment CI process. This guide uses DigitalOcean cloud to host Jenkins. Next steps will guide you through installing Jenkins on DigitalOcean:

  1. Create your own server with Ubuntu OS
  2. Install Jenkins on the DigitalOcean server

Note: if you prefer to start from local Jenkins environment, follow next steps to install it on the local machine. But, be aware that Github Webhooks work only with Jenkins service that hosted on open to the world server.

  1. Download the installer from official site
  2. Install Jenkins using downloaded file
  3. Once Jenkins installed it should open the page http://localhost:8080/, if not you should open the page manually
  4. Provide secret password from the file in the Jenkins environment to unlock the server. Jenkins should display the path to the file
  5. Install suggested by Jenkins plugins
  6. Configure Admin user for Jenkins server
  7. Then you should be redirected to the page with button Start using Jenkins that will open you home page

Create raw Jenkins push build

Open your Jenkins server environment where you should see something like this:

Click on the link to Create new job. Then provide a name of the build and select a type Freestyle project:

Setup the build Git repository in Source Code Management section with your own repo like displayed on the image below:

Check Poll SCM option in the Build Triggers section to provide an ability to run build through Github Webhook:

Note: if you work with Jenkins that is hosted on your local machine, you will not be able to use Github Webhook. You should enter cron expression H/15 * * * * to the Schedule text area. Jenkins will poll the repository for changes periodically based on the entered expression. The expression provided before configuring Jenkins to poll every 15 minutes.

Great, that is what you need for now. You will continue to configure the build in the further section.

Configure JWT-Based Authorization Flow to DevHub org

This step is really important in setting up CI process. Every build will authorize to the DevHub to manage Scratch Orgs during script logic execution. You will create a connected app and a digital certificate to allow CLI to authorize into the DevHub org. The steps described in this section will be handled once. Then you will be able to implement different builds using created the connected app and the digital certificate.

  1. Create a Private Key and Self-Signed Digital Certificate. In general, JWT Flow idea is that you generate the certificate (server.crt) and save it to the connected app. Then the build uses private key (server.key) to sign the certificate. And if the private key is approved by the connected app the authorization process is successful.
    server.key — The file used during authorization command from CLI force:auth:jwt:grant. In your case, Jenkins will hold this key and it will be used on every build
    server.crt — The certificate which you will use while creating the connected app for CLI
  2. Create Connected App in your DevHub org. To prepare the application for JWT Flow follow the official documentation. Be sure you configured callback URL correctly. Also, save the Consumer Key from the created application. You will use this key during further Jenkins configuration.

Configure Github Webhooks

The idea is that Jenkins should listen for changes in the repository and run the build when they occur. But it is expensive to go to the repository by the scheduled manner. You will create the repository Github Webhooks. Github will notify Jenkins about changes on every push using Webhook. Then Jenkins will run the build script.

Note: if your Jenkins hosted locally, it should poll Github by the schedule cron expression configured before and you have to omit Webhooks configuration.

Open the project Github repository settings and navigate to the Webhooks menu:

Click on the Add webhook button. In the open form, you should provide the URL to your Jenkins server with one parameter. The pattern for this URL: http://<jenkins_url>/git/notifyCommit?url=<path_to_githubrepo>. See the example on the picture below:

Click on the green Add webhook button. Now the repository is ready to send notifications to the Jenkins server on every check-in. It will run the build that you have configured in the previous sections.

Push the changes to your repository in the master branch. Then navigate to the Jenkins build dashboard to verify that the build is started. You should see the running build like on the image below in the Build History list:

Create Jenkins Environment Variables

Jenkins will use properties like username, connected app consumer key, private key file, Github access token during build script execution. These type of data you should store in the Jenkins Credentials. Credentials provide the secure way of saving properties and allow you to not hardcode them to the build script. Then you will use these credentials to create the build environment variables. The script will use environment variables for different authorization processes.

Open Jenkins home page and navigate to Credentials menu:

Then click on Jenkins link on the second table in the Store column. It will open the page with listed scopes where credentials stores. There should be the link to the global scope Global credentials (unrestricted). Open the global scope by the link and you should see Add Credentials option in the left side menu.

Follow the next steps to configure variables:

  • Click Add Credentials. Select Kind as Secret Text. Enter the username from your DevHub in to the Secret input field. And provide an ID of this credentials as HUB_ORG. Click Ok button to save credentials. The ID will be used from build configurations. Basically, it is like a name of the credentials. You can also enter your custom ID
Username credentials creation form
  • Create credentials for connected app consumer key like you did it for username. However, enter the ID as CONNECTED_APP_CONSUMER_KEY and to the Secret enter the consumer key from the connected app you created before
  • The next credentials is a little bit different from previous two. Again, click Add Credentials and select Kind as Secret File. Then for the ID enter JWT_KEY_FILE and select server.key file which you generated in the previous section
  • As was mention before, the build script will set Github commit status based on the result execution result. To do that it is require Personal Access Token of your Github account. Open the settings of the account and navigate to Personal access token in the left-hand side menu:

Click on the button to Generate new token and select repo checkbox to provide the access to the repositories by the generating token. Also, you should provide the description:

The open Jenkins page and create credentials for the Github Personal token like you did it for the DevHub username and for the consumer key. The Secret value should be your token and ID can be named as GITHUB_ACCESS_TOKEN:

  • The last thing to do is to enable required Jenkins environment variables that is build in. They assigned when the build is started. It is $BUILD_URL for example. The build URL will be used to navigate to the build console output from Github site.
    Open Manage Jenkins > Config System > Enable Environment variables.

Now you are ready with all environment variables. The next starting point is to update the Jenkins build with all CI logic using prepared resources and build script.

Complete Push build

You must bind credentials to environment variables to use them in the script.

Open the build page on Jenkins. Then to the build Configure page. Next, find Build Environment section and check User secret text(s) or file(s). Now create bindings of the variables and credentials like displayed on the image below:

Note: The variable name can be custom on your preference. But the recommendation is to stay with suggested variants because these names will be used in the script.

Find the Build section and add Execute shell build step. Then copy and paste the script from the Gist snippet below:

The script has two variables that you should change. Enter your account username to GITHUB_ORG variable. And the name of the repository to GITHUB_REPO:

GITHUB_ORG='ruslan-kurchenko'                                         
GITHUB_REPO='sfdx-dreamhouse'

Now the build is complete. Follow to the next section to test the CI logic execution.

The process testing

To trigger the build execution you should commit changes to the repository master branch.

Note: you can configure branches using Jenkins Git plugin. For example to the build only for master, staging and dev branches.

The build script set the status of the commit to The build is processing… while it actually in progress:

In case of failure the status will be also updated depending on the type of problem:

Authorization to DevHub failed
Source pushing to the Scratch org failed

And you should see green checkmark if the build finished successfully:

Note: you can navigate to the build console output by clicking on Github build checkmark.

Conclusion

This blog post include the most basic Continuous Integration configuration. There are a lot of great features that you can build around complete skeleton. You also can configure Jenkins to build Github Pull Requests. And even setup Continuous Deployment process to deploy the build to the Sandbox.


If this article was useful for you, please sign it with an 👏 or share with your friends. It will force me to stay work on articles like that more, thanks!


Ruslan Kurchenko is an Advanced Salesforce Engineer from Kiev, Ukraine. He is passionate of Salesforce, JavaScript and Modern WEB Development. Connect with him on LinkedIn, Instagram or say Hi! on Twitter.