Multisite Drupal based application deployment using Jenkins
Introduction
In this article you will learn how to build and deploy the Multi-site Drupal application on a remote application server using SSH protocol via Jenkins. You will also learn how to fetch the branches from the GitHub repository at runtime for code build.
Drupal multisite feature is an effective solution for large organisations who often need more than one website but a collection of related sites, which would be easy to manage. Drupal multisite has become popular among education websites, government websites, corporate websites, and so on. Multiple sites combined in the same Drupal installation that is the basic essence of Drupal multisite feature. Instead of one site, you have a collection of websites that are created, managed, deployed, and updated from one place because they share one codebase. Each multi-site has its own database, configuration, files and base domain or URL. For the multisites Drupal application setup refer to the link.
Jenkins is a popular open source tool to perform continuous integration and build automation. Jenkins manages and controls software delivery processes throughout the entire lifecycle, which includes build, document, test, package, stage, deployment, static code analysis and much more. Have you ever wondered why Jenkins has gained so much popularity, especially over the recent years? One of the major factors that contribute to its popularity is the flexibility of Jenkins pipeline and if you’re looking for such a Jenkins pipeline tutorial, this blog is your go-to.
Below points are covered in this article
1. Background
2. Main section
3. Prerequisites
4. Assumptions
5. Steps to follow
6. Pros
7. Cons
8. Conclusion
9. References
1. Background
Today many of you face problems in automating the deployment of multi-site Drupal applications and thus it results in manual intervention. In the manual deployment process you might end up making mistakes in command execution. In this article you will learn to automate the Drupal multi-site application deployment process.
2. Main section
Pipelines are Jenkins jobs enabled by the Pipeline plugin (formerly called “workflow”) and built with simple text scripts that use a Pipeline DSL (domain-specific language) based on the Groovy programming language. Most functionality provided by the Groovy language is made available to users of Scripted pipeline, which means it can be a very expressive and flexible tool with which one can author continuous delivery pipelines.
Scripted Pipeline is serially executed from the top of a Jenkins file downwards, like most traditional scripts in Groovy or other languages, providing flow control, therefore, resting on Groovy expressions.
3. Prerequisites
For this article below prerequisites are required:
- Basic understanding of Jenkins Pipeline
- How SSH protocol works
- The fundamentals of Git and GitHub
- The knowledge of Drupal application’s compile, build and deploy process using drush (Drupal shell) commands.
4. Assumptions
4.1 For this article, we assume that you have already installed Jenkins with default suggested plugins and on top of it we need to install below plugins from Jenkins plugin manager.
- SSH Pipeline plugin
- Git Parameter plugin
4.2 During this article, we will reference below servers for theoretical explanation and command executions -
- Jenkins Server — 192.x.x.5
- Jenkins Port — 8080 with HTTP protocol
- Drupal application Dev server — 192.x.x.6
- Drupal application UAT server — 192.x.x.7
4.3 The Drupal application discussed during the article have 3 sites example as below:
- site1.com
- site2.com
- site3.com
So the source-code base i.e., business logic for all the sites is the same and the only difference is the user-interface for which we have different themes. For building an application based on a theme we are using the input choice parameter in Jenkins pipeline.
4.4 In general we can have 3 application environments which are Dev (Development), UAT/Stage (User-Acceptance-Testing) and Prod (Production/Live) and based on the article we should have the connectivity to all the application environments from the Jenkins server.
4.5 The application directory structure on the remote Drupal application is assumed to be -
For parent directory — /var/www/multisites/web/
For custom theme folders based on the mult-sites functionality -
- /var/www/multisites/web/themes/custom/site1
- /var/www/multisites/web/themes/custom/site2
- /var/www/multisites/web/themes/custom/site3
4.6 The installation of the Drupal application-specific dependencies are assumed to be done by the system/infra admin.
5. Steps to follow
Here’s a list of implementation breakdown which we will cover in this article:
5.1 Steps to configure passwordless access for the remote server.
5.2 Adding the properties for parameterized job
5.3 Details of SSH Pipeline plugin and support
5.4 Preparing the commands for the execution.
5.5 Complete Pipeline structure
5.1 Steps to configure passwordless access for the remote server
- For configuring the passwordless access between Jenkins Server and the remote application server we need to configure the public-private key-pair authentication.
Command:
Use below command for public-private key generation on Jenkins server
ssh-keygenCheck the instruction during key generation and enter the passphrase
Enter the passphraseOnce keys are created then copy the public key from Jenkins server to the remote application server
ssh-copy-id -i id_rsa.pub ritesh@192.x.x.6ssh-copy-id -i id_rsa.pub ritesh@192.x.x.7
- Once passwordless connectivity is set between the Jenkins server and remote application server then configure the private key in Jenkins. For adding credentials follow below steps:
Open Jenkins url : http://192.x.x.5:8080/credentials/store/system/
Note: You can enter your Jenkins IP and respective port.
- Click on the domain where you want to add the credentials. By default there will be Global credentials, so click on it.
- On the left side click on add credentials. For more detail follow the below screenshot:
Now once we have added the credentials then we need to bind it with the Jenkins pipeline.
- Create a Jenkins pipeline, let say we have created pipeline project with name : Drupal Application Pipeline
- Once the project is created then hit url:
http://192.x.x.5:8080/job/Drupal Application Pipeline/pipeline-syntax/
Note: In above url Drupal Application Pipeline is the name of the project. You can enter your Jenkins IP and respective port.
- Open the Pipeline snippet generator and select withCredentials: Bind credentials to variables
- Select withCredentials option from sample step and fill in the details as below screenshot.
- After above step, copy the generated pipeline syntax and prepare the below pipeline structure:
node {withCredentials([sshUserPrivateKey(credentialsId: ‘Remote-Server-Access-Creds’, keyFileVariable: ‘PASSWORDLESS’, passphraseVariable: ‘PASSWORD’, usernameVariable: ‘USERNAME’)]) {//Some code block}}
5.2 Adding the properties for parameterized job
The parameters directive provides a list of parameters that you should provide when triggering the Pipeline. The values for these user-specified parameters are made available to Pipeline steps via the params object.
The parameter could be from either of the following type -
- String
- Text
- Boolean
- Choice
- Password
In this article, we will cover the Choice parameter and we will also cover Git parameter which is derived from the ‘Git Parameter Plugin’. We will use three parameters as below:
- Choice parameter: It will be used to define the multiple site names of the applications and the configured parameter values will be static.
- Choice parameter: It will be used to display the number of application servers where the code needs to be compiled, build and deploy. To simplify the things, we will use ‘hostname-ServerIP’. All the configured parameter values will be static.
Example:
UAT-192.x.x.7
DEV-192.x.x.6
For generation of choice parameter snippet follow below steps:
- For pipeline snippet generator hit the below url:
Open Jenkins url : http://192.x.x.5:8080/credentials/store/system/
Note: You can enter your Jenkins IP and respective port.
- Select ‘properties: Set jobs properties’ from the sample step.
- Tick the option ‘This project is parameterized’ which will show the various options to add the parameters.
- Select choice parameter and fill the details as below screenshot:
- Next add another choice parameter with remote application servers.
3. Git Parameter: This parameter will be used to fetch all the list of GitHub branches at runtime and will display for the selection. You can select only one branch as it will be a type choice parameter.
- For generating the git parameter snippet please enter the details as per screenshot below:
- Now we have added all the three parameters in the pipeline snippet generator so click on the generate pipeline script button for the code.
The Jenkins pipeline upto this step will look like as below:
node {withCredentials([sshUserPrivateKey(credentialsId: ‘Remote-Server-Access-Creds’, keyFileVariable: ‘PASSWORDLESS’, passphraseVariable: ‘PASSWORD’, usernameVariable: ‘USERNAME’)]) {properties([parameters([choice(choices: [‘site1’, ‘site2’, ‘site3’], description: ‘Select site for building the application’, name: ‘sitename’), choice(choices: [‘UAT-192.x.x.7’, ‘DEV-192.x.x.6’], description: ‘Select the server for deploying the application’, name: ‘Application-Servers’), gitParameter(branch: ‘’, branchFilter: ‘.*’, defaultValue: ‘origin/master’, description: ‘’, name: ‘BRANCH’, quickFilterEnabled: false, selectedValue: ‘NONE’, sortMode: ‘NONE’, tagFilter: ‘*’, type: ‘PT_BRANCH’, useRepository: ‘git@github.globant.com:ritesh-goyal/jenkins-pipeline.git’)])])// .. Some code block ..}}
5.3 Details of SSH Pipeline plugin and support
- SSH Pipeline steps use a remote (map) settings and then it executes the command on the remote node. Once the command is executed then it returns the output. It internally uses the library of Groovy SSH.
- Below is the sample remote map settings.
def remote = [:]remote.name = <Name of the server i.e., UAT / DEV / Prod>remote.host = <Hostname or IP address of the server>remote.allowAnyHosts = trueremote.user = <User-ID for server login>remote.identityFile = Passwordlessremote.passphrase = <Password for private-key>
- For executing the command on the remote server we will use the sshCommand step from the Pipeline plugin.
sshCommand remote: remote, command: "for i in {1..5}; do echo -n \"Loop \$i \"; date ; sleep 1; done"- So from above command we understand that it uses the remote (map) settings configuration and then executes the defined command.
5.4 Prepare the commands for the execution
- For executing the commands on the remote drupal application server we will prepare the commands using below Jenkins pipeline syntax written in groovy and store them in the variables
- For defining a variable in Jenkins scripted pipeline the syntax used is — def
- Define the site variable for capturing the user input from the choice parameter.
site = params.sites.trim()String[] server = params.servers.trim().split("-");
- Define the branchname variable for fetching the branch at the runtime using Git Parameter plugin list. The plugin fetches the name in “origin/UAT”,”origin/Dev” .. etc format. So for this we need to change “origin/UAT” -> “UAT” and so we are using the replaceAll function.
branchname = params.BRANCH.trim()def branch = params.BRANCH.trim().replaceAll("origin/","")
- Set the default path of the Drupal application on the remote server
def defaultPath = "cd /var/www/multisites/web/"- We will define the theme folder and will set the path of the custom theme of the specific site so that application is built and deployed based on the site.
def themesfolderif ( site == 'site1') {themesfolder = "${defaultPath}themes/custom/site1"} else if(site == 'site2') {themesfolder = "${defaultPath}themes/custom/site2"} else {themesfolder = "${defaultPath}themes/custom/site3"}
- Below command clears the cache on a remote application server. The site name comes from the choice parameter.
def cacheClear = "${defaultPath}; drush -l ${site} cr"- Enable the maintenance mode for the application
def enableMaintenance = "${defaultPath}; drush -l ${site} sset system.maintenance_mode 1"- Pull the source code from git repository on the remote application server
def gitpull = "${defaultPath}; git pull origin ${branch}"- We need to install the composer utility next to the composer.json config file. As a standard the composer.json config is placed in one folder before the default path. For this step please ensure your path.
def composer = "${defaultPath}../; composer install"- Execute the database queries
def updateDB = "${defaultPath}; drush -l ${site} updb -y"- Import the migrated configurations for the application
def importConfig = "${defaultPath}; drush -l ${site} cim -y"- Disable the maintenance mode for the application
def disableMaintenance = "${defaultPath}; drush -l ${site} sset system.maintenance_mode 0"- Install node using the npm utility in the specific site (i.e., site1, site2, site3) directory of the application
def nodeInstall = "${themesfolder}; npm install"- Execute gulp command in the specific site (i.e., site1, site2, site3) directory of the application
def gulp = "${themesfolder}; gulp"5.5 Complete Pipeline structure
Till now we saw various components of the Jenkins pipeline required for the CI/CD pipeline of the Drupal application. Let us view the entire Jenkins scripted pipeline code with different stages having different commands to be executed on the remote Drupal application. For viewing the entire pipeline code please refer to the GitHub URL (Multi Site Drupal Application Jenkins Project) which is publicly available.
For Pipeline execution click on “Build with Parameters” option from the Jenkins project and follow the steps as shown in interface.
- Select the site name for which you want to build the Drupal site.
- Select the branch name which you want to checkout and build the Drupal application. Here all the branches are fetched at runtime.
- Select the server where you want to deploy the Drupal application.
- Click the Build button so that the pipeline is triggered.
The execution output of the pipeline looks like as below:
Note: All the commands (as shown in above picture) are executed on the end server which is selected as a parameter during pipeline execution.
6. Pros
- This Pipeline will help you to execute the multiple remote commands through SSH protocol.
- The execution of the SSH command is secure as we are using a passphrase private key which is pre-configured in Jenkins.
- It will help you fetch and select the Git branch at runtime of the pipeline. It gives better control as compared to the multi-branch Jenkins pipeline project.
- The Drupal application having multiple sites can be handled efficiently through this pipeline.
7. Cons
- The Pipeline needs to be triggered manually due to the parameterised build as the user needs to select the branch-name, server to deploy and site to build the application.
8. Conclusion
- Using this pipeline you can compile, build and deploy your Multi-site Drupal application on the remote server using SSH protocol.
- We can use the plugins as shown in the article to get the git branch at runtime and select the specific server for deployment.
- Also the Drupal commands could be replaced with other commands for similar applications or use-cases.
