Django Deployment with Jenkins Pipelines and Chef — Part 2

Creating Django Artifacts and Chef Deployment

Tutorial Index

Part 1 — Plan, Prepare and Test
Part 2 — Creating Django Artifacts and Chef Deployment (Reading Now)

In Part 1 of our tutorial, we drafted out our deployment strategy, prepared our django application and ran the test suite. In this next chapter, we’re going to dive into creation of primitive artifacts and their deployment to target servers.

What’s in The Box?!

Typically when you ask django shops how they deploy their code, the answer sounds something like this:

“Well, we have a git/svn repo with our application. We go to all of our application servers, retrieve the latest version, activate virtualenv, install all requirements, and then restart the web server.”

This is a crude approach to deployment but it’s the one which gets chosen because it’s simple and it works. There are also not a lot of alternatives out there. At least that is what I found when I researched “alternative ways” to deploy a django application into a RHEL environment. Rather than continuing this trend I am going to propose a different route to deploy our application. We’re going to create a basic build artifact in the form of a tarball gzip archive.

One way for us to do this within the Jenkinsfile is to create a list of files and folders and then pass them into a shell tar command. Please note that you have to list out all of the files which are required to run your application along with bin, include and lib folders. The code snippet below should accomplish the creation of the artifact. I included inline comments to help you understand what’s going on and why everything is needed.

Side-Note: If your target platform is Debian then you’re in much better shape thanks to the efforts of Spotify developers who created dh-virtualenv. I have not tried this tool myself because I’m targeting RHEL but if you have, please feel free to drop a reply with your experience.

Once the build artifact is complete, you will need to make it available for deployment. One way to accomplish this is to make the gzip file available right from your jenkins server. However, this introduces some security complexity. Meaning that your jenkins server has to be made available to your target servers so that they can retrieve the file. This complexity becomes even more of an issue if the target servers can only communicate with your jenkins server via its public ip. (You should not expose your jenkins instance via a publically accessible ips)

Another approach that we can take is to transfer the gzip file (applicationZip) that was created earlier in our script to a dedicated artifact repository. For our sample code, we’re going to copy the artifact over to a dedicated private yum repo by using the scp command. The other thing that you will notice is that we’re using the jenkins user to perform the scp operation. In order for this to work, you will need to make sure that the jenkins user has ssh access to the server or use an alternative transfer protocol. You can also upload it to any other file server to achieve similar results.

Deploying Your Artifact

Now that your artifact has been transferred to your dedicated repository, it’s time to make something out of it. I am going to return to write a full chapter on chef recipes but for now, please make believe that we have a chef recipe which is responsible for downloading your artifact and unzipping it to a specific directory on the file system. If you’re running apache, then that place will most likely be /var/www/mysite.com/.

If you’re familiar with chef then you already know that on-demand application deployment can be a pain due to how a chef server deploys code. Typically you have a chef client which does a chef-run every 30 minutes which runs through your recipe to ensure that the server is in good shape. This is because Chef’s primary purpose is to provision your servers.

Since we would like to be in control of which artifacts get deployed, we’re going to bypass this 30 minute rule. In order to achieve this, we’re going to check a “deploy” attribute in our recipe before we deploy the code. This attribute will be set to False by default to prevent the deployment of code every half an hour.

After the recipe is aware of our new “deploy” attribute we are going to create a shell script which will set this attribute to True and trigger a chef-client run. The script will get triggered remotely from our Jenkins server on every successful run of our build and only after tests from Part 1 have all passed. The code snippet below will take care of this for us.

Please note that this will use a regular ssh command to execute the script on the remote server and pipe the output back to our original jenkins shell process. Additionally, we are telling the shell script to ignore warnings around existing entries in known_hosts that you might get if the ip of your target server changes for any reason. You can take out the StrictHostKeyChecking=no portion of this shell command if you are not expecting to get these types of warnings. Most importantly, we are also passing in the file name into our deploy.sh script to tell it which artifact it needs to download during the chef-client run.

Finally, please note that I am wrapping the script output into ansiColor which is provided via the Ansi Color Plugin. Since Chef commands get output with a color code, using the plugin will allow you to read the chef commands in a much more succinct manner.

Pro-Tip: If you’re trying to run this on multiple servers then you might be better off triggering a knife ssh command instead.

This concludes Part 2 of the tutorial. In the next chapter we’ll dive into configuring Apache, WSGI and Other Django virtualenv intricacies that you will have to deal with on the path to true continuous deployment. (Part 3 coming soon)

Image Credits
Icons made by Roundicons from www.flaticon.com
Django, Chef and Jenkins icons are properties of their respective owners