Continuous Integration of latex files with Jenkins Pipeline

5 min readMar 11, 2019

LaTeX is a typesetting system which lets you produce .pdf from source code. This idea of treating ‘documents as source-code’ has some powerful benefits. You can store LaTeX markup in a VCS repository to manage multiple versions of your document. This post is about how to automate your LaTeX workflow, which is compiling the source code and generating pdf file, using Jenkins Pipeline.

So why do we need to create a workflow rather than just compiling the LaTeX document and copy the .pdf to your Drive/Dropbox/Email? If you have been using LaTeX, you would know the pain of maintaining the latest version of it. Benefits of creating a Jenkins server to automate the LaTeX workflow:

No need to install a (sizable) LaTeX environment on your main development machine.
Automatically produces a PDF from the latest version checked into VCS — no more forgetting to email your boss the most up-to-date copy (oops).

Let’s get the party started.

I am using Overleaf, an online LaTeX editor with compiler, which is linked with my GitHub account to let me commit changes directly from Overleaf. This guide might help you set it up.

You need to know a little bit about Jenkins pipeline workflow. There is an excellent tutorial on Jenkins docs which helps you understand how Jenkins pipeline works inside a docker.

Set up your Jenkins server:

Prerequisite:

Minimum hardware requirements:

256 MB of RAM
1 GB of drive space (10GB recommended)

Software requirements:

Download and install Docker and Jenkins/Blueocean container — see this.

Run Jenkins in Docker:

sudo docker run \
  --rm \
  -u root \
  -p 8080:8080 \
  -v jenkins-data:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v "$HOME":/home \
  --name "jenkins-latex" \
  jenkinsci/blueocean

The above command:
1) Maps the /var/jenkins_home directory in the container to the Docker volume with the name jenkins-data. If this volume does not exist, then this docker run command will automatically create the volume for you.

2) Maps the $HOME directory on the host (i.e. your local) machine (usually the /Users/<your-username> directory) to the /home directory in the container.

Writing your pipeline code:

pipeline {
   agent none
   stages {
      stage('Build') {
         agent {
            docker {
               image 'blang/latex:ubuntu'
            }
         }
         steps {
             sh 'xelatex sample.tex'
         }
      }
   }
}

Save your pipeline code as ‘Jenkinsfile’ in the root directory of your LaTeX repository and commit the changes. The pipeline code should be self explanatory if you have gone through the Jenkins tutorial.

Deploying your SCM in Jenkins/Blueocean:

Open your Jenkins instance at http://localhost:8080.
Select Open Blue Ocean
Select New Pipeline
From SCM field, choose GitHub or whichever SCM you prefer, currently GitLab is not supported. Check back in August by that time Jenkins will have added GitLab support.
In the Repository URL field, add the link to your SCM (GitHub) repository. You may also add your SCM credentials and select Create Credentials to access your repositories and choose from the list.
Select Create Pipeline

Building your Pipeline code:

Now Blue Ocean will automatically build your project for you based on your Jenkinsfile.

Select master (if that is the branch containing Jenkinsfile) to view the process.

The Build process might take a little time as the docker container of XeLaTeX is about ~3 GB.

After the Build step you should find your sample.pdf placed into the workspace directory (within the/var/jenkins_home/workspace/overleaf-resume_master directory in the Jenkins container).

Accessing docker terminal:

sudo docker exec -it jenkins-latex bash

Basically, lets you access the Jenkins/Blue Ocean container (through a separate terminal/command prompt window) with a docker exec command.

After getting the Docker bash prompt:

bash-4.4# cd /var/jenkins_home/workspace/overleaf-resume_master
bash-4.4# ls
Fontin      LICENSE Ubuntu   sample.aux  sample.pdf
Jenkinsfile  README.md sample.tex  sample.log  
bash-4.4# cp sample.pdf /home

Now open a terminal on your local machine and if you are using Ubuntu:

xdg-open ~/sample.pdf

to see your generated pdf.

Installing publish-over-dropbox:

Open your Jenkins instance at http://localhost:8080.
Select Manage Jenkins | Manage Plugins | Available
Search for Publish Over Dropbox
Select Install without restart
Wait for it to install

To detect installed plugins:

Go to http://localhost:8080/about and find Publish Over Dropbox from the list of License and dependency information for plugins to ensure the plugin is correctly installed.

Generate a Dropbox API token:

Select Credentials | Global Credentials | Add Credentials
Select Dropbox API Token via the dropdown.
Select generate, give an ID like “dropbox-token” and a description like “Linking my dropbox api token to allow the plugin to access read/write files”
Select OK

Configure the Dropbox location:

You need to specify the location you wish to send the resulting sample.pdf to.

Select Manage Jenkins | Configure System | Dropbox
Set Name | Select Dropbox Account
Set path to Remote Directory, if path is correct the Test Configuration should return Success
Select Save

Generating Groovy script for our plugin:

You need a Groovy script to execute the plugin, you can refer to the doc here or utilise the snippet generator to get the Groovy script for steps of pipeline.

Get Snippet Generator:

Return to classic Jenkins at http://localhost:8080/
Select your repository from the jobs
Select Pipeline Syntax
From the Sample Step dropdown, select dropbox: Publish to Dropbox folder and fill up the fields | Select Generate Pipeline Script

Snippet Generator generates the script:

dropbox cleanRemote: true, configName: 'My Resume', remoteDirectory: '/', removePrefix: '', sourceFiles: 'sample.pdf'

You may now copy the above Script and delete this Job (or Item).

Extending your pipeline code to Deploy:

Modify your Jenkinsfile to add the deploy stage:

pipeline {
   agent {
      docker {
         image 'blang/latex:ubuntu'
      }
   }
   stages {
      stage('Build') {
         steps {
            sh 'xelatex sample.tex'
         }
      }
   stage('Deliver') {
      steps {
         dropbox cleanRemote: true, configName: 'My Resume', remoteDirectory: '/', removePrefix: '', sourceFiles: 'sample.pdf'
      }
    }
  }
}

Commit the changes to your SCM.

Rebuild your project in Jenkins Classic or Blueocean whichever makes you feel comfortable. The sample.pdfis generated and published to your Dropbox folder.

You can host this Jenkins server on Heroku or Digital Ocean for the workflow to automatically build the repository and publish the .pdfto your Dropbox everytime you commit changes to your repository.

References:

Leave a few claps if this helps 😃

Adiós.