Setting up CI/CD on GitLab for deploying python flask application on Heroku
Recently I came across a challenge to write a python flask application and deploy it on heroku. I had chosen gitlab to host my code repository and was pushing most of my code to it. When I logged into heroku, to my surprise I found that heroku supports deploying an app from github, Dropbox along with the usual heroku git. It had been quite sometime since I used heroku. I was just wondering if I can use my existing gitlab repository instead of using any of the sources mentioned above. As I could not find any information or documentation around deploying application using gitlab repository on heroku, I browsed a bit on gitlab. Turns out, gitlab has it and I was now intrigued. Before I list the steps here on how I did that, I would like to explain why use gitlab or github while you can simply get things done easily with heroku git.
- Easier code maintenance — With code repositories hosting services like github, gitlab, maintenance of code is easy. These are proven and very well used services.
- Customising pipelines — With gitlab, we can write our own yaml file and include the libraries required for running our application.
- For better understanding of Continuous Integration and Continuous Development (CI/CD)- For beginners, this setup helps you understand the workflow of coding -> testing -> version controlling -> code maintenance -> deploying application.
Now here are the steps you are eagerly waiting for till now (or jumped here ;) )
The steps here assume you already have a good understanding of python, flask, version control, gitlab and heroku. Also, this write up is helpful for someone who is looking for a start. I have kept it as simple as possible to get things up and running.
Uploading project to gitlab
- Create a python virtual environment for us to use. Get into the virtual environment.
- Create a sample python flask application on your machine.
- Verify if everything is running fine.
- Now pip freeze to catch all the requirements for running your application. Run this command from inside the main folder of your application. (Example : /var/www/my_app = you will run the below command from within my_app)
pip freeze > requirements.txt
- Now Create a Procfile which is used by heroku to run your application. Procfile usually consists of the web server used to run the application. In our case let us use the default gunicorn WSGI. The content of your Procfile will be
web: gunicorn <name of the app.py file>:<app-name>
app-name is usually “app”. (run this again from within my_app folder)
- Now login (or signup) to gitlab and create a project. The moment you do this, you will get a standard set of instructions on how to “link” your code on your development machine to gitlab project. Just follow the commands, and after that you can do a git push or git pull to/from this project. This is a bit of an elaborate step, your last statement should look something like
git push -u origin master. Once done, on refreshing the Project page on gitlab, you should see all your code [Image below]. (Since version controlling and code maintenance is not the actual intention of this blog, I am skipping them for user read.)
Linking gitlab and heroku
- Login to heroku web portal and create an application. Give it a nice name and runtime selection.
- Now back to my_app folder on your development machine. Create a file called “.gitlab-ci.yaml” [note the ‘.’ in the beginning].
- This yaml file will have the following structure.
— apt-get update -qy
— apt-get install -y python-dev python-pip
— pip install -r requirements.txt
— export MONGOHQ_URL=$MONGO_URL
— apt-get update -qy
— apt-get install -y ruby-dev
— gem install dpl
— dpl — provider=heroku — app=task-mgmt-app — api-key=$HEROKU_SECRET_KEY
- Change my_app_file_name to the file name of your flask application. You need to set HEROKU_SECRET_KEY variable in the project variables. You will get this key in heroku dashboard. To set it in your gitlab project, go to Settings > CI/CD Pipelines and search for Secret Variables. While using these variables in the yaml, we need to prepend the variable with ‘$’ sign. It is a good practise not to share secret keys with anyone and also restrict access to them in the project.
- You are almost there! just do a
git add .gitlab-ci.ymland
git commit -m <msg>and
git push -u origin master. You will see the file in the gitlab repository now.
- On gitlab “My Dashboard” click on Pipelines. Click on Jobs to see your job which has started running.
- If incase you are using any database in your app, you might want to link it to the app by placing the details of it in .gitlab-ci.yaml file. Please check here for an example. I have used MongDB in my application. Heroku provides adding a set of free libraries/apps to your application. There is an mLab link for adding MongoDB.
Yaay! you have now successfully integrated your gitlab with heroku with CI/CD configuration. Make all code changes you want in your repository, push it to gitlab project, and see a job starting every time there is a code push. For the current setup, I used gitlab public runners available here. You can set up a custom gitlab runner and set appropriate configuration.