How to Deploy a Production Grade Flask application to an AWS EC2 Instance using GitHub Actions: Part Two

Automatically deploy application using GitHub Actions

This is the second article in a series that takes the reader through the process of deploying a production ready flask API to an AWS EC2 instance. In the previous article:

• Created a Flask application using a template, then tested it and deployed it locally.
• We then committed the application code to a features branch then merged the code into the development branch.
• We then created an EC2 instance on AWS that runs Ubuntu 20.04 server and allows ssh traffic on port 22 and HTTP traffic on port 5000.
• We then updated the created server, installed python and venv then created a separate non-root user account for our application.
• We finally pulled the application code from our GitHub account, then launched the application and tested that it worked.

In this article we want to accomplish two major tasks:

• Automate the application startup; that is when the server is started, the application should automatically also start.
• Automate the application deployment, when code is merged into the development branch.

Step 1: Test out the application Locally

First let us ensure that the application code works locally. You can follow the previous tutorial to set it up or simply clone the project on GitHub. If you followed the previous tutorial:

source venv/bin/activate
make test # run the unit tests
make run # start the application
curl http://localhost:5000/ 

You should get back : {“hello”:”from template api”}

If you cloned the repository, navigate into the cloned project repo:

python3 -m venv venv # create a python3 virtual environment
source venv/bin/activate
make all # install project requirements and start the application
curl http://localhost:5000/

If the application works locally, then you are good to go.

Step 2: Test out the application on AWS

This step assumes that you followed the previous article and created an EC2 instance, then deployed the application. Let us make a small change to the application, then deploy it; but first, start up the EC2 instance that you created previously. Sign into your AWS account then navigate to the instances section. Select the checkbox next to the instance that we created previously, then in the Instance state drop-down select Start instance:

Now let us make a small change to our application, then commit the code to GitHub. Open up the <project-name>/api/blue-prints/default/views.py file and update it with:

We simply added ‘auto-deployed’. Then commit the code to GitHub. Make sure that you are in the features branch or create it; our workflow is such that any changes are made in a feature branch then merged into the development branch. On the command-line:

git add .
git commit -m “feat: added ‘aut-deployed’ to the default reply.”

This might fail due to the pre-commit hooks installed, just run the commands again. If the failure persists, try and correct the errors.

make bump-tag # creates a new tag for us
git push 
git push origin v0.1.0 # the tag created by make bump-tag

Then create a pull request and merge the feature branch into the development branch. Some of the checks will definitely fail. The template that we used comes with configurations for deploying the application to Heroku, and these are currently not set. Just go ahead and merge the two branches.

Log into the EC2 instance; navigate to the directory where where you downloaded the private key created in the previous tutorial and issue this command:

ssh -i “ec2.pem” lyle@<your-instance-hostname>

Refer to the previous article, where I explained how to log into the created EC2 instance. Then navigate to the project directory:

cd flask-ec2-deployment # use your own project name
source venv/bin/activate
git pull # pull the recent changes
make run # start the application
curl <your-instance-hostname>:5000

You should get: {“hello”:”from template api auto-deployed!”}

Step 3: Starting the application on server startup

We will use an application called gunicorn to launch the flask api from now on. Stop the api on the EC2 instance. You may have noticed that the application indicates that this is a production environment and not a development environment; let us change that;

touch  /home/lyle/.env # replace lyle with the user you created yesterday
nano /home/lyle/.env

Then type in:

FLASK_ENV=development
PORT=5000

Save and close the file. The open up /home/lyle/.profile :

nano /home/lyle/.profile
set -o allexport; source /home/lyle/.env; set +o allexport #and add the at the bottom of the file

source /home/lyle/.profile # source the bash profile
make run # launch the application

You should see:

Stop the application. The issue the following command:

gunicorn -w 4 -b 0.0.0.0:5000 main:app # launch application using gunicorn

This should start gunicorn, which is a production grade serve. This is how we will launch the application from now on. Next, let us create a system service that will automatically start out application for us:

sudo touch /etc/systemd/system/gunicorn.service
sudo nano /etc/systemd/system/gunicorn.service

Then paste in the following code:

Replace the username in this lyle, with the name of the user that you created in the previous tutorial and the project name in this case flask-ec2-deployment with your own project name. The file is also found in the project repo for reference.

We now need to enable the service, so that it automatically starts at boot-up then also start it; make sure that gunicorn that you started has been terminated:

sudo systemctl start gunicorn # start the service
sudo systemctl enable gunicorn # make sure it starts automatically
sudo systemctl status gunicorn # check on the service

You will get:

Make sure that it is enabled and running. Now try out the curl command again:

curl <your-instance-hostname>:5000

You should get: {“hello”:”from template api auto-deployed!”}.

Now reboot the EC2 instance and try out the two commands again:

sudo reboot

Wait for two minutes before logging in to the EC2 instance. Once logged in(shows that the server is up, try the curl command again). You should get the same response, even though you did not manually start the application.

Step 4: Automatically deploying changes to our development application.

The final step involves automating deployment of changes to our application instance. So far, when we make changes to the application in a feature branch, we commit the code to GitHub. We then merge the code to the development branch, then log into the EC2 server, then pull the latest code from the development branch. We then restart the application in order to see the changes take effect. We now are going to automate all these.

We will use GitHub actions to accomplish this. We wil create two workflows, one for th e feature branches and the other fot the development branch. Let us start with the feature branches workflow; open up the .github/workflows/feature-development-workflow.yml:

• This workflow is triggered any time code is pushed to any branch apart from the development, staging, release and production branches.
• It has two main jobs, a build and a test job.
• In the Build job, three versions of python are set i.e 3.7, 3.8 and 3.9. Each of these python versions are then used in the subsequent steps.
• With each python version, the project requirements are installed, then the unit tests are run. In this workflow, we have not included code quality checks since we ran those locally. However, before the unit tests are run, you should include a step to lint the code i.e run flake8, pylint,pydoc even black as well as isort. Try it out.
• If the Build job is successful, we test the application in the Test-Local job. This is similar to what we did at the beginning, where we activated the virtual environment, installed the project dependencies, ran the application then ran curl to test the application output.
• In this workflow, when starting the application, line 60 and 64, we supplied an input called the FLASK_ENV, just as we did in our EC2 instance. This value is set in our GitHub account and we will do that shortly.

The development workflow is responsible for deploying our application. Open the .github/workflows/development-workflow.yml file and update it with the following:

• This workflow runs when we push or merge code into the development branch. The Build and Test-Local jobs are similar to those in the previous workflow.
• The Deploy-Dev job is responsible for the deployment. It only runs if the Test_local job was successful. We supply a couple of values to the Deploy to EC2 step such as the our AWS private key, the AWS host name, the user name, the user password, the application directory as well as the service name. These are used to ssh into the EC2 server, navigate into the project directory, make a pull request and restart the service.
• If the DeployDev was successful, we then test the deployed application again, using the curl command.
• Both the feature development and the development workflows require a Development environment to be created on GitHub.

Let us then create the Development environment as well as the secrets. Head on over to your GitHub account. Select the Settings tab, then in the resulting page scroll down to the Environments section:

In the resulting page, select the Development link. This environment was automatically created for us by the existing workflow that we just replaced:

Scroll down to to the Environment secrets section and click on the Add Secret button:

For the Name use FLASK_ENV and Value development:

• Create another one with Name AWS_PRIVATE_KEY and for the Value, navigate to the folder where your AWS private key was downloaded i.e the ec2.pem file. At the command-line issue the command cat ec2.pem and copy the value and paste it in the Value field. Make sure to copy everything, starting at — — -BEGIN RSA PRIVATE KEY — — - and ending at — — -END RSA PRIVATE KEY — — -
• Create another with Name HOST_NAME and Value, the instance host name that you used with the curl command.
• Create another with Name USER_NAME and Value as the name of the user that we created previously.
• Create another with the Name USER_PASSWORD and the Value as the password of the user created previously.
• Create another with Name as APP_DIR with Value as the complete path to the application directory i.e /home/user/lyle/flask-ec2-deployment in your case this would be /home/user-name/project-name.
• Create another with Name as SERVICE and Value as the name of the service that we just created above in this case gunicorn.
• Create another with Name as PORT and Value as 5000, or the port that flask listens on

With our configs done, let us push the code to GitHub and also create a new tag:

git add .
git commit -m “feat: auto-deploying to AWS.”
make bump-tag
git push
git push origin v0.2.0 # the tag generated by make bum-tag

Under the Actions tab you should see the Feature Development Workflow run output:

Next, make a pull request to merge the feature branch into the development branch, and let all the checks run. Here is a sample screen shot of the merge checks running; let all of them run (14 checks):

If all the checks pass, in the actions tab you should see:

Now finalize the merge.

Step 5: Test Automatic changes to application.

In this final part, let us test this whole workflow. Open up the <project-name>/api/blue-prints/default/views.py and make a small change:

We just added ‘with GitHub actions’. Now commit your code and push them to GitHub. Let the feature development workflow run, then create a pull request and once all the checks are done, merge into the development branch. Then use curl to check the API output:

curl <your-instance-hostname>:5000

You should get:

{
 “hello”: “from template api auto-deployed with GitHub actions!”
}

And that’s it. You have successfully set up a workflow that automatically deploys an application to AWS.

Conclusion:

So in this article, we :

• Tested the application that we developed previously to ensure that it still worked locally.
• Tested that the application still worked on AWS by making a small change to the application.
• Created a service that automatically starts our application when the EC2 server starts up.
• Created a workflow that automatically pushes changes to our application to the development server and restarts the application.

In the next tutorial, we will use nginx to route HTTPS traffic to our application as well as add a domain to our application. So, give me a follow so that you don’t miss that.

That’s it for this article. I hope you enjoyed it and learnt something. Give it a clap or share it out and do not hesitate to reach out to me in-case of an issue. The code for this application is here flask-ec2-deployment. I am Lyle, a junior software engineer with a passion for developing, testing and deploying scalable services. You can find me on twitter, linkedin, github and here’s my portfolio. See you next time.