Creating and Deploying a Python Flask Microservice on Amazon Fargate — Part I

This post is going to be the first chapter of a series of posts explaining how to create a Python Microservice using the Flask Framework, building an image on Docker, sending this image to Amazon ECR (Elastic Container Registry), setting up CircleCI to build and upload the image, and deploying the service on Amazon Fargate, with Nginx as a reverse proxy and uWSGI.

On this first part, I’ll explain what and how we are going to build this service, what we are going to need in order to do that and finally the initial setup of the project’s structure and routes.

ToDo List Microservice

The name of our microservice is going to be ToDo List. It consists of a very simple REST API that allows us to Create, Delete, Update and Remove a ToDo item. In order to accomplish this, we are going to need a few things:

1. An AWS Account — Note that running this project on AWS will occur some costs, be careful and make sure you delete everything after you are done;
2. Python 3.8 (preferably on a virtualenv);
3. A MongoAtlas free account (https://www.mongodb.com/cloud/atlas);
4. Everything was developed on Linux, it might (and should) work on other OS, but the commands and everything else will be based on that.

Project Structure

All Python projects that I create using Flask follow the same base structure, as explained below. If you want to download the entire project for this post, click here:

├── app.py
├── config
│   ├── default.py
│   ├── development.py
│   ├── __init__.py
│   ├── production.py
├── requirements
│   ├── requirements.txt
│   └── testing.txt
├── tests
└── todo_list
    ├── apis
    ├── schemas
    ├── services
    └── utils

• The requirements folder stores all dependencies for our project. I usually separate them on requirements.txt and testing.txt . The second one stores dependencies used only for testing purposes. It helps us reducing the size of our image when building and running it on production (or outside localhost).
• The tests folder, as the name says, stores tests related files;
• The todo_list folder is the main folder of our application (and is the name of our application). Everything related to the microservice (APIs, Services, utils and schemas) are stored there;
• The apis folder is responsible for our APIs definitions (routes, namespaces, validations, etc);
• The services folder is where our business logic is. Usually the APIs call services within this folder. If you come from Java or other language like that, they are like “controllers”;
• The schemas folder stores all marshmallow schemas (I’ll explain what marshmallow is later);
• The utils folder stores utilitary codes, like datetime on a specific timezone, some kind of specific calculation used in multiple places and so on;
• The config folder stores all configuration related values that will be loaded in our flask application (You can use it to store secrets, or to gather secrets from env variables, for example);
• The app.py is the entrypoint of our application. All initialization related code will be there.

Now that you understand the folder structure, let’s start building the project.

Installing the Dependencies

We are going to use few specific dependencies in order to build this project and make it scalable and more secure:

1. Flask — I won’t explain why do we need this dependency because I’m assuming that you know what is Flask, if you don’t, you can access the documentation;
2. uWSGI — If for some reason you are not able to install this dependency, click on the link above and follow the steps on the documentation;
3. Flask-RestX — It is a fork from the Flask-RestPlus project, that is not being maintained anymore. It is really helpful to manage APIs, it allows us to create blueprints, add decorators to our blueprint (like token validation), API documentation and few other things.
4. Flask-Cors — This dependency helps us dealing with CORS. If you want to understand what is CORS, access this website.
5. Marshmallow — Marshmallow is a package used for Serialization/Deserialization of JSON objects in Python. It is really helpful, given that you can create custom types, define exactly the body expected by your API, and few other things.
6. PyMongo — Pymongo is a python client that allows us to interact with the MongoDB API. We are going to use this on the next chapter;

We are going to install all those dependencies in our system, you can add them in our requirements/requirements.txt file, as below:

flask==1.1.1
uwsgi==2.0.18
flask-cors==3.0.8
flask-restplus==0.13.0
marshmallow==3.5.1
pymongo[srv]==3.10.1

After adding it to the requirements, just run pip install -r requirements/requirements.txt from your virtualenv and you are good to go.

Setting up the Microservice

Let’s start by creating our app.py file and adding some basic content to that file. Note that this is not the final version of the app.py file. We will be creating this way so I can explain better what each part of the code is doing.

Example of app.py file

The first thing we are doing here on line 8 is creating a make_app function. I prefer doing it this way so when we start building tests and uWSGI config, it is easier to reuse the make_app function.

• First we load the environment variable called APP_ENV . This variable will store the environment that our service is running, and defaults to development .
• Then we initialize the Flask application, and load the configs from the configs folder for the desired environment. Note that we first load the default.py , so we can define configurations that are the same no matter which environment we are using, after that we load the environment specific configurations.
• After loading Flask, we enable CORS in our application. On this case, we allow requests from any origin, but we could restrict this on production, if we want to.
• Then we start building the API using flask-restx . This is a small example of how we build an API, but there are few things happening: First we define an API — every route has to be inside a namespace, and the namespace has to be inside an API. We can have multiples APIs on the same project, separated by each other using blueprints (What is a blueprint?). We will talk about this later;
• The todos on the namespace is like a prefix for our URL, so, in this case, we have the route / inside the namespace with prefix todos , consequently, the final route is going to be GET /todos/ , and it will return the message Hello World .

After creating the app.py , you can run your python application and check if it is working. In order to run and test it, you can do:

1. python app.py from your virtualenv;
2. It will be available at http://localhost:5000/todos/ . Since we’ve only defined the GET operation, you can only access the endpoint this way.

Creating the TODOs API

On the previous section, I gave you an overview of how you create an API, namespace and route. However, when dealing with huge projects, we won’t be creating everything inside the app.py file. As I said, this file is only the entrypoint for our application, we have to keep things separated from each other, and now we will start building our TODOs APIs on the correct location, and then load it on the app.py.

1. Start by creating the apis package within the todo_list folder. Remember that, in Python, every package has to have the __init__.py file inside it. After creating this folder, create a new file called api.py.
2. We are going to create a new Flask Blueprint (If you don’t know what is a blueprint, look at this documentation) in order to create this API in a separate file. So, inside our api.py file, add the following line:

from flask import Blueprint

todo_list_bp = Blueprint("todo_list_api", __name__)

3. After creating the Blueprint, we have to create the API itself, we can do that by adding line below after todo_list_bp:

api = Api(todo_list_bp)

4. With the api and blueprints created, we have to load them on the app.py , in order to do that, let’s cleanup ou app.py and add only the following:

Final version of app.py file

Note that we added the line application.register_blueprint(todo_list_bp) . By doing that, we add the API created before to the flask application, and we will have access to all its namespaces and routes later. As I said, you can add as many blueprints as you want on a Flask application, it is used to make flask modular.

We have the API created and working, however, there is no namespace and route associated to it yet, on the next section, we are going to create both things and add them to the API itself.

Creating TODOs Namespace and Routes

On this section, we are going to create all routes and namespaces that our service will need. We won’t create validations, insert TODOs on databases or anything like that, we will only create the routes and return some examples of responses. The goal on the part I of this serie, is to create the base setup for our microservice.

1. In order to create our first namespace, we have to create a new file called todos.py within the apis folder.
2. After creating the file, lets setup the namespace the same way we did before by adding the following to the todos.py file:

On the code above, we’ve defined a new namespace todos_ns that will be on the prefix /todos . Then we created two different base routes: /todos/ and /todos/<id>/ . The first one allows POST and GET methods, and the last allows PUT and DELETE methods, but we need to send a variable on the URL, that will be TODO’s id.

3. After defining the namespace, we have to add it to the API we’ve created on the apis.py file. In order to do that, we are going to import the namespace, and add it to the newly created API:

from flask import Blueprint
from flask_restx import Api

from todo_list.apis.todos import todos_ns

todo_list_bp = Blueprint("todo_list_api", __name__)

api = Api(todo_list_bp)

api.add_namespace(todos_ns)

This is how our api.py file will be after adding the todos_ns to it. If you want to check if everything is working as supposed, you can run the project python app.py and hit the endpoints that will be at http://localhost:5000/todos/ (GET and POST) and http://localhost:5000/todos/<id>/ (PUT and DELETE) .

Important: Note that ALL APIs will have the / (slash)on the end.

Conclusion

That’s it for today. We’ve created a service with a TODO api, that allows GET, POST, PUT, DELETE methods on a specific route. On the next posts, things will start getting more interesting, since we will connect to the database using PyMongo , we are going to create schemas to validate the payload of a new TODO, building the Dockerfile and few other things.

The next chapter is now available — Creating and Deploying a Python Flask Microservice on Amazon Fargate — Part II