Covid19 Data Extractor Using Flask

Retrieves Covid-19 Updates from api.covid19india.org API and creates excel sheet for state and district wise timeline data.

Introduction

This is a personal project to help people to get data automatically. You need to select the state from the dropdown and click on submit. You will get an excel sheet for downloading and that will contain the data of the state selected and all the districts, if any, in that state in different sheets in the file.

The data is extracted from https://www.covid19india.org. This website has its own API, https://api.covid19india.org. The API contains state-wise, district-wise and whole country’s COVID19 data with dates. The excel sheet is generated using this data.

Flask App

First of all, we will import the necessary libraries needed for the app on the backend side. As we are using Flask as a background app, we will import flask. Also, we are going to create an excel file with the data from the API. So we will import xlsxwriter. Also, the API uses CSV files. So to read it, we need pandas library. So we will import pandas.

Next thing we are going to do is define the global variables — states & unknowns. “states” will be a list of all the states and union territories whose data exists in the API and “unknowns” will be a list of all the states and union territories which doesn't have districts.

Now we will make utility functions for extracting state data and district data. We will read the CSV files from the API and select the necessary columns from it. Also if the data is for districts we need the names of the districts in ascending order and so we will traverse through the names of the districts and add the unique names in a list. The utility function for districts will return district data and sorted list of the names of the districts. The utility function for states will return the data of the whole state.

We are all ready to create our Flask App.

Lets begin

We start by creating the app.

Now we will add routes to the app. there will be 4 routes.

• Home — GET and POST
• About — GET
• Contact — GET
• Page Not Found — GET and POST

First, we will discuss the “GET” request to the home page. Our home page will be at the route — “/”. The “GET” request will render “home.html” with the global variable “states”. We will discuss the Frontend later in the article.

Now we will discuss the “POST” request to the home page. The “POST” request will send the name of the state selected by the user and we will extract the data accordingly using utility functions. Here we will use the XlsxWriter library to create the excel file for the user. The excel file will have one sheet for state data and one sheet each for every district data. The format of the state sheet will be:

• Column 1 — date
• Column 2 — State Name
• Column 3 — Daily Confirmed Cases
• Column 4 — Daily Recovered Cases
• Column 5 — Daily Deceased Cases
• Column 6 — Daily Tested Cases
• Column 7 — Total Confirmed Cases
• Column 8 — Total Recovered Cases
• Column 9 — Total Deceased Cases
• Column 10 — Total Tested Cases

The format of the district sheet will be:

• Column 1 — date
• Column 2 — State Name
• Column 3 — District Name
• Column 4 — Daily Confirmed Cases
• Column 5 — Daily Recovered Cases
• Column 6 — Daily Deceased Cases
• Column 7 — Daily Tested Cases
• Column 8 — Total Confirmed Cases
• Column 9 — Total Recovered Cases
• Column 10 — Total Deceased Cases
• Column 11 — Total Tested Cases

Now we have 3 more routes to add which are very small. The about page has only “GET” request and the route to about page is “/about” and it renders “about.html”. The contact page has only “GET” request and the route to the contact page is “/contact” and it renders “contact.html”. Now the “Page Not Found” error is handled the errorhandler and it renders “404.html”

Now we will run the app is the “__name__” is “__main__”. When we run the app and not specify the port number it will select the default port number as 5000.

So after you run the server, go to URL http://127.0.0.1:5000 or http://localhost:5000.

We are now ready to make our Frontend files.

For Frontend we will create a folder named “templates”. So our project folder will look something like this.

Project Folder

We will be using Jinja2 template designer used by Django and Flask for dynamic content embedding.

Now first we will create “template.html”. This file will have header and footer that is common in all the web pages.

You can notice that there is “block content” block between which our other HTML pages’ content will be added. Now we will create the rest of the pages. We only need to extend the “template.html” file. The home page will have 1 dynamic content i.e “states”. This dynamic content is used in the dropdown menu for state selection.

Similarly, we will create “contact.html”, “about.html” and “404.html”.

Our project is now ready to run on the local system.

To run on the local system:

$ python3 extract_covid_data.py

Output

You can see the output of the above project at http://extract-covid-data-who.herokuapp.com

Github Repository

To refer the code again, follow https://github.com/kandarpkakkad/Covid19-Data-Extractor-WHO. For more projects follow https://github.com/kandarpkakkad/