Schedule a Python script in Heroku

Quick story..! During this COVID-19 outbreak period, I wanted to collect daily data on COVID-19 cases for analysis. One of the most promising resource which I could find was https://www.worldometers.info/coronavirus. But I could not find any API endpoint for that particular site and I ended up with my own python script (using Beautiful Soup) to scrape data and push into my database. Then the next step was to host that script as a scheduled job. I have tried with https://www.pythonanywhere.com and the problem was they limit access to external web sites for their free tier. So I had to find another way and Here I am writing my steps to schedule a python script in Heroku.

Note: Following is suitable for your own pet projects. And it’s FREE up to some extend. There are advance ways to do the same with many technology stacks.

In here I’m using example python script to push some data into my NoSQL database.

1. Create your python script

I’ll use below python code (scheduler.py) to run. When it runs, it will simply create a date-time stamp and save that data into my free MongoDB database (hosted on mLab).