Scraping Covid-19 Data from Websites
I know you are bored while sitting home seeing those boring charts. You are not able to create your data because of no data present. Now the Government started to make an official website with counts of people who are infected.
I got this idea thanks to Amazon.in you guys put that Ministry of Health
and Family Welfare Website in your site. Now we have an official site for scraping the data.
Let’s start this process. I am using Python and Pandas for this process.
We got data now in a tabular format. If you check the source for this site.
We need to scrape data for the filter table and class name. You have your data in 2 minutes. There is a problem with beautifulsoup process. I tried that I got data but there is too much processing that needs to be done.
BeautifulSoupScript
I got output but there are a lot of things that do not make sense. If anybody able to figure out let me know.
So I will change my program to pandas seems to be the only effective way to scrap data at this moment. In the second program we are going to use the PD.readhtml method.
With this method, I can scrap data in Dataframe. You need to convert some columns to numeric as data scrap in string in the data frame. Let’s aim for a little bigger.
Let’s see can we used this for Worldometer site. Same code worked on Worldometer site
In this method you get panda datagram. Some preprocessing needs to be done before using this in graphs like change data types.
Here is video link for this program https://youtu.be/8tQBFX8H8bU
if you want to know about how to fetch stock data from YFinance for automation and data science stuff.Link for the same-https://youtu.be/oGC-AfqW1SA.
if you feel stuck please hit me back to saurabhkdm721@gmail.com.
