Dataflow kit blog
Published in

Dataflow kit blog

😷 How to create Coronavirus (COVID-19) API and Web widgets?

Like all us, we’ve been observing the spread of Coronavirus (COVID-19) with great concern. We are grateful to all the health workers and public safety officials. Our thoughts are with everyone affected. They help us to survive.

Reasons

Recently we’ve reloaded our Dataflow Kit framework. I believe🙏🏼it becomes more focused and more understandable for our users.

So, we need to take a test drive of our re-implemented core services Visual Data Selector and Headless Chrome running in our cloud.

The challenge is to find out how much time and effort it would take us to complete creating the code which can

  1. Extract actual data from the trustable COVID-19 source.
  2. Make extracted data available to others through the API.
  3. Create JavaScript web widgets that consume data from the API layer.

Running ahead, it took us about a day to complete a draft covering all three stages without the design of cards. 👻

Stage 1. Extract data from the source.

We found the Worldometers website to be an excellent source of COVID-19 live stats data after watching it.

The very first step is to click elements on a source page to specify data for extraction. We use Visual Data Selector to select desired data from the Worldometers’ table.

You can preview the extracted data several times until we satisfied with the results. We have to be sure we’ve selected the right CSS elements from the source HTML web page. As a result, we generate the payload, which is used for requesting the source web page and grabbing new data.

Check out the first article in the series for more details.
Stage1: How to scrape COVID-19 cases?

Stage 2. Coronavirus Tracker API.

At this stage, we develop a simple HTML server written in GO. It requests Dataflow Kit API endpoint at https://api.dataflowkit.com/v1/parse with the generated payload.

At the same time, we make a code to handle two endpoints for us:
GET /v1 - List all COVID-19 cases.
GET /v1/{cntr} - Get COVID-19 cases for a specified country.

Available {cntr} values: world, USA, spain, Slovakia and etc.

We run our open API at
https://covid-19.dataflowkit.com/v1
https://covid-19.dataflowkit.com/v1/world
You may use it absolutely for free.

Read more about this part at
Stage2: Coronavirus Tracker API.

Stage 3. Coronavirus live statistics free widgets.

Finally, we create web widgets. They visualize Coronavirus (COVID-19) live statistics coming from the API layer.

After placing a widget on any website, it shows live statistics local to a visitor depending on one’s location.

Check out the final part at
Stage3: Coronavirus live statistics free widgets.

Resources

Dataflow Kit open COVID-19 Free API:
https://covid-19.dataflowkit.com/v1
https://covid-19.dataflowkit.com/v1/world

The full code is available in our public repository at https://github.com/slotix/COVID-19.git

Free Coronavirus (COVID-19) Widgets:
https://covid-19.dataflowkit.com/

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store