We are ecstatic to announce that we are partnering with the World Resources Institute and Development Seed to open up air quality data from more sources and to scale our system to meet growing demand for data. Read on for the details and for how you can get involved in the #FixtheAQGap Campaign! Plus, fill out this form to suggest data sources to add in!
Demand is Spiking for Open Air Quality Data
When we started this project two years ago, we had a hunch that people and organizations would be able do awesome work in the fight against air pollution, if there were just easier access to even existing air quality data sources.
And it turns out this hunch was correct — more than we would have ever initially guessed. We’ve seen a dramatic increase in data requests to our system over the past few months:
And the projects behind the data requests are where the magic happens. Software developers have built tools communicating real-time air quality to the public (e.g. Smokey the Air Quality Bot, Open Air Quality App, Envi4All, and hackAIR). Students have used the system for their classwork (Examples 1 and 2). Open-source projects (See: ropenaq, py-openaq and OpenAQ Data Visualizer) and Google’s public dataset program have helped make the data even easier to access and visualize. Companies have used the data to help calibrate their low-cost sensors or as a data layer for comparisons (Examples: Airveda and ARISense). Researchers have used the system to inform their real-time wild fire model, and to predict air quality in some of the most polluted places in the world.
Seeing this usage has been incredible, but it’s also been pretty demanding on our current system, as those of you who follow along on our #dev channel on Slack may have already noticed. We need to scale.
Evolving Our System to Meet This Demand
Since the start, OpenAQ has relied largely on volunteer contributions to our open source platform to add in air quality data from across the world, and also to build the technical backend infrastructure that supports our system. The next phase of re-tooling the platform to meet this increasing demand is a hefty and specialized task, and we’d rather the community focus as much as possible on building cool things off of the platform. So we’re working with WRI and Development Seed to do the following:
- Moving away from using the database as the main storage mechanism for the data, which makes the system less costly to maintain and more reliable for users.
- Implementing rate-limiting to make the system more useable for everyone. The ballpark rate limit will be ~30 request per minute or roughly 1000 requests per hour.
If you’re interested in seeing our progress over the next couple of months, follow along — or even participate, if you’re interested — via #dev on our Slack Channel.
Current Data Coverage
To date, we aggregate air quality data from 48 countries and 5628 stations (BTW: we looked this up on Dolugen Buraalda’s handy site) — again, largely through the incredible help of volunteers letting us know about government-level data sources and software developers who then are able to write adapters to ingest the data, if all of our data format requirements are met from the originating source.
This coverage is great. But there are lots of data sources missing — both from other countries and within the countries where we already access data. We estimate there are roughly 70 countries out there with real-time air quality data of some sort.
Help us #FixtheAQGap
WRI and Development Seed are helping our community to add in more data sources to the OpenAQ Platform. This will mean connecting with new partners at air quality agencies around the world, as well as having a team of folks writing data source adapters to existing data sources we have in our “new data” queue.
How to Contribute Data Sources to Add
Over the next couple of weeks, we’ll be seeking YOUR input on potential new data sources through our #FixtheAQGap Campaign. It’s a big push to find those air quality data points that are out there, but are currently falling through the gaps.
Do you know of air quality agencies we should be connecting with to access open data? Do you know of data sources already out there, already sharing data? Fill out this form, and let us know!