How to Import Weather Data into Python

In this article we are going to use a cloud based weather API to import weather data into a Python application. These techniques can be used to extend many Python scripts and applications that need to include weather forecast data or historical weather observation data.

Why include weather data?

There are numerous reasons why weather data can be an important feature of Python applications.

One of the most common applications of Weather Forecast data we see is to help plan events or business processes. For example, in retail, weather forecast data can be used plan inventory for weather-influenced products such as warm or cold weather clothing or weather emergency related items in the event of severe weather.

Insurance companies will use weather forecast data to plan for upcoming weather events. Construction companies can plan temperature and weather-critical activities such as concrete pours by knowing the weather forecast.

Historical weather observation data helps businesses and individuals understand what happened in the past on particular days. This can be used to correlate to business metric to answer such questions as “Does rain affect my sales”? Performing statistical analysis of weather data in Python can be extremely valuable.

Historical weather summaries describe the overall weather conditions that are normally experienced at a location at a certain time of year. In addition to the ‘normal’ weather conditions, these summaries describe the extreme weather too. For example you can then understand what the normal maximum and minimum temperatures for a particular location in January. You can also easily identify the highest and lowest temperatures.

Why use Python?

Python makes it easy to consume weather data in a programmatic way so that you can perform analyses described above quickly, repeatedly and across multiple locations and time periods.

Creating a simple python script to import weather data

Importing data from a weather API is straightforward in Python. In this example we are going to use the Visual Crossing Weather API which is built around their weather data product. Other Weather APIs exists out there — choose the one that fits your need the best.

Step 1 — sign up for an account for your chosen weather API

Most weather APIs require an account. If you are following along with our sample, you can create an account here.

Step 2 — construct a weather data query.

The API we are using uses restful APIs to return either comma-separated values (CSV) or a JSON based structure. We are going to use the CSV format for simplicity. CSV is a very simple structure to describe tables of data using commas as value separators.

The API has a number of endpoints for the different types of weather.

For weather forecast, an example URL is:

https://weather.visualcrossing.com/VisualCrossingWebServices/rest/services/weatherdata/forecast?location=Herndon,VA&aggregateHours=24&unitGroup=us&key=APIKEY

For historical weather data, the simple API structure is:

https://weather.visualcrossing.com/VisualCrossingWebServices/rest/services/weatherdata/history?goal=history&aggregateHours=24&startDateTime=2019-12-04T00:00:00&endDateTime=2019-12-11T00:00:00&contentType=csv&unitGroup=us&locations=Herndon,VA&key=APIKEY

Both of these return CSV which we can import and process directly in our python script.

You can submit these directly into a browser window to see the output (remember to replace the ‘APIKEY’ parameter with your own value!)

Step 3— write a python script

We will use a couple of libraries to help download and process the weather data in Python

import csv //to process the CSV data
import codecs //to download and decode the information
import urllib.request //to request data from a URL
import sys

The next part of the script constructs the weather API query based upon the type of query and location requests. Weather APIs support many more parameters that could be added but we are going to keep it simple.

First we construct define the root of the URL:

BaseURL = ‘http://weather.visualcrossing.com/VisualCrossingWebServices/rest/services/weatherdata/'

Then we add the common parameters including the location and API key. Note that we are using the input parameters to the Python Script to allow the user to specify the location and key. These values are contained in the sys.argv array.

DateParam = sys.argv[2].upper() 
# Set up the location parameter for our query
QueryLocation = ‘&location=’ + urllib.parse.quote(sys.argv[1]) 
# Set up the key parameter for our query
QueryKey = ‘&key=’ + sys.argv[3]

The next part of the query is specific to the type — weather forecast and historical weather data. For the weather forecast we simply need the period of the forecast. aggregationHours=24 equates to a day — we could retrieve an hourly forecast by changing the value to 1.

# Set up the specific parameters based on the type of queryif DateParam == ‘FORECAST’: 
 print(‘ — Fetching forecast data’) 
 QueryTypeParams = ‘forecast
        &aggregateHours=24&unitGroup=us&shortColumnNames=false’
else: 
 print(‘ — Fetching history for date: ‘, DateParam) 
 # History requests require a date. We use the same date for start
   and end since we only want to query a single date in this example
 QueryDate = ‘&startDateTime=’ + DateParam +
        ‘T00:00:00&endDateTime=’  + sys.argv[2] + ‘T00:00:00’ 
 QueryTypeParams = ‘history
        &aggregateHours=24&unitGroup=us&dayStartTime=0:0:00
        &dayEndTime=0:0:00’ + QueryDate

Now we construct the full query from all the parts:

# Build the entire query
URL = BaseURL + QueryTypeParams + QueryLocation + QueryKey

We are now ready to submit the query. This can be achieved in three lines using the libraries we imported at the start:

# Parse the results as CSV
CSVBytes = urllib.request.urlopen(URL)
CSVText = csv.reader(codecs.iterdecode(CSVBytes, ‘utf-8’))

This requests the data and then parses the results as a CSV. The CSVText instance helps us extract the data easily.

Finally we can use the data. In our simple example, we’ll just show the output as a simple table output:

for Row in CSVText:
  if RowIndex == 0:
    FirstRow = Row
  else:
    print(‘Weather in ‘, Row[0], ‘ on ‘, Row[1])
    ColIndex = 0
    for Col in Row:
      if ColIndex >= 4:
        print(‘ ‘, FirstRow[ColIndex], ‘ = ‘, Row[ColIndex])
      ColIndex += 1
    RowIndex += 1

You can find the full code here.

Next steps

The simple code sample above shows how easy it is to retrieve weather from a weather API and integrate it into a Python application. From this point it would be straightforward to integrate the weather forecast into a planning application or use the historical data for statistical analysis.

Questions? Please let me know below!