Mapping the broadcasted news related to natural disasters
We experienced heavy shifts in temperature and weather patterns. Climate change is related to natural shifts and much more beneficial human activities. The broadcasted news articles related to climate change and the rise of natural disasters describe various cases.
From a geospatial engineer’s perspective, we need to inspect the spatial patterns of natural disasters. There are only a few dedicated natural disaster databases available. The EM database is a global database of disasters containing foundational data of occurrence and effects. It helps policymakers identify the most common disaster types at the country level and the ones having significant historical effects on specific human populations. For our geospatial scenario, we need a broader thematic aspect and fine grained location aware database.
We decided using one of the most comprehensive news collection named “Global Database Events of Tone and Language” (GDELT) as the ground truth. This massive knowledge graph saves the broadcasted news and allows access using a dedicated Google BigQuery instance.
An overview of natural disasters
Climate change can contribute to various natural disasters, intensifying their frequency, severity, and impact.
Hurricanes and Tropical Storms
Warmer ocean temperatures fuel the formation and intensification of hurricanes and tropical storms. Climate change leads to increased rainfall, storm surges, and higher wind speeds. All these effects exacerbate the destructive potential of these events.
Droughts
Climate change alters precipitation patterns and leads to prolonged periods of reduced rainfall. We will see increased evaporation rates. Such conditions result in severe droughts, affecting agriculture, water resources, and ecosystems.
Floods
Changes in precipitation patterns can also contribute to more intense rainfall events, increasing the risk of floods. The rise of sea levels and storm surges forces coastal flooding.
Heatwaves
As global temperatures rise, the frequency, duration, and intensity of heatwaves can increase. Heatwaves have severe effects on human health, agriculture, and ecosystems.
Wildfires
Higher temperatures, prolonged droughts, and altered vegetation patterns create conditions conducive to more frequent and intense wildfires. Climate change contributes to the expansion of wildfire-prone areas.
Storm Surges
Rising sea levels, primarily caused by climate change, heighten the impact of storm surges associated with hurricanes, cyclones, and other coastal storms. This results in extensive flooding and erosion along coastlines.
Landslides
Increased rainfall and the destabilization of soil due to changes in temperature and moisture patterns elevate the risk of landslides in certain areas.
While climate change influences the occurrence and severity of these disasters, they can also be influenced by other factors, such as local geography and land-use practices.
Mapping the broadcasted news related to natural disasters
A massive knowledge graph saves the extracted semantic entities of the broadcasted news worldwide. Understanding the spatial patterns of such a massive knowledge graph is often difficult for non geospatial experts. Finding hotspots of mentioned locations is one common use case. News article contain various location entities. These locations not only represent the event location. But, the knowledge graph allows the extraction of the mentioned locations related to a specific theme. So that we can extract every mentioned location using news related to natural disasters.
We need to inspect the hierarchical taxonomy for natural disasters. Filter the knowledge graph using the specific themes like hurricane, wildfire, cold snaps, heat waves and so on. Aggregate the mentioned locations using a spatial grid.
We should expect some false positives, but the sum of all extracted locations should detect spatial patterns and give us a coarse-grained overview.
Accessing the geospatial features
Geospatial features play a crucial role in understanding and monitoring natural disasters. By analyzing the spatial distribution of these events, we can gain valuable insights into their patterns, severity, and effects. In this chapter, we will explore how geospatial features are created and accessed, specifically focusing on their generation from broadcasted news related to natural disasters. We will discuss the concept of hotspot identification, spatial grid aggregation, and the availability of geospatial features daily.
Broadcasted news serves as a valuable source of information for identifying natural disasters globally. Geospatial features represent extracted locations from broadcasted news to identify the locations of these events. Natural disasters such as hurricanes, floods, wildfires, and others are captured and represented as geospatial features.
In your dedicated spatial data science environment, you just need ArcGIS API for Python and the geodisasters API.
# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0
from arcgis.gis import GIS
from arcgis.features import FeatureLayer, FeatureSet
from datetime import datetime, timedelta
import os
import pandas as pd
import requestsWe need some utility functions querying the broadcasted news related to natural disasters using a dedicated Rapid API account. You need to activate your Rapid API account. Please, check out the RapidAPI Account Creation and Management Guide for more details.
# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0
def hotspots_disasters(seen_date, api_key):
"""
Queries the news related to natural disasters and returns the hotspots as a feature set.
You need to specify the date of interest.
The underlying knowledge graph collects locations since 2023-05-24 and yesterday should be the latest available date.
:param datetime seen_date: The date of interest.
:param api_key str: Your RapidAPI API key.
"""
host = 'geodisasters.p.rapidapi.com'
url = f'https://{host}/hotspots'
headers = {
'x-rapidapi-host': host,
'x-rapidapi-key': api_key
}
params = {
'date': seen_date.strftime('%Y-%m-%d'),
'format': 'esri'
}
featureset_result = requests.get(url, headers=headers, params=params)
featureset_result.raise_for_status()
featureset = FeatureSet.from_dict(featureset_result.json())
return featureset
def hotspots_disasters_since(start_date, api_key):
"""
Queries the news related to natural disasters and returns the hotspots as a feature set.
You need to specify the start date, the end date equals yesterday.
Be aware if you specify a date range greater than 10 days you exceed the free tier!
:param datetime start_date: The first date of interest.
:param str api_key: Your RapidAPI API key.
"""
end_date = datetime.utcnow() - timedelta(days=1)
datespan = end_date - start_date
featureset_combined = None
for days in range(0, datespan.days + 1):
seen_date = start_date + timedelta(days=days)
featureset = hotspots_disasters(seen_date, api_key)
if None is featureset_combined:
featureset_combined = featureset
elif not None is featureset:
featureset_combined.features.extend(featureset.features)
return featureset_combinedThe following snippet queries the mentioned hotspots related to natural disasters of 24th May 2023.
seen_date = datetime(2023, 5, 24)
api_key = '<RapidAPI API key>'
hotspots_featureset = hotspots_disasters(seen_date, api_key)
hotspots_featureset.sdf
The geospatial features primarily represent the hotspots of locations associated with natural disasters. But, we are only interested in hotspots where the intensity of a particular event is notably high. These hotspots allow us to identify regions FLOOD, have a relatively high mentioned count cause news related to “Italy floods aftermath” were broadcasted on 24th May 2023.
We want to use the mapping capabilities and manage the geospatial features using ArcGIS Online. Therefore, we create a simple geospatial intelligence portal wrapping the mapping and spatial capabilities.
# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0
class GeospatialIntelligencePortal(object):
def login(self, username):
"""
Login into ArcGIS Online using a password input.
:param str username: Your ArcGIS Online username.
"""
self._gis = GIS(username=username)
def plot_features(self, featureset):
"""
Creates and returns a simple map view with the plotted geospatial features.
:param FeatureSet featureset:
"""
map_view = self._gis.map('Europe')
featureset.sdf.spatial.plot(map_view,
renderer_type='c',
method='esriClassifyNaturalBreaks',
class_count=5,
col='count',
cmap='YlOrRd',
alpha=0.35)
return map_viewLet us visualize the hotspots using the map view. Please, check out the Sign up for a free ArcGIS Developer account for more details.
geoint_portal = GeospatialIntelligencePortal()
arcgis_user = '<ArcGIS username>'
geoint_portal.login(arcgis_user)
...
geoint_portal.plot_features(hotspots_featureset)
We used a spatial grid to aggregate and analyze geospatial features efficiently. The spatial grid divides the geographical area into grid cells, enabling the aggregations of hotspots within specific grid cells. Aggregating hotspots in a spatial grid provides a structured representation of the spatial distribution of natural disasters.
We need some utility functions aggregating the broadcasted news related to natural disasters. The dedicated endpoint is the only difference to the existing utility functions.
# author: Jan Tschada
# SPDX-License-Identifer: Apache-2.0
def aggregate_disasters(seen_date, api_key):
"""
Queries the news related to natural disasters and returns the aggregated hotspots as a feature set.
You need to specify the date of interest.
The underlying knowledge graph collects locations since 2023-05-24 and yesterday should be the latest available date.
:param datetime seen_date: The date of interest.
:param api_key str: Your RapidAPI API key.
"""
host = 'geodisasters.p.rapidapi.com'
url = f'https://{host}/aggregate'
headers = {
'x-rapidapi-host': host,
'x-rapidapi-key': api_key
}
params = {
'date': seen_date.strftime('%Y-%m-%d'),
'format': 'esri'
}
featureset_result = requests.get(url, headers=headers, params=params)
featureset_result.raise_for_status()
featureset = FeatureSet.from_dict(featureset_result.json())
return featureset
def aggregate_disasters_since(start_date, api_key):
"""
Queries the news related to natural disasters and returns the aggregated hotspots as a feature set.
You need to specify the start date, the end date equals yesterday.
Be aware if you specify a date range greater than 10 days you exceed the free tier!
:param datetime start_date: The first date of interest.
:param str api_key: Your RapidAPI API key.
"""
end_date = datetime.utcnow() - timedelta(days=1)
datespan = end_date - start_date
featureset_combined = None
for days in range(0, datespan.days + 1):
seen_date = start_date + timedelta(days=days)
featureset = aggregate_disasters(seen_date, api_key)
if None is featureset_combined:
featureset_combined = featureset
elif not None is featureset:
featureset_combined.features.extend(featureset.features)
return featureset_combinedThe following snippet aggregates the mentioned hotspots related to natural disasters of 24th May 2023.
seen_date = datetime(2023, 5, 24)
aggregated_featureset = aggregate_disasters(seen_date, api_key)
aggregated_featureset.sdf
Every mentioned hotspot has its own spatial grid cell. So that the count of the aggregated geospatial features exactly matches the hotspot location features. Mapping the aggregated geospatial features using the map view displays the grid cells using the sample color band.
Every day, serverless functions update the geospatial features related to natural disasters to provide near real-time information. Daily access to geospatial features allows for timely decision-making, disaster response, and resource allocation.
Summary
Geospatial features derived from broadcasted news provide a valuable resource for understanding and responding to natural disasters. By identifying hotspots and aggregating them using a spatial grid, we can gain insights into the spatial distribution and severity of these events. Accessing geospatial features daily enables active monitoring, response planning, and resource allocation. Continued advancements in geospatial analysis and data availability will further enhance our ability to mitigate the effects of natural disasters and build resilient communities.
For the next months, we are monitoring broadcasted news related to natural disasters and sharing common practices related to geospatial intelligence and designing a geospatial knowledge graph.
