Mini-Project: Meteorites Throughout History — A Dash Web App To Visualize One Of Humanity’s Biggest Existential Threats
Introduction
Meteorites are a fascinating existential threat for humanity, as our systems to detect them, predict their trajectory, and destroying, or deflecting them are very primitive. I will refer to the Chelyabinsk meteorite that fell in 2013. It injured plenty of people, and the scary part is that it wasn’t detected before reaching our atmosphere!
That said, while browsing NASA’s available data, I stumbled upon a data-set of all meteorites fallen on earth from the 1400s, to 2013.
The goal of this project is to create a dashboard to plot out all the fallen meteorites on a world map, with a filter for the year and the most significant meteorites (the one’s that have a mass (in kg) that has scored better than the 99.9th percentile).
Data cleaning and preparation
Let’s see what out data looks like
Missing data
Let’s see if there’s some missing data. We will use the missingno library to visualize the missing data in each column. It’s a fast, and superficial, way to see missing data in a data-frame.
# Visualize missing values
import missingno as msno
msno.matrix(df)
The white stripes represent NaN data. As we need the reclat (latitude) and reclong (longitude) columns to position our meteorite, we will drop all rows that are NaN in this column.
Distribution
Let’s see how the data is distributed:
sns.distplot(df['year'], bins = 150)
Most of the data is around the 2000s.
Year attribute
Let’s see if there are some anomalies in the year column. Let’s group by the year to have a full data-frame with the unique year and its frequency in the data-frame.
df.groupby('year').count()
There seems to be a mistake in the data at the last line (year = 2101). After looking into it, the year this meteorite fell is 2010 (Northwest Africa 7701).
Modifying and adding attributes
Let’s convert the year from float to year. As we have a decimal point (see above), we will want to convert it to an integer to remove the decimal, and then to a string.
df['year'] = df['year'].astype(int)
df['year'] = df['year'].astype(str)Then, we will convert the mass column (which is in grams) to kilograms, as mass is easier to interpret in this unit.
#Converting grams to kg
df['mass (kg)'] = df['mass (g)']/1000Let’s add a boolean column which will mark the meteorites that are above or equal to the 99.9th percentile, mass-wise. This will be used for our biggest meteorites in history filter.
#Boolean column for top 0.1% meteorites in terms of mass
df['top 0.1% mass'] = df['mass (g)']>=np.percentile(df['mass (g)'], 99.9)Now, let’s create a new column which will be a text string combining three columns (name, mass, and year). This column will be used as tool-tip when hovering over a meteorite on the map.
df['text'] = 'Name: ' + df['name']\
+ '; Mass of meteorite in kg: ' + df['mass (kg)'].astype(str)\
+ '; Year: ' + df['year']Good! The data is ready for the visualizations.
Visualization
Let’s start out with a simple matplotlib scatterplot visual. Our x axis being the longitude, and y axis the latitude.
#Let's plot the data in a scatterplot
plt.scatter(x=df['reclong'], y=df['reclat'])
plt.rcParams['figure.figsize'] = (50,35)
plt.show()
This looks a bit like the world map, no?
Let’s use the plotly library to create a world map, and plot the points out.
#Let's improve the visual
fig = px.scatter_geo(df, lon='reclong', lat='reclat', hover_name='name')
fig.show()
That’s more like it.
That said, we will use plotly.graph_objects to create the visual which will be embedded in the web app.
Here’s the code to produce it:
fig = go.Figure(
data=go.Scattergeo(
lon = df['reclong'],
lat = df['reclat'],
text = df['text'],
mode = "markers",
marker = dict(
size = 8,
opacity = 0.8,
reversescale = True,
autocolorscale = False,
symbol = 'circle',
line = dict(
width=1
),
colorscale = 'Viridis',
cmin = 0,
color = df['mass (kg)'],
cmax = df['mass (kg)'].max(),
colorbar_title="Mass of meteorite (in kg)"
)
)
)fig.update_geos(
showcountries=True
)fig.update_layout(
title = 'Meteorite landings',
height = 600,
)
Good stuff! We will use this visual for our web application dashboard.
App developement
Dash will be used to develop our web app. First, here are the imports needed to develop the web app:
import numpy as np
import pandas as pd
import plotly.express as px
import plotly.graph_objects as go
import dash
from dash import dcc
from dash import html
from dash.dependencies import Input, Output, ALL, StateThen, we need to initialize the app and create a server variable. This variable is needed to deploy on Heroku (and is optional if you only want to run it locally):
app = dash.Dash(__name__)
server = app.server #Used for deploymentLet’s create the header and introduction text of our app.
app.layout = html.Div(
children=[html.Div(
style={
'backgroundColor':'#303030',
'color':'white',
'fontFamily': '"Lucida Console", "Courier New"'
},
children=[
#Title
html.H1(
style = {
'textAlign': 'center',
},
children="Meteorites fallen on earth",
className="header-title"
),
#Description
html.H2(
style = {
'textAlign': 'center',
},
children="A dashboard to visualize one of humanity's biggest existential threats",
className="header-description"
),
],
className="header"
)Now, let’s create the year filter. It will be a dropdown filter:
#Year filter
html.Div(
children=[
html.Div(children = 'Year',
style={'paddingTop':'5px',
'fontSize': "20px",
'fontFamily': '"Lucida Console", "Courier New"'
},
className = 'menu-title'),
dcc.Dropdown(
id = 'year-filter',
options = [
{'label': Year, 'value': Year}
for Year in np.append(np.sort(df.year.unique()),"All")
],
value = "All",
className = 'dropdown',
style={'fontSize': "20px",
'textAlign': 'center',
'fontFamily': '"Lucida Console", "Courier New", monospace'},
),
],
className = 'menu',
)Now let’s create the filter for the biggest meteorites. It will be in a checkbox format:
#Biggest meteorite filter
html.Div(
children=[
html.Div(children = 'Would you want to see the most significant meteorites in history (top 0.1% in terms of mass in kg) ?',
style={'paddingTop':'15px',
'fontSize': "20px",
'fontFamily': '"Lucida Console", "Courier New"'},
className = 'menu-title'),
dcc.Checklist(
['Yes'],
id = "checklist",
inline=True,
value = "",
style={'paddingTop':'5px',
'fontSize': "20px",
'fontFamily': '"Lucida Console", "Courier New"'}
)
],
className = 'menu'
)Now let’s include our world visual (created above) in the web app:
#Adding the world visual
html.Div(
children=[
html.Div(
children = dcc.Graph(
id = 'world_chart',
figure = fig, #fig is created above
),
style={'width': '100%', 'display': 'inline-block'},
)])])We now have the “skeleton” of our web app. Here’s the UI:
We now need to create the interactivity and dynamic filtering of our web app. To do this on Dash, we need to set the app call-backs.
#Callback to activate filters
@app.callback(
dash.dependencies.Output("world_chart", "figure"),
[dash.dependencies.Input("year-filter", "value"),
dash.dependencies.Input("checklist", "value")],
)Now let’s define the function that will update our world visual. We will filter our data-set based on the incoming parameters (year and checkbox).
def update_charts(Year, val):
if val == ["Yes"] and Year == "All":
filtered_data = df[df['top 0.1% mass']==True]
else:
if Year == "All":
filtered_data = df
else:
Year = int(Year)
filtered_data = df[df["year"] == Year] #Creating the figure with our filtered dataset
fig = go.Figure(data=go.Scattergeo(
lon = filtered_data['reclong'],
lat = filtered_data['reclat'],
"""
The function is incomplete. See the full code above to create the world map.
"""
Ta-da! The apps’ filters are now activated, and the world map visual will be refreshed once a filter is applied.
Let’s test it out and see how it works:
Nice stuff! Let’s deploy it to Heroku.
Deployement
We need to add 3 files in the repository before deploying to Heroku.
- Procfile (actions on startup)
- requirements.txt (to install dependencies)
- runtime.txt (specify python version, optional)
The Procfile (app being the name of my application which is a py file):
web: gunicorn app:serverThe requirements.txt file:
numpy
pandas
Brotli==1.0.9
click==8.0.3
dash==2.2.0
dash-core-components==2.0.0
dash-html-components==2.0.0
dash-table==5.0.0
Flask==2.0.2
Flask-Compress==1.10.1 itsdangerous==2.0.1
Jinja2==3.0.3
MarkupSafe==2.0.1
plotly==5.5.0
six==1.16.0
tenacity==8.0.1
Werkzeug==2.0.2
gunicorn==20.1.0The app is now ready to be deployed!
Here the link to the app! Don’t hesitate to play around with it!
Thank you for reading!