Building a web application for active low earth orbit satellites
Of late, one of my active interests has been ‘space’. You will catch me watching a live stream of rocket launches and eventual satellite(s) deployment (most times an animated deployment) into orbit.
It is always fascinating to see how different fields of engineering (material, propulsion, instrumentation, software, fluid among others) are at play to ensure a rocket launch is successful.
I have wanted to work on a web application where the data changes in near real-time and showing how satellites change position orbiting the earth seems a perfect fit. So I decided to build one that shows the position of active satellites in low earth orbit (more on why I was specific on the orbit later)
Pyorbital
Pyorbital is a python library that provides data about the position of a satellite in orbit. The library uses what is known as ‘TLEs (Two-line element set) https://en.wikipedia.org/wiki/Two-line_element_set’ entry for individual satellites on the backend to find the position of the satellite.
One only needs to provide the satellite name, and the time of interest, the library takes care of the rest as shown below;
Pyorbital has a limitation. It can only be used for satellites in the low earth orbit (LEO-orbit with altitudes between 160km and 2000km ) (read more on the classification of orbits https://en.wikipedia.org/wiki/List_of_orbits#:~:text=planet%20Uranus.-,Altitude%20classifications%20for%20geocentric%20orbits,-%5Bedit%5D)
When the library is used for satellites in other orbits aside from the low earth orbit, it returns a ‘NotImplementedError: Deep space calculations not supported’ as shown below ;
Workflow
The workflow for building the web application is shown below ;
To build the web app, I needed the data on the active satellites and that’s exactly what the CelesTrak csv hosted on the web ( https://celestrak.com/NORAD/elements/?FORMAT=json#:~:text=or%20so)%20Brightest-,Active%20Satellites,-Analyst%20Satellites) provides.
data wrangling.py file
The etl.py in summary access the data from the CelesTrak csv hosted on the web, cleans up the data and stores the cleaned DataFrame in a google cloud storage bucket.
The etl.py file is explained in detail below;
- reading the active satellites data into a DataFrame and checking for the info attribute of the DataFrame
import pandas as pd
active_sat_df = pd.read_csv('https://celestrak.com/NORAD/elements/gp.php?GROUP=active&FORMAT=csv')
active_sat_df.info()
because of the limitation of Pyorbital, the active satellites in low earth orbit needs to be filtered out of the active satellites’ DataFrame.
To do this, I referred to another definition of low earth orbit in terms of ‘Mean Motion and Eccentricity’ which states that;
A low Earth orbit (LEO) is an Earth-centered orbit near the planet, often specified as having an orbital period of 128 minutes or less (making at least 11.25 orbits per day (orbit per day refers to the mean motion) and an eccentricity less than 0.25. (Source: https://en.wikipedia.org/wiki/Low_Earth_orbit)
#active satellites in the low earth filtered out of the active satellites DataFrame using Mean Motion and Eccentricity features of the satellites.
active_leo_sat_df = active_sat_df.loc[(active_sat_df['MEAN_MOTION']>=11.25) & (active_sat_df['ECCENTRICITY']<0.25)]
#reset the index
active_leo_sat_df.reset_index(drop = True, inplace = True)because the Pyorbital library only needs the satellite name (OBJECT_NAME in this case) and time, I decided to drop all other columns aside OBJECT_ID.
#dropping all other columns
active_leo_sat_df = active_leo_sat_df.drop(active_leo_sat_df.loc[:,'EPOCH':'MEAN_MOTION_DDOT'], axis = 1)The OBJECT_ID column contains the International Designator for each satellite and the first part of the International Designator data is the launch year of the satellite (Source: https://en.wikipedia.org/wiki/International_Designator). I decided to replace the OBJECT_ID column data with only the first part of the data to get the launch year for each active satellite in low earth orbit, rename the columns and drop duplicates if any.
#replace the OBJECT_ID column data with the first part of the data
for object_id in active_leo_sat_df['OBJECT_ID']:
if '-' in object_id:
index = object_id.find('-')
object_id_format = object_id[0:index]
active_leo_sat_df['OBJECT_ID'] = active_leo_sat_df['OBJECT_ID'].replace(to_replace = object_id, value = object_id_format)
else:
pass
#rename the columns
active_leo_sat_df.columns = ['ObjectName', 'YearOfLaunch']
#dropping duplicates if any
active_leo_sat_df = active_leo_sat_df.drop_duplicates(subset = 'ObjectName')
active_leo_sat_df.reset_index(drop = True, inplace = True)CelesTrak also provides data that groups satellites according to the purpose such as communications, weather, earth observation, navigation among others and I figured I could use some more information for the web application. So, I collected all the links in each category into a list.
#collecting all the data links in each category into a list
weather_sat_list = ['https://celestrak.com/NORAD/elements/gp.php?GROUP=weather&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=noaa&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=goes&FORMAT=csv']
earth_observation_sat_list = ['https://celestrak.com/NORAD/elements/gp.php?GROUP=resource&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=sarsat&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=dmc&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=tdrss&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=argos&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=planet&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=spire&FORMAT=csv']
communications_sat_list = ['https://celestrak.com/NORAD/elements/gp.php?GROUP=intelsat&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=ses&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=iridium&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=iridium-NEXT&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=starlink&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=oneweb&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=orbcomm&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=globalstar&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=swarm&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=amateur&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=x-comm&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=other-comm&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=satnogs&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=gorizont&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=raduga&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=molniya&FORMAT=csv']
navigation_sat_list = ['https://celestrak.com/NORAD/elements/gp.php?GROUP=gnss&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=gps-ops&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=glo-ops&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=galileo&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=beidou&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=sbas&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=nnss&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=musson&FORMAT=csv']
scientific_sat_list = ['https://celestrak.com/NORAD/elements/gp.php?GROUP=science&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=geodetic&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=engineering&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=education&FORMAT=csv']
miscellaneous_sat_list = ['https://celestrak.com/NORAD/elements/gp.php?GROUP=military&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=radar&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=cubesat&FORMAT=csv',
'https://celestrak.com/NORAD/elements/gp.php?GROUP=other&FORMAT=csv']Then I defined a function that;
- reads all the links in each list.
- concatenates data from the individual link.
- drop unwanted columns and rename the only column left.
#function to read all the links in a list, concatenates data from the individual link, drop unwanted columns and rename the only column left.
def sat_classification(sat_list):
df = pd.DataFrame()
for url in sat_list:
sats_df = pd.read_csv(url)
df = pd.concat([df, sats_df])
df.reset_index(drop = True, inplace = True)
df = df.drop(df.loc[:,'OBJECT_ID':'MEAN_MOTION_DDOT'], axis = 1)
df.columns = ['ObjectName']
return dfI am then able to use the function to obtain the data for each individual satellites category (DataFrame) by purpose and also create a new column named ‘Purpose’.
weather_sat_df = sat_classification(weather_sat_list)
weather_sat_df['Purpose'] = 'Weather'
earth_observation_sat_df = sat_classification(earth_observation_sat_list)
earth_observation_sat_df['Purpose'] = 'Earth Observation'
communications_sat_df = sat_classification(communications_sat_list)
communications_sat_df['Purpose'] = 'Communications'
navigation_sat_df = sat_classification(navigation_sat_list)
navigation_sat_df['Purpose'] = 'Navigation'
scientific_sat_df = sat_classification(scientific_sat_list)
scientific_sat_df['Purpose'] = 'Scientific'
miscellaneous_sat_df = sat_classification(miscellaneous_sat_list)
miscellaneous_sat_df['Purpose'] = 'Miscellaneous'After that, I concatenates all the individual satellites category DataFrame into one master DataFrame.
master_df = pd.concat([weather_sat_df, earth_observation_sat_df, communications_sat_df, navigation_sat_df, scientific_sat_df, miscellaneous_sat_df])
master_df.reset_index(drop = True, inplace = True)Then I merged the active satellites in low earth orbit DataFrame with that of the master DataFrame using the ‘ObjectName’ column as merge/join condition, the merge type a left type… similar to when one is using LEFT OUTER JOIN queries in SQL).
active_leo_sat_df = pd.merge(active_leo_sat_df, master_df, how ='left', on =['ObjectName'])What this does is add an additional column ‘Purpose’ to the active satellites in low earth orbit DataFrame using the condition that ObjectName in both DataFrames are equal/same.
Finally, for ObjectName that didn’t meet the condition, I replaced the ‘nan’ in the purpose column with ‘Miscellaneous’, drop duplicates if any and save the DataFrame as a CSV file in a google cloud storage bucket named ‘active-leo-satellites’.
#replaces nan with Miscellaneous in the purpose column
active_leo_sat_df['Purpose'] = active_leo_sat_df['Purpose'].replace(np.nan, 'Miscellaneous')
#drops duplicates in the DataFrame and reset index
active_leo_sat_df = active_leo_sat_df.drop_duplicates(subset = 'ObjectName')
active_leo_sat_df.reset_index(drop = True, inplace = True)
#stores the DataFrame in a google cloud storage bucket as a csv
active_leo_sat_df.to_csv('gs://active-leo-satellites/active leo satellites.csv', index = False)With that done, I deployed the etl.py file as a function in google cloud function to make it event driven. The trigger is an HTTP trigger (https://europe-west2-leo-satellite-overview-project.cloudfunctions.net/etl-function) which ensures the function (data wrangling.py) runs anytime the HTTP is loaded.
Anytime the http is loaded, it return that the ‘function ran successfully’
To ensure that I don’t have to load the HTTP at random, I decided to set up a cron job using google cloud scheduler. This ensures the function runs every midnight.
The beauty of all this is to ensure new satellite data are captured while also ensuring satellites that are now inactive are also removed from the web application.
main.py file
The main.py file is where I read the CSV file that was generated by the data wrangling.py file and then go ahead to create the web framework for the web app using the Dash library and the visualisations using the Plotly library.
The main.py file can be found here.
The web app generated by running the main.py file works on the local computer, so to be able to share the web app with other people, I had to deploy it to the google app engine.
The main.py file alongside the requirements.txt file (names of libraries used), assets folder (contains a CSS file which styles the background colour and margin of the web app) and the app.yaml (file that sets up the app engine) were then deployed to the google app engine.
Quick Insight
There has been an increasing number of satellites launched into low earth orbit year on year since 2019 and this is mostly due to the increasing number of communications satellites (mostly Starlink and OneWeb) being launched to provide internet to everywhere on planet earth.
With more communication satellite (Kuiper, ASTS) projects to come into operation in the next few years, the increasing trend is sure to continue.
The web application can be seen here.
The repository can be found here.
Read my other posts at https://medium.com/me/stories/public
Fun fact: A rocket at a lower altitude can reach orbit while another rocket at a higher altitude may not make it to orbit (This launch at an altitude of 531km and speed of 6.5km/s https://youtu.be/HztFm2XGO7s?t=8420 didn’t make it to orbit while this launch https://youtu.be/6nODVPGHQcc made it to orbit at 216km altitude and 7.93km/s).
While orbital altitude starts at around 160km, what ensures a rocket launch reaches orbit is that the travel speed of the rocket must reach the orbital velocity of around 7.5km/s.
Thank you for reading.
