Using Taxonomy to Monitor the Environment with a Sky Hub Tracker
Sky Hub is the new scientific nonprofit behind the pioneering effort to create a free, public database with open-source software that will collect data about anomalous objects (read UFOs) in the skies above us. The project’s founder discusses the project more here.
The Sky Hub system is centered around a low cost, low maintenance unit called the “Tracker,” which is connected via the internet to the Sky Hub Cloud. Once fully functional, you will be able to set up a Tracker outside your home where it will automatically track and categorize objects in flight overhead using its multi-functional sensor array and camera system. Dr. David Moore briefly described the Tracker in this recent article. Either as a single unit, or as part of a matrix of sensors over a given geographical area, the Tracker enables users and those accessing the Sky Hub Cloud data suite to understand and analyze the wide open spaces above our homes.
Critical engineering and development tasks are still underway for the initial release, including fully defining a machine learning framework for use in categorizing sensor data collected at the Trackers. Continuing volunteer time spent on the project by users and the engineering community has been, and will be, key to the project’s success. Some of the software development is discussed here.
Unidentified Phenomena
The inspiration for this effort was initially sparked by the continued reporting of unknown objects witnessed in the skies across the globe — as well as one unknown object caught during Sky Hub’s early camera tests.
Although the vast majority of UFO/UAP/UAV sightings are misidentified prosaic celestial bodies, aircraft, or quad-copter enthusiasts, many genuinely anomalous objects have been observed overhead for decades (and longer) — and continue to be reported daily all over the world. That alone merits a serious study of this topic if is feasible. We think it is.
Furthering the public’s interest in this subject, the Navy recently released and authenticated several videos of reported military encounters with “unidentified phenomena” in American airspace. These objects were not marked, had no transponder signal, did not respond to efforts to contact them, and appeared to the servicemen and women who encountered them to behave in ways thought not possible for conventional and military aircraft. To date, multiple United States senators have gone on record to state their desire to see movement towards the analysis and understanding of these events, if a small number may be of defense related concern. Source: Politico
Although it is noteworthy that various governments plan to investigate these events again (see Project Sign, Project Grudge, Project Blue Book, AATIP & presumably others), the ability to track anomalous objects in the sky above us is not merely the government’s concern.
Commercial Projects
As the range of commercial and consumer aircraft and drone technology continues to increase, the need for successful and fast sensing, analysis, and categorization of overhead objects will continue to rise. Drones, although initially limited to military applications, have seen a tremendous increase in their application and market share in various commercial fields. Currently, the Drone Market opportunity is estimated at nearly $100 Billion by Goldman Sachs, with significant investment increases by non-governmental entities. These applications suggest that the skies above our homes will become busier than ever, with little thought given to the safety and security concerns such technology may create.
Exacerbating this issue is the lack of investment and interest in commercial drone tracking/categorization technologies from ground-based, static locations. Much of the effort in this area has been focused on the reverse problem: tracking of objects on the ground through drone-mounted cameras has seen significant increases in research intensity.
Satellite Based Endeavors
Historically, the effort to collect atmospheric data utilizing automatic static cameras has been in astronomy. The Baker Nunn camera system created in the late 1950s in the United States by the Smithsonian Astrophysical Observatory was a particular success. Initially, these systems were designed to track human-made satellites as they orbited the Earth. They were composed primarily of a large aperture telescope and camera system, which allowed for tracking and kinematic analysis of satellites through various exposure shots. In later years many of these systems were converted for other applications, such as automatic astronomical data collection. Still others were expanded to become part of the United States Space Surveillance Network.
While this system allows for data collection from various set points, use as an accurate orbital surveillance system for large areas has always been limited due to the costs inherent in creating and maintaining these units.
Until now, private space surveillance networks were unfeasible due to the limited computational capabilities available to researchers. This challenge has led many of the potential applications for these systems outside astronomy to be only partially realized.
Proliferation of Machine Learning
Advancements in computational and hardware engineering have allowed for significant mitigation of these two challenges, allowing for atmospheric tracking devices to be deployed in significantly greater numbers and with exponentially more computational power. The crowd-sourcing/crowd building method has been successful for other systems with similar goals, where individual users could build and collect data using a stationary collection system. Efforts led by SETI, the Weather Underground, and the American Meteor Association have all been successful in this area, collecting data with users tying into a centralized repository for analysis.
However, none of these systems attempted to automatically track and identify flying objects, which is Sky Hub’s goal.
Taxonomy is the Science of Sky Hub
Taxonomy is the branch of science that is concerned with classification, grouping and sorting of all things. Besides the hardware requirements and engineering needed to develop a system like Sky Hub, one major hurdle the science team faces is creating a “sky-bound taxonomy” for use by the Tracker’s internal machine learning algorithm to do its automated data sorting. Taxonomic tools and decision trees for identifying unknown objects are relatively commonplace in optical machine learning systems. However, no such project has been undertaken for such a wide range of objects flying in the sky.
The Sky Hub team has developed an initial taxonomy and decision schema for this project, which we will explain and define below.
Taxonomy: Is it a bird? Is it a plane? Is it a man in a Jetpack?
The goal of the Sky Hub platform is to observe events of unknown origin occurring in the skies. However, the attempt to characterize an unknown by looking for specific features we expect creates significant challenges and introduces the implicit biases of the same team that builds it.
First, Sky Hub will characterize known objects using a machine learning approach. Anything left will be by default “unknown.” By becoming better at understanding and categorizing these known objects, the software will gain a more robust and more compelling set of unrecognized events.
The initial taxonomy will include the objects and events commonly expected to be captured by Sky Hub units deployed in the field. These will be used to “silo” known events. If the observed object fits into any of these known event silos, based on comparing the observed object’s characteristics versus the performance and features of known objects, it will be categorized and filed away.
In order to develop this taxonomy, it will be necessary to feed the system enough data so that it learns what the factors are, like commercial and military aircraft, animals, bugs, weather phenomena, meteors, satellites, and other common aerial phenomena.
For example after the unit observes an object, collecting the video data using the fisheye camera, the onboard computer will treat the image as a single point object initially, and its movement/displacement will be tracked across the fisheye lens, frame by frame. By comparing the objects’ apparent displacement vectors over various frames, the software will obtain normalized velocity, normalized acceleration, and the objects’ displacement vectors throughout its time being observed.
The Sky Hub team assumes that the possible distributions of these kinematic parameters will be relatively distinct across different object classes. In other words, taken together with the velocity, acceleration ranges, and displacement vector sums, those for a bird will be significantly different from those for an airplane.
Similar analysis methods will be performed for things like the shape of the objects in view, their apparent size (the amount of screen they take up), the coloration and light patterns on the objects, and other data points. Eventually, flattening of the fish eye to provide linear kinematic data will also be performed. However, our initial set of data utilizes one standard camera model and type, making unwrapping unnecessary.
Secondary data sets such as known flight paths, weather conditions/patterns of interest, and satellite/meteor activity can also be used to refine down the set of possible cases which go into the “unknown” bucket.
Over time, other data sets can also be included, for example, noise patterns, radiation spikes, acceleration or displacement of the units themselves, and more sophisticated video/camera techniques. Again, the team assumes that these parameters, when taken together, will provide enough variance to allow for distinctions to be made by our system between classes of objects.
If the unit obtains the initial kinematic data about an object (normalized velocity/acceleration and displacement vectors), and if the event fails to fall within one of the available event buckets and cannot be initially categorized into that bucket, what happens next? In that case, it passes onto the next set of analysis checks and is again scrutinized by the machine learning algorithm based on the distributions the system obtains from all the available object data. Depending on the number of these checks failed, the event is given a rating, and those with the highest number of failed checks (the most unknown) are those of particular interest for this study.
At the end of the exploratory data analysis phase, additional cross-validation checks are conducted. Crowdsourcing the final validation on this system will allow us to train and re-train our machine learning systems, to fix any issues.
Analysis of the Unknowns
Given enough time and attention the Sky Hub system will, in theory, be capable of obtaining a robust set of both known and unknown events, which can then be studied by other groups interested in this subject. Besides just the videos themselves, these events will also have the data sets collected for analysis of the machine learning system and others that can be added later if desired. Although that is not the goal of the Sky Hub system (our goal is to create the machine itself to obtain this data), one can imagine that a similar silo and categorization strategy could be employed once information on unknown events is collected in a large enough quantity to be statistically relevant.
Assuming unknown objects visit our atmosphere, the analysis of these should be similar to the study of commercial and military aircraft currently known. For example, the shape, light patterns/colors, movement parameters, and radiation emitted by unknown objects has been a matter of significant interest for some time. By utilizing the same sort of data collection scheme used to create data sets on known objects, the unknowns can be sorted based on our taxonomic approach. For example, by comparing the displacement vectors of unknown objects based on their coloration, shape, or even geographical region in which they appear interesting commonalities may appear.
Although this would be an exciting extension of the Sky Hub system, it is not the only possible application of course. We hope that as we continue developing the system and its analysis structure, other researchers from various fields and disciplines will offer suggestions of applications we can attempt. It is our goal for this system to allow for significantly improved near-Earth surveillance of the skies.
