BOB: The Universal Object Tracker
When I wrote my last Medium article way back in October 2020, I had recently joined the Sky Hub project. At the time, the world was in the midst of the pandemic and I found participating in Sky Hub a welcome diversion and an opportunity to meet, learn from, and collaborate with some cool people around the world on a technical project. The perfect hobby for me.
Sky Hub was born out of the wide public interest in Unidentified Anomalous Phenomena (UAP) ignited by a New York Times article and the release of videos by the Pentagon showing seemingly anomalous airborne objects captured by state of the art military platforms. The data released however was limited, short and fleeting and left open a lot of questions. The goal of Sky Hub therefore was to develop an open source edge solution using cameras and other sensors for scientific data collection on UAP.
Despite the Sky Hub team pouring effort into the project and making significant grounds on software development and hardware, there were some internal tensions caused by project scope creep and some misalignment on system vision that eventually led to its dissolution.
Although Sky Hub did not achieve its ambition, the endeavour lit a spark for a citizen science approach to data collection on UAP and some Sky Hub members went on to form and get involved with other projects like Sky360 and UAP@home.
What is BOB?
It’s four years on and a few of the Sky Hub members (myself included) and some members from Sky360 have released an early beta version of an open source software solution to detecting, tracking and recording objects in the sky — meet BOB: The Universal Object Tracker.
Firstly, BOB is not a clever acronym for anything. It’s simply the name the team assigned to the project at the start as a bit of a joke, but it stuck and the team kind of likes it. The camera character you see above grew out of it. An example of anthropomorphism I guess where technology takes on human characteristics (the team are all Johnny Five fans😉).
Secondly, the BOB team views this project as complementary to other ongoing efforts, such as those by UFODAP and Sky360, who each have their approaches. BOB’s mission is to provide a ready to use open-source solution that is accessible to individuals with limited hardware budgets. The system is designed to be as simple to operate as possible and provide a reliable means of data collection.
Software design
Early in development, the team decided to move away from a strictly edge-based solution using NVidia or PI devices and allow users to utilize their personal computers if they desired. Also the SBCs of the time were limited in processing capability.
The team wanted users to be able to operate with off-the-shelf security cameras, which are relatively accessible and affordable, but also enable those with larger budgets for more advanced hardware to still run BOB ensuring it is adaptable for a wide range of scenarios.
BOB is built on the Robot Operating System 2 (ROS 2) which is the perfect system for enabling all of this (the BOB team acknowledge Sky360 for identifying this — it’s great!). ROS 2 is platform-agnostic, allowing it to run on different operating systems, and is modular and scalable. The BOB team have developed algorithms and integrated existing open source ones as ROS 2 ‘packages’ and ‘nodes’ to address the detection, tracking and recording requirements. BOB’s source code is available here:
Other developers can join in and propose additional packages and nodes to the project to support other camera types or algorithms for processing data. The BOB team welcomes contributions through GitHub pull requests and issues, and of course, developers can fork the project and use it in their own way.
What does it do?
At the time of writing BOB is in early beta release. However, it is already quite advanced. BOB supports any cameras that uses the RTSP protocol. Once a camera is connected it will detect movement through background subtraction and blob detection processes, and hand off detections to a multi object SORT tracker to maintain track on objects in the sky.
A video recording of events will be made. The user can set a detection mask area within the camera field of view, set parameters and visualise data from a custom UI.
BOB also estimates environmental conditions and automatically tunes it’s sensitivity based on conditions. For example, in ‘stable’ weather conditions with clear sky or blanket cloud it will run at a higher sensitivity than in broken fast moving cloud. At higher sensitivity settings it will capture smaller (in pixel space) moving objects. Also it adapts the users detection mask at night to filter out stars to help avoid false detections as they move in and out of view behind cloud.
Along with any event video, it will output a JSON file with all of the detection points of the moving objects, and it will generate a image representation of the event - what the team call a BOB ‘heatmap’. In the following example heatmap you will see there are two birds that were tracked. This form of visualisation is helpful for understanding what the object is and the flight pattern it took over time. Any anomalous activity will hopefully stand out to users as they review their data through the UI. We have found that it is very easy to identify patterns from birds, bugs, planes etc using the heatmaps so are pleased with this novel contribution to data presentation in this domain.
The team have also integrated RStudio into BOB for those interested in pursuing more advanced data analysis and plotting than BOB provides through the heatmap or in it’s recording summary pages.
Roadmap
The BOB team have drafted a roadmap outlining the aspirations in developing this further.
The first step is performance. Whilst BOB’s detection and tracking algorithms are very nicely optimised to allow BOB to run at high resolutions and frame rates, some changes need to be made to maximise performance of this in combination with ROS. Currently BOB will skip the occasional frame due to known large data message passing issues through ROS DDS, but a solution to this has been identified and is currently being worked on.
After addressing that, the team plan to enable support for multiple cameras and hopefully a steerable PTZ camera to facilitate capturing a closer view of objects. Eventually the plan is to get on to using classification which will allow commonly recorded objects such as birds to be filtered out and ignored (unless bird recording is your thing), and bring forward ‘unknowns’ to the fore for analysis.
How to install BOB?
Currently BOB will run on Linux, and also on Windows 10 or 11 through Windows Subsystem for Linux (WSL). If you would like to give it a try you can install it from here bobcamera/bobinstall (github.com).
There are some guides on YouTube for install and running BOB e.g. BOB Install for Windows (youtube.com)
There is also a Wiki: Home · bobcamera/bobinstall Wiki (github.com)
Summary
From a personal point of view it’s been very satisfying to have met new friends, collaborated with likeminded people, as well as contribute ideas and write a bunch of code. Whist I’m stepping away from development work on this project, I will continue using the equipment I bought during the Sky Hub days as an active BOB user. I look forward to seeing how the team develop BOB further.
There truly are mysteries in our skies as evidenced by the ongoing release of official data and reports. Plus the wide range of accounts from credible witnesses, pilots and military personal etc. I’m sure BOB will help in shedding some light on matters and will assist those interested in engaging more actively in this topic from their own home. Hopefully BOB can be used for other scientific applications other than UAP spotting too.
Already after a couple of weeks out in the wild, beta BOB is providing some interesting data. Check out the following examples:
