How to prevent animal extinctions with drones and machine learning
In recent years there has been a growing awareness of the impact that our species is having on others. The various forms of habitat destruction and climate disturbances that we cause in the name of economic progress has a significant impact on the well-being of our fellow co-inhabitants of planet Earth. Indeed the number of extinctions in the last hundred years has risen immeasurably, such that even creatures that are so iconic to us (for example the White Rhino) are doomed to disappear. So let’s face the facts, our acts as human beings are killing animals en masse on an unprecedented scale. There it is, brutal as it may be.
How can we help prevent these extinctions? In order to raise awareness around these issues, the first step is to quantify just how much damage is being done to wildlife. Perhaps the best measure of this, is their population. To be clear, the population count doesn’t necessarily provide a direct count of how many animals are dying as there could be other reasons for a decreasing population in an area such as migrations or changes in behavioral patterns, both of which could also be a result of environmental shifts. It’s up to researchers to parse all this data and other relevant factors to create a clearer picture of what is going on.
By getting a better understanding of what the current state of a species is and how its population counts are changing over time, we can provide the context and information necessary to motivate a course of action that may prevent their extinction.
In this article we’d like to focus our attention on a particular use case by the University of Santa Cruz in California, their research in conducting more efficient wildlife surveys and how a technology such as Picterra can boost this efficiency to new heights. We really do think the Picterra platform can play a major role in expediting the process of conducting wildlife population surveys by orders of magnitude!
Save the “Pinnipeds”!
Let’s start by going into detail about our use case and relevant context. Wildlife surveys traditionally have been conducted by humans directly, either in some sort of vehicle or on foot, yet still by line of sight. This can often times be both inaccurate and slow and more importantly is more environmentally invasive. Researchers from UC Santa Cruz are hoping to change this norm by instead having drones fly over larger regions of inhabited areas, take pictures, and then count the populations off site, without disturbing animals in their natural habitats. UC Santa Cruz’s goal is to quantify the accuracy and time gains that can be achieved by using drone imagery. However, this strategy still involves manual counting over the images after they’ve been taken, which as we will see, may take a very long time.
In particular, the UC Santa Cruz is using seal and sea lion populations (collectively known as “pinnipeds”) at Año Nuevo island (part of the Año Nuevo Reserve, off the shore of California) to demonstrate and assess their approach. So, why pinnipeds? Long story short, global warming — > melting ice and water temperature changes — > destruction of habitats and shifts in ecosystem — > changes in pinniped behavior and population (a.k.a more dead seals/sea lions). There is a wide body of literature that studies the impact of climate change on seals. To list just a small handful [1], [2], [3], [4].
UC Santa Cruz has flown drones over Año Nuevo and aggregated about 50 orthomosaics corresponding to 50 different flights, each covering the entire island at different dates. The goal is to count populations of these animals and examine how they change over time on the island. Counting just a single orthomosaic by hand takes about half a day and as is such they’ve only done a handful of them. We approximated that counting across all of the available imagery would take more than a month (more on this later). With Picterra, we can make this whole task take a matter of hours! Let’s take a more in depth look into Picterra to see how.
Using the Picterra Platform
For the sake of demonstration let’s focus our efforts on 25 of UC Santa Cruz’s orthomosaics (that’s 50 days of manual counting work) taken over the year of 2018. We start by uploading these 25 orthos to the platform. You can do this in ‘My Images’ in the Dashboard, from which we’ll create a folder and upload our orthomosaics to this folder.
Creating your detector
Once they’ve been uploaded we can click on ‘My Detectors’ and ‘Create a new Custom Detector’ to start training our Seal & Sea Lion detector.
After clicking ‘Create’ you’ll see that the first step here is to add images that we want to use for training our detector from our image collection. Let’s start with just a few for now. We click ‘Add to Training’ for each of these images and then ‘Start Training’ to start the training detector mode.
Now in training detector mode, we can start providing some examples to Picterra’s AI, giving it examples of what to look for and what not to look for. The purpose of this article is not to be a tutorial on how to train a custom detector (for which we have a tutorial here) but to show you how your custom detector can be easily applied at scale on the platform. Thus we’ll just show you examples of a few of training areas and annotations we’ve created. This process, including training iteration of reviewing intermediate results and improving our annotations, took in total about 3 hours.
Running your detector at scale
Now how do we run our trained detector on the rest of our images? From our training detector mode we can click ‘Run Detector’ to go into ‘Detector Mode’ for the current detector, from which you will be shown your image collection. Entering the folder in which you uploaded your images you’ll see all your images with option to run the current detector on each image. Running it on these images is just as simple as clicking ‘Detect’ on each of your images. Note that in the future we plan on adding the ability to click ‘Detect’ directly on a folder to run detections for the entire folder contents at once (even less clicking!).
Your requests to detect will be queued up on our servers after which it’s just a matter of waiting. Detection across each image (30k by 20k pixels) takes on average 5 minutes so it will take about 2 hours to complete. You can check the status your detections on each raster in the ‘Status’ column.
Now to view all of your results, we can go to ‘My Images’ in the Dashboard and click on the ‘Results’ button next to your folder. This brings up the ‘Stats & Report mode’ and from here you can view all of your results across each of your images, download the results files and even generate a small PDF report on each image. Here’s what the results look like!
So how did we do in terms of accuracy and time?
Accuracy- As mentioned previously UC Santa Cruz did manually count a few of the orthomosaics so we do have some numbers to compare against. For the image taken on June 26th, 2018 UC Santa Cruz counted 3950 pinnipeds, though it’s likely that with manual counting some number of them were missed. Picterra counted 4201. This is an error margin of at most ~6%. It’s not perfect but for the sake of seeing a trend in the population it’s sufficient. Additionally with more iteration in training the detector (more annotations and training areas) the accuracy can always be improved.
Time- The whole process we outlined took 5 hours. 3 hours to annotate and train the model and 2 hours to run detections, which is just time spent waiting so you can just leave that running in the background and do something else anyways. Counting all the pinnipeds across one image took a researcher half a day of work, and that was actually on an image that had fewer than average. In total Picterra counted ~140k seals and sea lions across all 25 images and the image that was counted only had 4k. That’s 35 times more, which would bring the total time to count manually to 17.5 days. To summarize:
- 25 images
- ~140k seals and sea lions
- 17.5 days vs 5 hours
The final shared project results can be found here:
https://app.picterra.ch/reports/v1/a19c1fa8-ba9b-43cd-90e0-329b6f3da277/
Count analysis over 2018- As a point of intrigue here is a graph showing the changing populations on Año Nuevo island in 2018. UC Santa Cruz has plenty more image data on the island. If we performed this analysis the full image dataset and looked at it together with this statistical information on climate change for this time range perhaps we could uncover some helpful insights! But as I pointed out earlier it’s the job of the researchers to interpret these results, as I am no pinniped expert myself. However, already just from this data you can see what you might call the “high seasons” and “low seasons” for the island’s population.
Final words
Using and combination of drone surveying technology and Picterra we’ve turned 17 and a half days of work into 5 hours. Additionally, we can apply our trained detector at scale across the remaining 25 images for a different year without too much additional waiting time, perhaps going from 5 to 8 hours. So that’s roughly 35 days of work in 8 hours (1 work day) with only a small percentage drop in count accuracy! It’s quite clear from this that this is a much more effective approach than on-site wildlife surveying.
By turning the act of counting these animals, previously the most arduous and longest task of a wildlife survey, we can open up researchers’ time for much more important tasks than sitting in front of a screen trying to decide whether or not a grey blob in their image is a seal or a rock. 35 days is a pretty significant amount of time. How many more populations can be counted in that time? How much more research and analysis can be done? 35 less days spent counting is perhaps 35 less days before being able to share meaningful results with the community. That’s 35 less days before potentially inspiring some sort of positive action and change. And that’s just one lab at a university. How many more hundreds if not thousands of days can we save, and how much of an impact can that having on saving the biodiversity of the planet? These are not particularly easy questions to answer but at the very least, Picterra is here to help make them questions that we have the luxury of asking at all!
Thanks for reading! Picterra out!
