How to download images and bounding boxes from FathomNet using Python
FathomNet provides fathomnet-py, a Python client-side API to help scientists, researchers, and developers interact with FathomNet data. In this article, we’ll step through code demonstrating a common use case: downloading images and bounding boxes.
Setup
If you want to follow along, you’ll first need to install the fathomnet Python package via pip. You’ll need Python ≥3.7.
pip install fathomnetComplete documentation of the fathomnet-py project is available at fathomnet-py.readthedocs.io, and the code is available at github.com/fathomnet/fathomnet-py.
1. Import the modules
First things first, we need to import the modules we want to use from fathomnet. The API modules live in the fathomnet.api subpackage, so we’ll import from there:
>>> from fathomnet.api import images, boundingboxesNow, for example, if we want to get a count of all bounding boxes:
>>> boundingboxes.count_all()
Count(objectType='BoundingBox', count=134906)2. Pick a concept
A concept is simply a term used to describe something seen in an image. It could be a species name, a description of a substrate, a piece of equipment, etc.
For the purpose of this demonstration, we’ll just pick a random concept from FathomNet to focus on. We can grab a list of all concepts used in FathomNet:
>>> all_concepts = boundingboxes.find_concepts()and pick a random one:
>>> import random
>>> random_concept = random.choice(all_concepts)
>>> random_concept
'Paragorgia arborea'Great! We’ve now randomly picked a concept (in this case, a coral: Paragorgia arborea) that has some existing images and bounding boxes in FathomNet.
3. Get a list of images for that concept
Now that we know what we want, let’s ask FathomNet for a list of images that have a bounding box of Paragorgia arborea in them. We’ll call the images.find_by_concept function and check how many images we’re working with:
>>> random_concept_images = images.find_by_concept(random_concept)
>>> len(random_concept_images)
1174So FathomNet has records for 1174 images of Paragorgia arborea, which are now stored in our random_concept_images list. Let’s check one out.
>>> random_concept_images[746]AImageDTO(
id=2430897,
uuid='adee5c17-cf2b-426b-a2fe-7f40021c854c',
url='https://fathomnet.org/static/m3/framegrabs/Doc%20Ricketts/images/0850/04_03_26_00.png',
valid=True,
imagingType='ROV',
depthMeters=902.0,
height=1080,
lastValidation='2021-10-01T07:50:25.267686Z',
latitude=36.32948,
longitude=-122.312989,
salinity=34.4370002746582,
temperatureCelsius=4.065000057220459,
oxygenMlL=0.31200000643730164,
pressureDbar=911.4000244140625,
sha256='270bf3f6d7a1d855ad0fb5b03e966477e24cfce90097cc4f4460f84733e45faa',
contributorsEmail='brian@mbari.org',
tags=[
ATagDTO(
uuid='32290ba4-3f01-4ca7-aa99-9afd73bad1b8',
key='source',
mediaType='text/plain',
value='MBARI/VARS'
)
],
timestamp='2016-06-02T18:10:47Z',
width=1920,
boundingBoxes=[
ABoundingBoxDTO(
uuid='d425d035-4892-460b-bb2b-38b71813ca45',
userDefinedKey='d39d288f-357d-438e-d76c-e5e0a9c4aa1e',
concept='Paragorgia arborea',
height=671,
occluded=None,
observer='lonny',
width=654,
x=834,
y=133,
rejected=False,
verified=False
),
ABoundingBoxDTO(
uuid='a110bdbb-a5cf-4242-b1ff-e0465d9de23c',
userDefinedKey='71657a55-7f48-4299-7b63-fee0a9c4aa1e',
concept='Paragorgia arborea',
height=831,
observer='lonny',
width=439,
x=392,
y=40,
rejected=False,
verified=False
),
ABoundingBoxDTO(
uuid='0adac0ed-ad1f-4734-8f0a-859db27d60bd',
userDefinedKey='803d87c7-bd30-484e-146f-68dfa9c4aa1e',
concept='Paragorgia arborea',
height=341,
observer='linda',
width=274,
x=939,
y=637,
rejected=False,
verified=False
)
],
createdTimestamp='2021-09-29T21:23:44.533Z',
lastUpdatedTimestamp='2021-10-01T07:50:25.277Z'
)
Here we can see all the data associated with that image, including the list of bounding boxes. At this point, we can download the images and munge the bounding boxes into our favorite format. But first, let’s follow the image URL and take a look!
Aside: Advanced constraining
In the example above, we constrained our search by the concept alone by using the images.find_by_concept function. There are a few other ways to search for images (see the fathomnet-py documentation) as well as a generalized way to specify search constraints.
Let’s say we want to find images of Vampyroteuthis infernalis at depths between 600 and 800 meters. To do this, we set up a constraint object:
>>> from fathomnet.models import GeoImageConstraints
>>> constraints = GeoImageConstraints(
... concept='Vampyroteuthis infernalis',
... minDepth=600.0,
... maxDepth=800.0
... )and then call the general images.find function:
>>> vampire_squid_constrained_images = images.find(constraints)Aside: Using the taxonomic tree
Often times, we want to gather images and bounding boxes for multiple concepts by their taxonomic association. The taxa module provides functions to perform taxonomic look-ups to inform this process.
To use the functions in this module, we’ll need to specify a taxonomy provider. We can get a list of FathomNet’s available providers with taxa.list_taxa_providers:
>>> from fathomnet.api import taxa
>>> taxa.list_taxa_providers()
['mbari', 'worms']A note on taxa providers: mbari is mostly specific to California, but it’s fast and robust. worms is global but, due to the large number of available names, should only be used to find taxa at or below the genus level.
For example, let’s say we want images of Bathochordaeus and all its descendant concepts (according to the mbari taxonomy provider). We’ll call the taxa.find_taxa function:
>>> all_batho_taxa = taxa.find_taxa('mbari', 'Bathochordaeus')This gives us a list of Taxa objects:
[
Taxa(name='Bathochordaeus', rank='genus'),
Taxa(name='Bathochordaeus charon', rank='species'),
Taxa(name='Bathochordaeus mcnutti', rank='species'),
Taxa(name='Bathochordaeus stygius', rank='species')
]We can then loop over those taxa and query all their images:
>>> all_batho_images = []
>>> for batho_taxa in all_batho_taxa:
... batho_images = images.find_by_concept(batho_taxa.name)
... all_batho_images.extend(batho_images)Alternatively, we can do the same search by setting a taxaProviderName in the technique described in “Aside: Advanced constraining”. The search will use that taxa provider to lookup the descendant taxa and automatically include them in the search results.
>>> from fathomnet.models import GeoImageConstraints
>>> constraints = GeoImageConstraints(
... concept='Bathochordaeus',
... taxaProviderName='mbari'
... )
>>> all_batho_images = images.find(constraints)For more granular control of the concepts to use, we can walk up or down the taxonomic tree with the taxa.find_children and taxa.find_parent functions:
>>> taxa.find_children('mbari', 'Bathochordaeus')
[
Taxa(name='Bathochordaeus stygius', rank='species'),
Taxa(name='Bathochordaeus charon', rank='species'),
Taxa(name='Bathochordaeus mcnutti', rank='species')
]>>> taxa.find_parent('mbari', 'Bathochordaeus')
Taxa(name='Bathochordaeinae', rank='subfamily')
4. Download the images and generate annotations
This section will walk through downloading the images and generating annotation files for the bounding boxes. There are a multitude of image annotation formats out there, but for the purpose of demonstration we’ll pick a common, human-readable one: Pascal VOC.
Install pascal-voc-writer
We will make use of the pascal-voc-writer package to help format the bounding boxes, so let’s install it:
pip install pascal-voc-writerCreate a target directory
To get the images themselves, we’ll extract the URL from each image record and download the image via HTTP. We need a directory to store the downloaded images and generated annotation files. Let’s make one named after the selected concept (in this case, Paragorgia arborea/):
>>> import os
>>> data_directory = random_concept
>>> os.makedirs(data_directory, exist_ok=True)The plan is to loop through each image record, download its corresponding image, and write out a Pascal VOC annotation XML file. We’ll end up with a file structure like:
Paragorgia arborea/
1d763849-5f8a-4cef-bbbc-c12968497abe.png
1d763849-5f8a-4cef-bbbc-c12968497abe.xml
eba151e3-35bf-4bb9-8bf4-9482dd2178e5.png
eba151e3-35bf-4bb9-8bf4-9482dd2178e5.xml
94b5660e-74d9-4f05-8bb3-1a6d782ff1e5.png
94b5660e-74d9-4f05-8bb3-1a6d782ff1e5.xml
5dc67d66-5fa0-425f-893c-f3543b1847e3.png
5dc67d66-5fa0-425f-893c-f3543b1847e3.xml
3a65854c-59b7-4d73-af49-d6a59bc08c24.png
3a65854c-59b7-4d73-af49-d6a59bc08c24.xml
...Run it
Now, let’s make it happen. We’ll define a function to download an image:
>>> from urllib.request import urlretrieve
>>> def download_image(image_record):
... url = image_record.url # Extract the URL
... extension = os.path.splitext(url)[-1]
... uuid = image_record.uuid
... image_filename = os.path.join(data_directory,
... image_record.uuid + extension)
... urlretrieve(url, image_filename) # Download the image
... return image_filenameand a function to write an annotation:
>>> from pascal_voc_writer import Writer
>>> def write_annotation(image_record, image_filename):
... writer = Writer(image_filename,
... image_record.width,
... image_record.height,
... database='FathomNet')
... for box in image_record.boundingBoxes:
... concept = box.concept
... if box.altConcept is not None:
... concept += ' ' + box.altConcept
... writer.addObject(box.concept,
... box.x,
... box.y,
... box.x + box.width,
... box.y + box.height)
... xml_filename = os.path.join(data_directory,
... image_record.uuid + '.xml')
... writer.save(xml_filename) # Write the annotationNote: In this example, we’re renaming the file to be its image UUID (universally unique identifier) so that we won’t have filename conflicts.
Tying those two functions together, we can run the process:
>>> for image_record in random_concept_images:
... image_filename = download_image(image_record)
... write_annotation(image_record, image_filename)Bada-bing bada-boom.
We have now downloaded images and bounding boxes from FathomNet. Awesome! To verify, we can check out a generated annotation file:
<annotation>
<folder>Paragorgia arborea</folder>
<filename>1d763849-5f8a-4cef-bbbc-c12968497abe.png</filename>
<path>/data/Paragorgia arborea/1d763849-5f8a-4cef-bbbc-c12968497abe.png</path>
<source>
<database>FathomNet</database>
</source>
<size>
<width>1920</width>
<height>1080</height>
<depth>3</depth>
</size>
<segmented>0</segmented>
<object>
<name>Paragorgia arborea</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<bndbox>
<xmin>259</xmin>
<ymin>242</ymin>
<xmax>191</xmax>
<ymax>125</ymax>
</bndbox>
</object>
</annotation>Thanks for reading!
If you want to use this code, here’s a gist with the snippets above in script form.
