Monitoring boat traffic with public satellites

Michel Deudon

Follow

Published in

CARRE4

5 min readSep 10, 2020

--

⛵ Open sourcing S2Boatless 0.1.0

Satellite imagery of New-York City rivers, Venice lagoon and Istanbul strait. — **Left to right:** Pictures of New York City, Venice and Istanbul in 2019. RGB imagery from Sentinel-2, ESA.

With Zhichao Lin, we’re happy to open source some code, a dataset and a pretrained model to remove, detect or count boat traffic in satellite imagery 🛰️. Why you should care? What’s coming next? We’ll answer these questions in this post. Ready?

Setting sail for the Decade on Ocean Science

Initially, our work was motivated by the United Nations Decade on Ocean Science for Sustainable Development (2021–2030). A report from WWF published last year highlights how 19 out of 23 EU member states are falling behind their commitments to lifeguard Marine Protected Areas (MPA) in the context of the Aichi Targets set for 2020. While attending different workshops on Ocean Science, I heard the same thing coming again and again: “MPA and regulations are useful if we can enforce them, but the size of areas to monitor are beyond the scope of any regulatory body.” Put differently, “we can’t ask a cop to monitor every square kilometer of our oceans”. At that time, I was working on generating cloud-free, boat-free pictures of C40 Cities, Unesco World Heritage Centers, Marine Protected Areas, etc. My goal was to build a portfolio to raise awareness on the importance and beauty of Water and Life. As Captain Paul Watson said, “if the oceans die, we die”. It’s that simple.

Satellite imagery of a marine protected area with vegetation, beach, coral reef and deep water. — **Picture of a Marine Protected Area**. False negative color reveals vegetation and coral reefs in red. Contains modified Copernicus Sentinel-2 data. Credits: ESA ACT and VITO.

So when Europe’s lockdown arrived, when ports, straits and rivers were really free of boats and human activity, and when ESA announced a call for proposal to help Europe recover, I thought this is a unique opportunity to share my portfolio for a green and blue recovery 🌍. And guess what? By just sending two pictures highlighting boat traffic in Venice, and a slide deck called “Foreseeing the transportation modal shift”, I won a prize 🥇 and 1000€. Crazy, no? But that was just the start of something bigger. ESA offered me the opportunity to upscale my idea and contribute to a European dashboard for policy makers. That’s when I pinged Zhichao and said we had to do this together. I knew I couldn’t make it alone and this was the right thing to do.

Our final results and idea were presented to the European Commission by the European Space Agency. We reviewed the literature on boat detection from space, annotated 2000 images, trained a simple model, deployed it on a new platform for Earth Observation, analyzed two years of data in 21 areas in Europe, each covering 200 square km: Antwerp, Athens, Bosporus, Budapest, Corfu, Dardanelles, Dublin, Dover, Gibraltar, Grand Harbour, Helsinki, Kiel, Limassol, Messina, Oresund, Rotterdam, Sines, Split, St Tropez, Venice, Vigo. We open sourced S2Boatless ⛵ and reached an extra milestone together.

Background and boat density heatmaps in European ports and straits during Spring 2020 (covid-19 lockdown). — [**COVID-19] Lockdown in the E.U.** Left to Right: NIR band and predicted boat density maps near Antwerp, Budapest, Messina strait and Oresund bridge. Spring 2020. Contains modified Copernicus Sentinel-2 data processed by Euro Data Cube.

Less is better

There’s no need for a bazooka to get started, especially if you’re aiming peace on Earth ☮️. We conceived a simple solution by design to monitor boat traffic. Our model learns to detect boats as “hotspots on water” ☄️ and outputs a heatmap indicating where boats are found, up to a precision of 100 meters, using Sentinel-2’s near infra red band as well as a precomputed cloud free, boat free background image (the “trick” that makes the model learn well with few resources and supervision). In some situations, our model clearly fails and there’s room for improvement, but it’s light, fast and scales. Our model can scan one month of Sentinel-2 imagery collected over 400 square km (117 Central Parks) in a minute on a graphic card (GPU). The trained model itself is less than 50kB. That’s less than a picture taken from your smartphone.

Ships2K training examples. 1 square km chips annotated with boat counts. Weakly Supervised Learning. — **Ships2K training examples.** 1 square km chips were annotated with boat counts, using Superintendent by Jan Freyberg et al. Contains modified Copernicus Sentinel-2 data processed by Euro Data Cube.

What’s coming next?

Our next step is to move from the platform we used for European policy makers to free Earth Observation tools. In terms of science and ethics, we believe transparency shouldn’t be a luxury. We want to empower non profits, artists, citizens, people operating on the ground, raising public awareness or enforcing regulations aimed to reduce pollution, protect biodiversity and local communities.

Satellites are not going to stop smugglers but since some data is public and some MPA have no measures to safeguard life, monitoring boats from public imagery could be used to notify local authorities, non profits or regulatory agencies to help them be more efficient. As Captain Jacques Y. Cousteau said during a 1976 interview at NASA Headquarters, “everything that happens is demonstrating the need for space technology applied to the ocean”. Why don’t we use intelligence to regulate boat traffic, protect and conserve oceans?

Last words

A PhD student said last year at NASA FDL “we’re searching for life in space but we ignore what’s hidden in our oceans”. Covid-19 has only amplified the urge to protect them with drastic measures. Take a walk on a beach and you’ll see masks, gloves, plastic we wear to protect ourselves ending in the sea 🤿. Let’s end overfishing, dumping, smuggling which have not stopped with the lockdown. Let’s stop sinking the boat on which we live and act today 🛶.