Intro
The aim of the project is to classify satellite images that contain ships and those that do not and furthermore detect the ships on a pixel level in relation to the image as a whole.
Ship detection is a vital aspect of maritime surveillance as it allows the monitoring of maritime traffic, illegal fishing and sea border activities. This is typically carried out using Automated Identification Systems (AIS) which use VHF radio frequencies to wirelessly broadcast the ships location, destination and identity to nearby receiver devices on other ships and land-based systems.
AIS are very effective at monitoring ships which are legally required to install a VHF transponder, but fail to detect those which are not, and those which disconnect their transponder. So how do you detect these ‘dark’ ships? Using satellite detection!
Let’s dive into it
The use of satellite detection of ships will significantly bolster the arsenal of stakeholders in maritime surveillance. Used in conjunction with AIS, satellite detection can flag previously unidentified ships and build up a more accurate picture of the world’s seas and oceans that are becoming ever more crowded. Some key challenges that this form of surveillance will attempt to tackle are: illegal fishing operations, maritime accident identification and illegal border immigration. In the last decade, numerous companies have deployed surveillance satellites in an attempt to bolster their monitoring capabilities. In turn, detection algorithms such as the one I propose will offer an automated approach to analysing the imagery fed back.
Part I
Identifying whether or not there are ships in a satellite image is the first piece of the puzzle and I tackled this with an image classification approach. The data I used was in JSON format so I began by importing and reading the images and their labels. Below are some examples of the images I was working with.
After several preprocessing steps, I ran an initial baseline model using a decision tree classifier which performed at 86% accuracy on my validation set. I tuned my baseline, adjusting hyperparameters and then ran several further models using random forest classifiers as well as a binary logistic regression. These initial models set a benchmark for the commonly used convolutional neural network (CNN) to beat. Using CNNs, I was able to correctly identify images containing ships 98% of the time on my test set. My final CNN model for the first stage consisted of four convolutional and pooling layers along with two fully-connected, dense layers. I also included five dropout layers to decrease the model’s complexity and in turn reduce overfitting. It performed very strongly on the training and test data — both 98%.
Some important metrics taken from my final model included a recall of 98%. Recall is defined as the model’s ability to accurately classify instances of the ship class.
Accuracy Score 0.97875
ROC_score 0.9926974797399591
Precision 0.9444444444444444
Recall or TPR 0.9760765550239234
F1 score 0.96
Sensitivity 0.9796954314720813The data for the first part was taken from Planet’s Open California dataset (https://www.kaggle.com/rhammell/ships-in-satellite-imagery).
Part II
Next, the objective was to detect and delineate the ships from the image background to identify the ships more accurately, on a pixel-level. The dataset I was using contained satellite images with some containing one or multiple ships and some without. All images were also described in a CSV file with the images containing ships having a corresponding encoded mask next to them. When decoded, the mask delineates the ship from the rest of the image and this is what we want our model to perform.
The method I used employed for this task is called Mask R-CNN. It is a variation of R-CNN (region-based CNN) and is used to solve instance segmentation problems. There are two main stages of Mask R-CNN. First, it generates regions of interest where there might be an object based on the input image. Next it predicts the class of the object and generates a mask in pixel level of the object based on the initial proposal.
I have used a transfer learning approach for this part of the project and started by forking Matterport’s repository which outlines a Mask R-CNN approach using Resnet 101 and pre-trained weights using Microsoft’s Coco dataset (a commonly used object detection dataset). Starting off using this architecture would save me time and give better results.
The model is still currently training as there are over 15k training images! I shall update when it is complete.
With the models I have proposed, maritime surveillance can now be carried out using a two pronged approach through the use of traditional AIS tracking as well as satellite detection.
In future, I would like to continue the project using geographical features to be able to simultaneously plot the position of each ship on a map to assist with tracking. Additionally, with access to AIS surveillance data, I would be able to build a model that identifies ‘dark’ ships — those that are picked up using satellite detection but not using AIS. This is how I see my models being practically implemented in the real world. For example in combating illegal fishing, marine police will have access to a map showing all ships located using AIS and overlain, a satellite image of all ships in the particular area then revealing the ‘dark ships’.