MarsiAn-omaly Detection Through Deep Adversarial Training
“Through every rift of discovery, some seeming anomaly drops out of the darkness, and falls, as a golden link into the great chain of order” — Edwin Hubbel Chapin.
Since I heard about A.I. and machine learning I was fascinated with it. Apart from more computational power, more data, and a better understanding of algorithms, the existence of communities continues to facilitate learning and collaboration while passionately solving problems with a high impact.
After a very fast and intensive introduction to machine learning (ML) last year and learning that it can be extremely beneficial, very complex, controversial and all over the place once you get it, it made me ask myself,
‘What’s next?’
For me, that was being part of an Omdena AI Challenge to detect anomalies on Mars surface. Here is what I built.
A Deep Convolutional Catch Me If You Can Network
For those who don’t know what a generative adversarial network (GAN) is (I didn’t only a couple of months ago), just think in the conclusion of the movie “Catch me if you can”.
It is an expert system that evolves from impostor (or fake) to an expert security consultant, which we will call Frank (as the main character in the movie who advised security companies after having faked his identity in dozens of cases). Instead of fake identities, our expert system is capable of classifying images with an anomaly score, based on the normal data distribution of the mars surface.
What is an anomaly?
An anomaly here refers to any kind of technological signature detected on the surface of Mars. A techno-signature is any measurable property or effect that provides scientific evidence of past or present technology. On the Mars surface, we have focused on visible physiological properties which help us determine if the object is part of the training set or not (anomaly).
Tecno-signatures include landing sites, parachutes, helmets and any other kind of equipment left during any of the missions.
Tensorflow meets GANs
I found AnoGAN’s implementation on TensorFlow and gave it a try with a labeled dataset I found at Zenodo.org containing a total of 73,031 landmarks. For each original landmark, they cropped a square bounding box that includes the full extent of the landmark plus a 30-pixel margin to left, right, top and bottom. Each cropped landmark was resized to 227x227 pixels and then was augmented to generate 6 additional landmarks. The dataset is labeled in 7 classes: crater, dark dune, slope streak, bright dune, impact ejecta, swiss cheese, spider and other [1].
AnoGAN is a deep convolutional generative adversarial network that could be represented and explained in a very simple way:
The generator G learns a distribution over normal data (without techno-signatures) via a mapping of samples, which are 1D vectors of uniformly distributed input noise sampled from the latent space to 2D images in the image space, which is populated by healthy examples. In AnoGAN the network architecture of the generator G is equivalent to a convolutional decoder that utilizes a stack of stridden convolutions. The discriminator D is a standard CNN that maps a 2D image to a single scalar value D(·). The discriminator output D(·) can be interpreted as the probability that the given input to the discriminator D was a real image x sampled from training data X or generated G(z) by the generator G. We can consider in our case Real=Normal and Fake=Anomaly. The D and G are simultaneously optimized through a minimax game [3].
After training for 50 epochs only with the craters class (4.900 images), I appreciated the learning capabilities of the generator.
Image 2. GIF made from generator’s output
I shared my results a few days after Murli Sivashanmugam, another Challenge Collaborator, had already tried an implementation of AnoGAN on Keras. Using my findings, after not more than a week Murli’s model had given the following results:
tn: 13, fp: 37, fn: 3, tp: 47
acc: 60%, prec: 55%, recall: 94%
“The results show a high number of true and false positives. The model is doing a good job identifying anomalies that are on the side and barely visible but is identifying abnormal natural surfaces also as anomalies” Murli Sivashanmugam.
Not satisfied yet, the collaboration continues.
We decided that one way of improvement was expanding the dataset, working with images on a higher resolution and with image enhancement and pre-processing steps that could help with the feature extraction and improve the performance.
Other participants of the challenge had already been working on a dataset API scrapping images on a high resolution from a HiRISE database, which could work better to train discriminators and other ML methods.
If you want to know more about the API please read this amazing article written by Samir Sheriff:
One of the particularities of the HiRISE imagery was that some of the images were rotated and contained a black margin. This was a problem because after cropping it will result in some regions being mostly black and we didn’t want to include those in our training dataset for obvious reasons, reducing its size. So, I decided to give it a try.
Apparently, the black margin removal solution was trivial with cv2.findContours and cv2.boundingRect function. crop_black_margin(img, show_contour = False) can be found in tools.py [6].
The function I wrote was working fine with an already straightened image. But straightening an image was something that required much more of attention.
Initially, I tried with a function which calculates the angle of rotation of an image and then corrects it. align_image(img) can be found in tools.py [6].
But this function was just working for discrete angles. Angles of rotation like 27,5 were incorrectly and the output was different:
Homography: a way to image perspective transformation
A homography is a perspective transformation of a plane, that is, a reprojection of a plane from one camera into a different camera view, subject to change in the translation (position) and rotation (orientation) of the camera.
So if we could have a template of an aligned image of the surface of Mars, in theory, we could use it to re-project the rotated images into it, fixing our problem for any kind of rotation and translation. It could even work for other kinds of transformations.
PyStackReg is an opensource library used to align (register) one or more images to a common reference image. It provides the following four types of distortion [8]:
· Translation
· Rigid body (translation + rotation)
· Scaled rotation (translation + rotation + scaling)
· Affine (translation + rotation + scaling + shearing)
· Bilinear (non-linear transformation; does not preserve straight lines)
I created a simple template with was an image with the same shape of the target (the image to be registered) and with a black margin at 10% of borders, creating a contour filled with noise, with was aligned with 0 in the x-axis.
All these functions are condensed in tools.py [6] with some other enhancement and augmentation functions to help with feature extraction, image segmentation and to expand the dataset. Some of them are:
Continuing with the enhancement we cross-chatted with teams in previous Omdena Challenges trying to solve the same problem.
Thanks to the article from James Tan, we could add up Deep Image Prior and Decrappify as great tools for our purpose.
This highlights the importance of sharing result between projects and how the network of resources and support grows in the community and extrapolate beyond each individual project.
My journey with Omdena
Working as part of Omdena is working on A.I projects aside from anything else you do daily, contributing with cool projects, which will give you outside academic experience, fast-pace learning and networking while solving a high impact problem.
This is an amazing way to learn, sharp your skills, share tools and merge ideas from different perspectives, backgrounds and time schedules. It is great to see how everything takes shape via remote collaboration, proactivity and initiative of a fascinating group of people. I truly feel it’s possible to tackle any kind of problem this way. I am thankful and glad to be part of Omdena.
Conclusion
The predictive performance of purely unsupervised anomaly detection often fails to match the required detection rates in many tasks and there exists a need for labeled data to guide the model generation. Classical semi-supervised approaches, originating from a supervised classifier, are inappropriate and hardly detect new and unknown anomalies. A semi-supervised anomaly detection needs to ground on the unsupervised learning paradigm and devise a novel algorithm that meets this requirement.
The work is not done yet. It is possible to obtain a higher accuracy for anomaly detection via generative adversarial networks. We need to work on improving and expanding our training dataset and include supervised feedback to support the learning.
Collaborating is about caring and being active, coordinating and taking the initiative to work in solving any of the building blocks of the solution. You get to know how competent you are and how people are willing to help you sharpen your skills.
“If everyone is moving forward together then success takes care of itself” — Henry ford
If you want to receive updates on our AI Challenges, get expert interviews, and practical tips to boost your AI skills, subscribe to our monthly newsletter.
We are also on LinkedIn, Facebook, and Twitter.
References
[1] https://zenodo.org/record/2538136#.XTHh3OgzbIV
[2] https://www.akiraanalytics.com/ai-insights/what-are-generative-adversarial-networks/
[3] Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery, https://arxiv.org/pdf/1703.05921.pdf
[4] https://colab.research.google.com/drive/1xku7bNUGMNoBcFy6_7fHdY3hG8gY7qrs#scrollTo=HhIzmfdfH7_T
[7] https://docs.opencv.org/3.4.1/d9/dab/tutorial_homography.html
