Detecting the Unseen: Anomaly Detection with GANs

An ironic depiction of adversarial training of two neural networks (Yes, this image was generated by generative AI on Canva!)

Anomaly detection is a significant problem faced in several research areas, including finance, healthcare, and cybersecurity. Detecting and correctly classifying something unseen as anomalous is a challenging problem that has been tackled in many different manners over the years. One of the most promising approaches to anomaly detection is through the use of generative adversarial networks (GANs). GANs are a type of deep learning model that can learn to generate realistic data samples that are similar to a given dataset. This characteristic of GANs suggests that they can be used successfully for anomaly detection, although their application has been only recently explored.

What are GANs?

Generative Adversarial Networks (GANs) are a class of artificial intelligence algorithms used in unsupervised machine learning. They were introduced by Ian Goodfellow and his colleagues in 2014. GANs consist of two neural networks, the generator, and the discriminator, which are trained simultaneously through adversarial training.

1. Generator: This part of the GAN is responsible for generating new data instances. It takes random noise as input and transforms it into data that ideally is indistinguishable from real data.
2. Discriminator: This part of the GAN acts as a classifier. It is trained to distinguish between real data and the synthetic data produced by the generator.

The training process involves a continuous back-and-forth between these two networks:

• Generator’s Role: The generator aims to produce synthetic data that is so convincing that the discriminator cannot differentiate between real and generated data.
• Discriminator’s Role: The discriminator is simultaneously trained to become more adept at distinguishing between real and generated data.

The objective is for the generator to create data that is increasingly realistic, while the discriminator becomes more skilled at telling the difference. This adversarial process continues until the generator produces data that is essentially indistinguishable from real data.

The equilibrium point, where the generator produces highly realistic data and the discriminator cannot reliably tell it apart from real data, represents the successful training of the GAN.

How can GANs be used for anomaly detection?

A sophisticated android robot inspects mechanical parts as they roll off an assembly line for defects (generative AI again!)

Generative Adversarial Networks (GANs) can be leveraged for anomaly detection by training them to generate normal or typical data distributions. The basic idea is to use the generator component of the GAN to learn the underlying patterns of normal data. Once trained, the generator can be used to produce synthetic data samples that should resemble the normal distribution.

Here’s a step-by-step overview of how GANs can be applied for anomaly detection:

1. Training on Normal Data:

• Train the GAN using a dataset containing only normal or typical instances of the data you want to analyze (e.g., normal images, normal sensors readings, etc.).
• The generator learns to produce synthetic samples that mimic the distribution of normal data, while the discriminator is trained to distinguish between real and synthetic data.

2. Generation of Synthetic Data:

• After training, use the trained generator to produce a set of synthetic data samples. These synthetic samples should ideally resemble the normal instances present in the training data.

3. Anomaly Detection:

• Combine the synthetic data generated by the GAN with the original normal data.
• Use a traditional anomaly detection technique or a simple thresholding method to identify instances that deviate significantly from the expected distribution.
• Instances that are dissimilar to both the real and synthetic data are considered potential anomalies.

4. Discriminator as Anomaly Detector:

• The discriminator of the GAN can also be repurposed as an anomaly detector. Instead of using it solely during the training phase, apply it to both real and synthetic data during the anomaly detection phase.
• Instances that the discriminator classifies as more likely to be real may be considered normal, while those it classifies as more likely to be synthetic may be flagged as potential anomalies.

Recent research in anomaly detection with GANs

A recent study (1) presents a review of the use of GANs in anomaly detection. The authors discuss the advantages and limitations of GANs in anomaly detection and provide an overview of the recent research in this area. They also identify the challenges and future directions for research in this field.

Another survey (2) focuses on the application of GANs in anomaly detection for medical time series and presents promising results.

A recent paper (3) proposes a detection model that combines the USAD generative adversarial training architecture and convolutional autoencoders (CAE) to enhance stability during adversarial training by generating the distribution of normal data for adversarial training and improving the ability to extract features from the data. The authors demonstrate that their proposed model outperforms existing methods on several benchmark datasets.

Another recent study (4) applies GANs to detect anomalies in power generation plants. The authors use different augmentation techniques such as autoencoders and principal component analysis to improve the performance of GANs in various aspects of anomaly detection.

A recent paper (5) proposes a novel module of an encoder-decoder GAN based on attention feature fusion for anomaly detection of GAN industrial images. The authors demonstrate that their proposed method outperforms existing methods on several benchmark datasets.

Finally, a recent study (6) investigates the use of GANs for anomaly detection in biomedical imaging. The authors present an overview of using GANs for anomaly detection and investigate state-of-the-art GAN-based anomaly detection methods for biomedical imaging. They demonstrate that GAN-based methods outperform traditional methods on several benchmark datasets.

Let’s code it!

Building a complete Generative Adversarial Network (GAN) involves several components, including defining the generator and discriminator architectures, specifying loss functions, and setting up the training loop. Below is a simplified example using TensorFlow and Keras to create a small GAN for generating synthetic data.

import tensorflow as tf
from tensorflow.keras import layers
import numpy as np
import matplotlib.pyplot as plt

# Define the generator model
def build_generator(latent_dim):
    model = tf.keras.Sequential()
    model.add(layers.Dense(256, input_dim=latent_dim, activation='relu'))
    model.add(layers.BatchNormalization())
    model.add(layers.Dense(512, activation='relu'))
    model.add(layers.BatchNormalization())
    model.add(layers.Dense(784, activation='sigmoid'))
    model.add(layers.Reshape((28, 28, 1)))
    return model

# Define the discriminator model
def build_discriminator(img_shape):
    model = tf.keras.Sequential()
    model.add(layers.Flatten(input_shape=img_shape))
    model.add(layers.Dense(512, activation='relu'))
    model.add(layers.Dense(256, activation='relu'))
    model.add(layers.Dense(1, activation='sigmoid'))
    return model

# Define the GAN model
def build_gan(generator, discriminator):
    discriminator.trainable = False  # Freeze discriminator during GAN training
    model = tf.keras.Sequential()
    model.add(generator)
    model.add(discriminator)
    return model

# Function to compile models
def compile_models(generator, discriminator, gan, latent_dim):
    discriminator.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0002, beta_1=0.5),
                          loss='binary_crossentropy',
                          metrics=['accuracy'])

    gan.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0002, beta_1=0.5),
                loss='binary_crossentropy')

# Function to generate random noise for the generator
def generate_latent_points(latent_dim, batch_size):
    return np.random.normal(0, 1, size=(batch_size, latent_dim))

# Function to train the GAN
def train_gan(generator, discriminator, gan, dataset, latent_dim, epochs, batch_size):
    batch_per_epoch = dataset.shape[0] // batch_size

    for epoch in range(epochs):
        for batch in range(batch_per_epoch):
            noise = generate_latent_points(latent_dim, batch_size)
            generated_data = generator.predict(noise)

            real_data = dataset[np.random.randint(0, dataset.shape[0], batch_size)]
            labels_real = np.ones((batch_size, 1))
            labels_fake = np.zeros((batch_size, 1))

            d_loss_real = discriminator.train_on_batch(real_data, labels_real)
            d_loss_fake = discriminator.train_on_batch(generated_data, labels_fake)

            d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)

            noise = generate_latent_points(latent_dim, batch_size)
            labels_gan = np.ones((batch_size, 1))

            g_loss = gan.train_on_batch(noise, labels_gan)

            print(f"Epoch {epoch + 1}/{epochs}, Batch {batch}/{batch_per_epoch}, D Loss: {d_loss[0]}, G Loss: {g_loss}")

# Function to generate and plot synthetic data
def generate_and_plot(generator, latent_dim, examples=10):
    noise = generate_latent_points(latent_dim, examples)
    generated_data = generator.predict(noise)

    for i in range(examples):
        plt.subplot(2, 5, i+1)
        plt.imshow(generated_data[i, :, :, 0], cmap='gray_r')
        plt.axis('off')

    plt.show()

# Example usage
latent_dim = 100
img_shape = (28, 28, 1)

# Build and compile the models
generator = build_generator(latent_dim)
discriminator = build_discriminator(img_shape)
gan = build_gan(generator, discriminator)
compile_models(generator, discriminator, gan, latent_dim)

# Load and preprocess your dataset (e.g., MNIST)
(train_images, _), (_, _) = tf.keras.datasets.mnist.load_data()
train_images = train_images / 127.5 - 1.0  # Normalize images to the range [-1, 1]
train_images = np.expand_dims(train_images, axis=-1)

# Train the GAN
train_gan(generator, discriminator, gan, train_images, latent_dim, epochs=100, batch_size=64)

# Generate and plot synthetic data
generate_and_plot(generator, latent_dim)

This code provides a basic structure for a GAN using TensorFlow and Keras. It uses the MNIST dataset as an example. As you see, building a GAN architecture in Tensorflow is pretty straightforward, but keep in mind that the adversarial training phase is a bit tricky and must be performed carefully.

Limitations of using GANs for anomaly detection

It’s important to note that GANs for anomaly detection have some challenges and considerations:

• Balance: Ensuring a good balance between the generator and discriminator is crucial. If the generator is too weak, it might not accurately capture the normal data distribution. If it’s too strong, it might fail to generate diverse synthetic samples.
• Evaluation: The effectiveness of the anomaly detection approach should be evaluated rigorously using appropriate metrics and validation datasets.
• Data Quality: The success of GANs in anomaly detection depends on the quality and representativeness of the training data.

To conclude, applying GANs for anomaly detection can be a powerful approach, especially in cases where labeled anomalous data is scarce, as GANs can learn to represent the normal data distribution without explicit labels for anomalies. The training phase must be conducted carefully and training data quality is crucial for the result, but in the end the GAN approach has the potential to outperform traditional methods for anomaly detection on all kinds of data.