GANs and the Future of Synthetic Data
It was 2014 when Ian Goodfellow forever changed the world of AI when by publishing Generative Adversarial Nets. These networks were conceptually quite simple. A GAN is made up of two different neural networks. One network is trained to generate images while the other tries to discern which images are generated and which are real. As you train both networks they progressively get better at their jobs. Eventually, you end up with a generator capable of creating images almost indistinguishable from reality.
While the field of image generation has burgeoned since 2014 GANs have largely fallen to the wayside in the public eye. Nowadays the most popular forms of image generation take an input as text and spit out an image. To the average person, the ability to produce an image of an astronaut riding a horse is much more impressive than being able to produce a hyper-realistic face. Modern text-to-image models based on a diffusion architecture provide more freedom of choice and a wider array of outputs than GANs. Despite this, the hyper specificity of GANs may be their greatest benefit.
In the modern day, image detection models like CNNs (Convolutional Neural Networks) have drastically improved the ability of the average person to classify sets of images. While previously one might need to go through years of medical school to find the difference between a malignant and benign tumor, nowadays all you need is a dataset and a decent enough understanding of how to train a neural network. Even this latter expertise can be simplified drastically by services that will take care of the “hard stuff” for you. The primary issue with training a CNN is that they require a decently sized dataset to be trained properly. This issue becomes quite pronounced when training a CNN to classify images when there aren’t many training examples. You might run into this issue while training a classifier for an extremely rare disease, as you would have too little data. Conventional data augmentation techniques can only take us so far so eventually we might end up needing to wait for new cases of the disease to arise. As society begins to rely on AI for classification tasks we will begin to run into issues like this more often. When we do we can begin to rely on GANs as a synthetic solution.
In 2018 Maayan Frid-Adar and her team published GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. This paper highlighted issues such as the valuable medical resources that must be diverted to create useful datasets. The authors thought to remedy this using GANs to generate data in a more efficient way. The focus of this study was liver lesions, specifically on cysts, metastases, and hemangiomas. In a significant departure from Generative Adversarial Nets the research group used CNNs as the discriminator and generator rather than MLPs (Multi Layer Perceptron) based on the paper Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks.
Once the images were generated the researchers needed to judge their quality. To do this they first tested their synthetic dataset to a more traditional dataset by training a CNN on both sets. What they found was that for all three classes the CNN trained on synthetic data was able to outperform the CNN trained on traditional data. This showed how effective synthetic data could be to create datasets but to take it a step further they assembled a team of two industry professionals and asked them to decide whether each piece of data was generated or real. What they found was that only around 60% of the time were the two professionals able to agree on the correct answer. This means that the remaining 40% of the time one or both of the professionals were unable to correctly discriminate. It should be noted that essentially filling in for the discriminator in a GANis a far cry from both subject’s regular duties but it still shows how effective the methods in this paper can be for dataset augmentation.
GANs are innately limited in the range of images they are capable of producing but that comes at the tradeoff of a generator with a high “understanding” of its image classes, resulting in higher quality outputs. The ability to produce training data at will can become a boon for future societies but we must remember the mistakes of our past. Machine learning algorithms have been shown to perpetuate racism when trained improperly. Similarly a GAN can only produce what it is capable of seeing. It is up to those who use these GANs to make sure that their output is inclusive to all members of society, especially when applied to the field of medicine. This is just one among many concerns that naturally occur when ML begins to be applied to a new field. Despite this, with the proper oversight, GANs will drastically change the world of data collection.
Sources:
