Faster A.I. For Fortnite To PUBG Graphics Conversion
Overview of the paper “Contrastive Learning for Unpaired Image-to-Image Translation” by Park et al.
About a couple of years back, I showed how we can use style transfer AI like CycleGAN to convert graphics of one game to look like that of another with the example of Fortnite and PUBG. This project remains one of my most viewed projects till date, still pulling in new viewers after two years. This means there is a lot of interest in this type of AI research, but unfortunately, we have seen very limited advancements to convert such prototypes into reality. While higher resolution versions of this AI were introduced, they resorted to using multiple GPUs for training and it was impractical for real-world usage.
Thankfully, after a lot of time, we finally have a paper showing significant advancement in trying to reduce the computational power required to train this AI. The paper from UC Berkeley and Adobe research is titled “Contrastive Learning for Unpaired Image-to-Image Translation” (CUT).
Using the exact same data-set and the exact same GPU hardware that I used last time around, I was able to move up from 256p to 400p resolution of the synthesized images with this newer model. Not only that, but I was able to train this in just under 2 hours, compared to 8+ hours last time around.
CycleGAN vs Patchwise Contrastive Framework
This is a significant difference in the amount of computational power required compared to CycleGAN. So, what is different in this approach compared to CycleGAN? It now uses Patchwise Contrastive Learning framework which requires significantly less GPU memory and calculations compared to CycleGAN.
The generator network learns to convert Fortnite image into PUBG. Now, remember that in CycleGAN, we would create another network that tries to convert PUBG into Fortnite to calculate the reconstruction error and this creates a massive overhead in terms of GPU power and memory requirements.
So here, we instead use Contrastive Loss. First, instead of dealing with entire images at once, this method focuses on extracting patches from the input and output images. The task for our model to learn here is to identify which among the multiple input keys is a positive match for our query patch obtained from the synthesized image. This is called Contrastive Learning and it enables the model to learn better feature representations with Self-Supervision.
Comparison with CycleGAN
This new approach is why the synthesized images with this method have better object separation boundaries and it retains more information from the original image after conversion.
And remember, all of this comes with reduced GPU requirements, so that’s fantastic! To check out more results of this paper on other datasets, head over to the authors’ project page here.
Thank you for reading. If you liked this article, you may follow more of my work on Medium, GitHub, or subscribe to my YouTube channel.
