SlimGANs: Real-Time Adjustable Model Sizes

Generative adversarial networks (GANs) have been very successful with their increasingly larger model scales in recent years. GANs’ practical applications, however, are also becoming increasingly restricted by the unwieldy model sizes. Researchers from University of Chinese Academy of Sciences, ByteDance AI Lab, Inception Institute of Artificial Intelligence and Beihang University recently proposed novel “once-for-all” slimmable generative adversarial networks (SlimGANs) that can easily change model sizes during runtime to implement quality-efficiency trade-offs based on practical needs.

The researchers identify three major challenges in designing the approach:

• Accurate estimation of the divergence between generators at different widths and the real data through discriminators
• Consistency of the generated images between generators of different widths
• Incorporating the label information into generators at different widths in class-conditional generation

The team applied multiple partial parameter-shared discriminators to guide the generator at the corresponding width to deal with the first challenge, while the consistency issue was solved by a novel stepwise inplace distillation technique, which helps narrow generators learn from wide networks during training. The researchers propose a sliceable conditional batch normalization (scBN) for the label incorporation. The resulting SlimGANs thus comprise a slimmable generator with multi-width configurations and multiple partial parameter-shared discriminators.

For evaluations, the researchers used three datasets, CIFAR-10, STL-10, and CelebA, on two network backbones, DCGAN and ResNet. In most cases, SlimGANs surpassed or were competitive with individually trained GANs, demonstrating their effectiveness across various conditions. The SlimGANs with different batch normalizations also heavily outperformed the baselines in class-conditional generation experiments.

The paper Slimmable Generative Adversarial Networks has been accepted by AAAI 2021 and is available on arXiv. The codes are on the project GitHub.

Analyst: Reina Qi Wan | Editor: Michael Sarazen; Yuan Yuan

Synced Report | A Survey of China’s Artificial Intelligence Solutions in Response to the COVID-19 Pandemic — 87 Case Studies from 700+ AI Vendors

This report offers a look at how China has leveraged artificial intelligence technologies in the battle against COVID-19. It is also available on Amazon Kindle. Along with this report, we also introduced a database covering additional 1428 artificial intelligence solutions from 12 pandemic scenarios.

Click here to find more reports from us.

We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.