Following MIT, NVIDIA researchers have recently developed a new augmented method for training Generative Adversary Networks (GAN) with a limited amount of data. The approach is an adaptive discriminator augmentation mechanism that significantly stabilized training in limited data regimes.
Machine learning models are hungry for data. In fact, in recent years, we have seen that models that are fed with silos of data produce exceptional predictive results.
In parallel, with significant growth, Adversarial Generative Networks have been used successfully for various applications, including high-fidelity natural image synthesis, data augmentation tasks, image compression enhancement, etc. humans and machines to introduce new and unique art forms, GANs have you covered.
The need for less data
Although deep neural network models, including GANs, have shown impressive results, the challenge of collecting a large number of specific data sets remains. In the case of GANs, the challenge is to collect a set of images large enough for a specific application that imposes restrictions on subject type, image quality, geographic location, time period, privacy, copyright status, among others.
Also, one of the key problems with small data sets is that the discriminator adapts to the training examples, and thus its feedback to the generator loses meaning and the training begins to diverge.
According to the researchers, in almost all areas of deep learning, data set augmentation is known to be the standard solution against overfitting. In contrast, a trained GAN with augmentations of similar data sets learns to generate the augmented distribution, which is generally considered highly undesirable.
This is the rationale behind the development of the new approach that does not require changes to loss functions or network architectures and is applicable during training as well as during fine-tuning of an existing GAN on another dataset. The researchers also demonstrated how to use a wide range of magnification to prevent the discriminator from overfitting, ensuring that none of the magnification is leaked into the generated images.
Behind the model
The researchers called this approach Adaptive Discriminator Augmentation (ADA), where they tested the method against a number of alternatives on the artificial subset of larger data sets such as FFHQ and LSUN CAT to study how the amount of available training data affects GAN training.
The popular StyleGAN2 and BigGAN data sets are considered a baseline; however, the researchers chose StyleGAN2 because it provided more predictable results with significantly less variation between training runs. Additionally, adaptive magnification is compared to a broader set of alternatives, such as PA-GAN, WGAN-GP, zCR, auxiliary rotations, and spectral normalization.
During the process, the researchers studied the effectiveness of augmentation of the stochastic discriminator by performing exhaustive sweeps for different categories of magnification and sizes of data sets. They observed that the optimal strength of increase is highly dependent on the amount of training data, and not all categories of increase are equally useful in practice.
Ending
Results from the FFHQ and LSUN CAT dataset across all training set sizes demonstrated that increasing the adaptive discriminator (ADA) substantially improves FIDs in data-limited scenarios. Additionally, with ADA, the augmentations are not filtered out and therefore the same diverse set of augmentations can be safely used across all data sets.
Therefore, the researchers demonstrated that increasing the adaptive discriminator reliably stabilizes training and greatly improves the resulting quality when training data is sparse. Contributions in this work facilitate the training of high quality generative models with custom sets of images.
They stated, “Of course, augmentation is not a substitute for actual data; one should always try to collect a large, high-quality set of training data first, and only then fill in the gaps by augmenting.” They added: “As future work, it would be worth looking for the most effective set of augmentations and seeing if recently published techniques such as the U-net discriminator or multimodal generator could also help with limited data.”
For more details, read the document here.
If you loved this story, join our Telegram community.
Plus, you can write to us and be one of over 500 experts who have contributed stories to AIM. Share your nominations here.
