# Generative models collection

PyTorch implementations of various generative models to be trained and evaluated on **CelebA** dataset. The models are: *Deep Convolutional GAN, Least Squares GAN,
Wasserstein GAN, Wasserstein GAN Gradient Penalty, Information Maximizing GAN, Boundary Equilibrium GAN, Variational AutoEncoder and Variational AutoEncoder GAN*.
All models have as close as possible nets architectures and implementations with necessary deviations required by their articles.

For now all models except InfoGAN are **conditional** on attributes, with attributes vector concatenated to latent variable for generator and to
channels for discriminator. However if desired conditionality can be easily removed.

Most of the code is shared between the models, so adding a model, changing models architectures or usage on different dataset all require little effort. As most GANs differ only in the way generator and discriminator losses are computed adding a model might only require to inherit from GAN superclass and provide losses.

For visual control during training *Visdom* is used and requires only to provide values and images interested in.

# Comments on models and results

All models share architecture of *DCGAN* with slight deviations and were trained using Adam(0.5, 0.999) with batch size is 64 and learning rate is 0.0001.

Most models were not trained long enough.

Latent dimension is 128 for all models and models are conditioned on 40 binary attributes such as: ‘bald’, ‘eyeglasses’, ‘male’, etc.

Epochs count is based on discriminator steps (number of generator steps for Wassertein and non-Wassertein GANs differ greatly during one epoch).

## DCGAN

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Learning rate is 0.0001 and 2 generator steps per discriminator step provided better results.

## LSGAN

Least Squares Generative Adversarial Networks

Learning rate is 0.0001 and 2 generator steps per discriminator step.

Had slightly more stable training then DCGAN.

## WGAN

Learning rate is 0.0001 and 5 discriminator steps per generator step.

## WGANGP

Improved Training of Wasserstein GANs

Learning rate is 0.0001 and 5 discriminator steps per generator step.

## InfoGAN

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

Learning rate is 0.0001 and 2 generator steps per discriminator step.

In contrast to other models this one was not trained with conditional attributes. Instead they were used as binomial latent variables. Their true distribution was used as prior for fake images attributes, however their real meaning was to be understood by the model.

### TODO

Check whether attributes really got their true interpretation.

## BEGAN

BEGAN: Boundary Equilibrium Generative Adversarial Networks

Learning rate is 0.0001 and 2 generator steps per discriminator step.

Skip-connections proposed by the authors of the article were not introduced here. As this would require to heavily modify the nets architectures. The model experienced heavy mode-collapsing. Authors suggest to reduce learning rate to avoid it. This was not tried.

## VAE

Auto-Encoding Variational Bayes

Learning rate is 0.0001.

## VAEGAN

Autoencoding beyond pixels using a learned similarity metric

Learning rate is 0.0001 and 2 generator steps per discriminator step.

Generator loss was changed to exploit log(1-x) -> -log(x) trick. It slightly losses probabilistic interpretation, but at least it is able to train this way.

(Epochs count is based on generator steps)

### Real and decoded

Link to the repository