Add VanillaGAN tensorflow files (pclubiitk#17)

* Add Gan folder,VanillaGAN-tensorflow

* Create README.md

* Update README.md

* Update files and compress gif

* changes applied

Co-authored-by: ashishpm <[email protected]>

Author: ashishmurali

generative_models/VanillaGAN_TensorFlow/README.md

@@ -0,0 +1,121 @@
+# TensorFlow Implementation of VanillaGAN on MNIST Dataset
+
+### Usage
+```bash
+$ python3 main.py --epochs 50 --batch_size 128 --outdir "." 
+```
+NOTE: on Colab Notebook use following command:
+```bash
+!git clone link-to-repo
+%run main.py --epochs 50 --batch_size 128 --outdir "."
+```
+
+## Help Log
+```
+                        
+usage: main.py [-h] [--epochs EPOCHS] [--batch_size BATCH_SIZE] --outdir
+               OUTDIR [--learning_rate LEARNING_RATE] [--beta_1 BETA_1]
+               --encoding_dims ENCODING_DIMS
+
+optional arguments:
+  -h, --help            show this help message and exit
+  --epochs EPOCHS
+  --batch_size BATCH_SIZE
+  --outdir OUTDIR
+  --learning_rate LEARNING_RATE
+  --beta_1 BETA_1
+  --encoding_dims ENCODING_DIMS
+                        
+```
+
+### Contributed by:
+* [Ashish Murali](https://github.com/ashishmurali)
+
+# References :
+
+* **Title**: Generative Adversarial Networks
+* **Authors**: Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
+* **Link**: http://arxiv.org/abs/1406.2661
+* **Tags**: Neural Network, GAN, generative models, unsupervised learning
+* **Year**: 2014
+
+# Summary
+
+* What are GANs
+  * GANs are based on adversarial training.
+  * Adversarial training is a basic technique to train generative models (so here primarily models that create new images).
+  * In an adversarial training one model (G, Generator) generates things (e.g. images). Another model (D, discriminator) sees real things (e.g. real images) as well as fake things (e.g. images from G) and has to learn how to differentiate the two.
+  * Neural Networks are models that can be trained in an adversarial way (and are the only models discussed here).
+
+* Basic architecture of GANs
+  * G is a simple neural net (e.g. just one fully connected hidden layer). It takes a vector as input (e.g. 100 dimensions) and produces an image as output.
+  * D is a simple neural net (e.g. just one fully connected hidden layer). It takes an image as input and produces a quality rating as output (0-1, so sigmoid).
+  * You need a training set of things to be generated, e.g. images of human faces.
+  * Let the batch size be B.
+  * G is trained the following way:
+    * Create B vectors of 100 random values each, e.g. sampled uniformly from [-1, +1]. (Number of values per components depends on the chosen input size of G.)
+    * Feed forward the vectors through G to create new images.
+    * Feed forward the images through D to create ratings.
+    * Use a cross entropy loss on these ratings. All of these (fake) images should be viewed as label=0 by D. If D gives them label=1, the error will be low (G did a good job).
+    * Perform a backward pass of the errors through D (without training D). That generates gradients/errors per image and pixel.
+    * Perform a backward pass of these errors through G to train G.
+  * D is trained the following way:
+    * Create B/2 images using G (again, B/2 random vectors, feed forward through G).
+    * Chose B/2 images from the training set. Real images get label=1.
+    * Merge the fake and real images to one batch. Fake images get label=0.
+    * Feed forward the batch through D.
+    * Measure the error using cross entropy.
+    * Perform a backward pass with the error through D.
+  * Train G for one batch, then D for one (or more) batches. Sometimes D can be too slow to catch up with D, then you need more iterations of D per batch of G.
+
+* Results
+  * Good looking images MNIST-numbers and human faces. (Grayscale, rather homogeneous datasets.)
+  * Not so good looking images of CIFAR-10. (Color, rather heterogeneous datasets.)
+
+
+-------------------------
+# Our implementation :
+
+
+
+* We have implemented the GAN model with the following architectures :
+
+* Generator Architecture
+
+  ![Generator](https://github.com/ashishmurali/model-zoo/blob/master/generative_models/VanillaGAN_TensorFlow/assets/generator_architecture.png)
+  
+  
+* Discriminator Architecture 
+
+  ![Discriminator](https://github.com/ashishmurali/model-zoo/blob/master/generative_models/VanillaGAN_TensorFlow/assets/discriminator_architecture.png)
+
+
+
+# Results of our implementation :
+
+
+
+* The following GIF shows how our model has improved generating digits after 400 epochs of training
+
+  ![gif](https://github.com/ashishmurali/model-zoo/blob/master/generative_models/VanillaGAN_TensorFlow/assets/gan.gif)
+  
+* The image generated by our model after the first epoch
+
+  ![epoch1](https://github.com/ashishmurali/model-zoo/blob/master/generative_models/VanillaGAN_TensorFlow/assets/gan_image%201.png) 
+  
+* The image generated by our model after the 400th epoch
+
+  ![epoch400](https://github.com/ashishmurali/model-zoo/blob/master/generative_models/VanillaGAN_TensorFlow/assets/gan_image%20400.png)  
+ 
+* The Generator loss for our model 
+
+  ![gloss](https://github.com/ashishmurali/model-zoo/blob/master/generative_models/VanillaGAN_TensorFlow/assets/generator_loss.png)
+  
+* The Discriminator loss for our model 
+
+  ![dloss](https://github.com/ashishmurali/model-zoo/blob/master/generative_models/VanillaGAN_TensorFlow/assets/discriminator_loss.png)
+
+
+
+### Sources:
+* [Papers](https://github.com/aleju/papers/blob/master/neural-nets/Generative_Adversarial_Networks.md)

generative_models/VanillaGAN_TensorFlow/assets/discriminator_architecture.png

@@ -0,0 +1,10 @@
+import numpy as np
+import tensorflow as tf
+from tensorflow.keras.datasets import mnist
+
+
+def load_data():
+    (x_train,_), (_,_) = mnist.load_data()
+    x_train = (x_train.astype(np.float32) - 127.5)/127.5
+    x_train = x_train.reshape(60000, 784)
+    return x_train

generative_models/VanillaGAN_TensorFlow/assets/discriminator_loss.png

@@ -0,0 +1,84 @@
+import os
+import argparse
+import numpy as np
+
+from dataloader import load_data
+from utils import plot_generated_images,make_gif
+from models import create_generator,create_gan,create_discriminator
+
+def run_from_ipython():
+    try:
+        __IPYTHON__
+        return True
+    except NameError:
+        return False
+
+ipython = run_from_ipython()
+
+if ipython:
+    from IPython import display
+
+parser = argparse.ArgumentParser()
+
+parser.add_argument('--epochs', type=int, default=50)
+parser.add_argument('--batch_size', type=int, default=128)
+parser.add_argument('--outdir', type=str, required=True,default='.')
+parser.add_argument('--learning_rate', type=float, default=0.0002)
+parser.add_argument('--beta_1', type=float, default=0.5)
+parser.add_argument('--encoding_dims', type=int, required=True,default=100)
+
+args = parser.parse_args()
+
+outdir = args.outdir
+if not os.path.exists(outdir):
+    os.makedirs(outdir)
+
+epochs = args.epochs
+batch_size = args.batch_size
+outdir = args.outdir
+learning_rate = args.learning_rate
+beta_1 = args.beta_1
+encoding_dims = args.encoding_dims
+
+def training(epochs, batch_size):
+
+    X_train = load_data()
+    batch_count = int(X_train.shape[0] / batch_size)
+
+    generator= create_generator(learning_rate,beta_1,encoding_dims)
+    discriminator= create_discriminator(learning_rate,beta_1)
+    gan = create_gan(discriminator, generator,encoding_dims)
+
+    valid = np.ones((batch_size, 1))
+    fake = np.zeros((batch_size, 1))
+
+    seed = np.random.normal(0,1, [25, encoding_dims])
+
+    for e in range(1,epochs+1 ):
+        print("Epoch %d" %e)
+        for _ in range(batch_count):
+
+          noise= np.random.normal(0,1, [batch_size, encoding_dims])
+          generated_images = generator.predict(noise)
+
+          image_batch = X_train[np.random.randint(low=0,high=X_train.shape[0],size=batch_size)]
+
+          discriminator.trainable=True
+          d_loss_real = discriminator.train_on_batch(image_batch, valid)
+          d_loss_fake = discriminator.train_on_batch(generated_images, fake)
+          d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)
+
+          noise= np.random.normal(0,1, [batch_size, encoding_dims])
+
+          discriminator.trainable=False
+          g_loss = gan.train_on_batch(noise,valid)
+
+          print ("%d [D loss: %f] [G loss: %f]" % (e, d_loss, g_loss))
+        if ipython:
+            display.clear_output(wait=True)
+        plot_generated_images(e, generator,seed,outdir)
+    generator.save('{}/gan_model'.format(outdir))
+
+training(epochs,batch_size)
+
+make_gif(outdir)