Ganss Harward Uni Notes
Ganss Harward Uni Notes
m: Number of How realistic are the Make sure real samples Make sure generated
samples generated samples? are classified as being samples are classified as
z: Random noise real. unreal.
samples G wants to maximize this.
x: Real samples D wants to maximize this. D wants to minimize this.
m: Number of samples
z: Random noise samples
x: Real samples
Discriminator
D(x) = 0.9997 D(G(z)) = 0.1617
Generator - 1.0
Targets
CS109B, PROTOPAPAS, GLICKMAN
Applications
● (Conditional) synthesis
○ Font generation
○ Text2Image
○ 3D Object generation
● Data augmentation
○ Aiming to reduce need for labeled data
○ GAN is only used as a tool enhancing the training process of another model
● Style transfer and manipulation
○ Face Aging
○ Painting
○ Pose estimation and manipulation
○ Inpainting
○ Blending
● Signal super resolution
Applications: Style Transfer and Manipulation
Applications: Style Transfer and Manipulation
Applications: Style Transfer and Manipulation
Applications: Style Transfer and Manipulation
Applications: Style Transfer and Manipulation
Applications: Style Transfer and Manipulation
Applications: Style Transfer and Manipulation
Applications: Style Transfer and Manipulation
Applications: Style Transfer and Manipulation
Applications: Style Transfer and Manipulation
Applications: Signal Super Resolution
Applications: Signal Super Resolution
Common Problems: Oscillation
● Both generator and discriminator jointly searching for equilibrium, but model updates
are independent.
● No theoretical convergence guarantees.
● Solution: Extensive hyperparameter-search, sometimes manual intervention.
Common Problems: Vanishing gradient
● Discriminator can become too strong to provide signal for the generator.
● Generator can learn to fool the discriminator consistently.
● Solution: Do (not) pretrain discriminator, or lower its learning rate relatively to the
generator. Change the number of updates for generator/discriminator per iteration.
GANs and Game Theory
Inception score
TSTR score
Wasserstein GAN
Distance is everything:
In general, generative models seek to minimize the distance between
real and learned distribution.
K.mean(y_true * y_pred)
def get_config(self):
return {'name': self.__class__.__name__, 'c': self.c}
CycleGAN
CycleGAN
G AB G BA
Conditional c
Troubleshooting GANs
GANs can be frustrating to work with. Here are some tips for your reference:
● Models. Make sure models are correctly defined. You can debug the discriminator
alone by training on a vanilla image-classification task.
● Data. Normalize inputs properly to [-1, 1]. Make sure to use tanh as final activation
for the generator in this case.
● Noise. Try sampling the noise vector from a normal distribution (not uniform).
● Normalization. Apply BatchNorm when possible, and send the real and fake
samples in separate mini-batches.
● Activations. Use LeakyRelu instead of Relu.
● Smoothing. Apply label smoothing to avoid overconfidence when updating the
discriminator, i.e. set targets for real images to less than 1.
● Diagnostics. Monitor the magnitude of gradients constantly.
● Vanishing gradients. If the discriminator becomes too strong (discriminator loss
= 0), try decreasing its learning rate or update the generator more often.
Building an Image GAN
In the code example, if you don’t tune parameters carefully, you won’t
surpass this level by much:
Building an Image GAN: Discriminator
Target
Fake: 0
Real: 1
Building an Image GAN: Generator
Source
Noise vector
model = keras.Sequential()
model.add(generator)
model.add(discriminator)
discriminator.trainable = False
model.compile(...)
Building an Image GAN: Training Loop
Deep Generative Image Models using a Laplacian Pyramid of Adversarial 2015 https://arxiv.org/pdf/1506.05751.pdf
Networks (LAPGAN)
Autoencoding beyond pixels using a learned similarity metric (VAE-GAN) 2015 https://arxiv.org/pdf/1512.09300.pdf
Learning from Simulated and Unsupervised Images through Adversarial 2016 https://arxiv.org/pdf/1612.07828v1.pdf
Training (SimGAN)
Some Available GANs...
https://www.jessicayung.com/explaining-tensorflow-code-for-a-convolutional-neural-networ
k/
https://lilianweng.github.io/lil-log/2017/08/20/from-GAN-to-WGAN.html
https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html
https://github.com/tensorlayer/srgan
https://junyanz.github.io/CycleGAN/
https://affinelayer.com/pixsrv/
https://tcwang0509.github.io/pix2pixHD/