Generative Adversarial Network (+Laplacian Pyramid GAN)
-
Upload
namhyuk-ahn -
Category
Engineering
-
view
208 -
download
5
Transcript of Generative Adversarial Network (+Laplacian Pyramid GAN)
![Page 1: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/1.jpg)
Generative Adversarial NetworkNamHyuk Ahn
![Page 2: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/2.jpg)
![Page 3: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/3.jpg)
Generative Adversarial Network
![Page 4: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/4.jpg)
What is a Generative model?
• Goal: Wish to learn X → Y, P(y|x)
• Discriminative model (classifier)• Directly learn conditional distribution P(y|x) from training data• SVM, Logistic Regression
• Generative model (classifier)• Learn the joint probability, P(x,y) = P(x|y) * P(y)• Estimate parameter of P(x|y), P(y) from training data• Use Bayes rule to calculate P(y|x)• Naive Bayes, GMM
![Page 5: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/5.jpg)
Generative vs. Discriminative
• Generative• Probabilistic “model” of each class• Decision boundary is “where model becomes more likely”• Natural use of unlabeled data (unsupervised learning)
• Discriminative• Focus on the decision boundary• More powerful with lots of data• Only supervised task
IAML2.23: Generative vs. discriminative learning
![Page 6: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/6.jpg)
Generative vs. Discriminative
http://slideplayer.com/slide/6982498/
![Page 7: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/7.jpg)
Adversarial Learning
http://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-generative-models-and-adversarial-training-upc-2016
![Page 8: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/8.jpg)
Adversarial Learning
http://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-generative-models-and-adversarial-training-upc-2016
counterfeiters
police
fake currency
real currency
![Page 9: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/9.jpg)
(neural) Network
http://cs231n.github.io
![Page 10: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/10.jpg)
Adversarial Network
• Notations• 𝑝": Generator’s distribution over data 𝑥• 𝑝$(𝒛): Prior on input noise• 𝐺(𝑧; 𝜃"): Generator function (mlp with parameters 𝜃")• 𝐷(𝑥; 𝜃-): Discriminator function output single scalar
• If input is from real distribution, output is 1, otherwise return 0
• Goals (cost function)
![Page 11: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/11.jpg)
![Page 12: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/12.jpg)
Training procedure
• Optimize D completely in inner loop is bad idea(computation prohibitive, overfitting)
• Instead, first optimize D k steps and optimize G one step• D being maintained near optimal solution• G slowly moves to optimal
• In practice, eq1 may not provide sufficient gradient for G• In early stage, G output poor example, so D can reject with high
confidence• log(1 − 𝐷 𝐺 𝒛 ) saturate (log 1 = 0)
• Rather training G to minimize log(1 − 𝐷 𝐺 𝒛 ), maximize log𝐷(𝐺 𝒛 )
![Page 13: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/13.jpg)
![Page 14: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/14.jpg)
Global Optimality of 𝑝" = 𝑝-676
![Page 15: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/15.jpg)
Global Optimality of 𝑝" = 𝑝-676
![Page 16: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/16.jpg)
Convergence of Algorithm 1
Theoretically, cool.But in practice, GAN not always show good performance
![Page 17: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/17.jpg)
Result
![Page 18: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/18.jpg)
Why GAN is important?
• Use GAN in semi-supervised Learning• Features from discriminator could improve performance
when limited labeled data is available
• Vector Arithmetic• Generate fake image of bedroom (DCGAN)• [man with glasses] - [man without glasses] + [woman without
glasses] = [woman with glasses]
• Conditional GAN• GAN performs unsupervised manner, but also can model 𝑝 𝑥 𝑐
by adding class label in both G and D
![Page 19: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/19.jpg)
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
![Page 20: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/20.jpg)
Image Pyramid
• Image pyramid is multi-scale image representation• Generate pyramid• Blur previous pyramid image• Subsample pixels
• Variety type of pyramid• Gaussian pyramid• Laplacian pyramid• ...
Figure from David Forsyth
![Page 21: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/21.jpg)
Laplacian Pyramid
http://cs.brown.edu/courses/csci1430/2011/results/proj1/georgem/explained.jpg
![Page 22: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/22.jpg)
Conditional GAN
• Add conditional variable→ 𝐷 𝑥 𝑐 𝑜𝑟𝐺 𝑧 𝑐
• Variable 𝑐 can be anything• Class label• Tags correspond to image• Additional image information• ...
![Page 23: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/23.jpg)
![Page 24: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/24.jpg)
![Page 25: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/25.jpg)
Laplacian Pyramid GAN : Sampling procedure
![Page 26: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/26.jpg)
Laplacian Pyramid GAN : Sampling procedure
Start with generator that output scaled image (Gaussian pyramid) 𝐼<=
![Page 27: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/27.jpg)
Laplacian Pyramid GAN : Sampling procedure
1. Upsample Gaussian pyramid 𝐼<= to 𝑙? (green arrow)2. Input noise 𝑧?and 𝑙? to generator(𝑙? is conditional information – orange arrow)
3. Generator output Laplacian pyramid ℎA?
![Page 28: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/28.jpg)
Laplacian Pyramid GAN : Sampling procedure
1. Upsample Gaussian pyramid 𝐼<? to 𝑙B (green arrow)2. Input noise 𝑧Band 𝑙B to generator(𝑙B is conditional information – orange arrow)
3. Generator output Laplacian pyramid ℎAB
![Page 29: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/29.jpg)
Laplacian Pyramid GAN : Sampling procedure
Finally create generated image 𝐼<C
![Page 30: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/30.jpg)
Laplacian Pyramid GAN : Training procedure
![Page 31: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/31.jpg)
Result
![Page 32: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/32.jpg)
Result
![Page 33: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/33.jpg)
Believe or not
![Page 34: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/34.jpg)
Why LAPGAN is better?
• LAPGAN don’t use global Generator/Discriminator• Instead, separate image into multi-scaled pyramids• Other multi-scaled approach might be helpful
• Each G/D only cover each scaled pyramid• Believe or not, LAPGAN produce sharper images• (my thought) • Each Generator focus on generating Laplacian pyramid
which about high-pass (edges) with conditional information• This idea can make generator to produce much sharper images
![Page 35: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/35.jpg)
Other GAN topics
• GAN• LAPGAN• DCGAN• InfoGAN• Bidirectional GAN• EBGAN• ...• ...
![Page 36: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/36.jpg)
DCGAN (15.11)
![Page 37: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/37.jpg)
EBGAN (16.09)
![Page 38: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/38.jpg)
StackGAN (16.12)
![Page 39: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/39.jpg)
StackGAN (16.12)
![Page 40: Generative Adversarial Network (+Laplacian Pyramid GAN)](https://reader033.fdocuments.in/reader033/viewer/2022051520/58ee919f1a28ab9e038b4593/html5/thumbnails/40.jpg)
Reference
• Goodfellow, Ian, et al. "Generative adversarial nets." Advances in Neural Information Processing Systems. 2014.
• Denton, Emily L., Soumith Chintala, and Rob Fergus. "Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks." Advances in neural information processing systems. 2015.