Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative...

31
Generative Adversarial Networks And Their Applications Bindita Chaudhuri

Transcript of Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative...

Page 1: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Generative Adversarial Networks

And Their Applications

Bindita Chaudhuri

Page 2: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Unsupervised Learning: Autoencoders

Page 3: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Unsupervised Learning: Autoencoders

Page 4: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Unsupervised Learning: Variational Autoencoders

Page 5: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Generative Adversarial Networks: Idea

Page 6: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Generative Adversarial Networks: Idea

Page 7: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Generative Adversarial Networks

Page 8: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Generative Adversarial Networks

Page 9: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Distributions during training

Page 10: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

GAN: Sample Architecture (DC-GAN)

Page 11: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Generated Samples

MNIST Faces CIFAR 10

Page 12: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Bidirectional GAN (BiGAN)

Page 13: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Conditional GAN (cGAN)

Generator Discriminator

Page 14: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Conditional GAN (cGAN)

Generator Discriminator

Page 15: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Pix2Pix: Type of cGAN

L1 loss

Page 16: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

CycleGAN: Unsupervised Pix2Pix

Page 17: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

CycleGAN: Unsupervised Pix2Pix

Page 18: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

CycleGAN Results

Page 19: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Progressive Growing of GANs

Page 20: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Progressive GAN Results

Celebrities Bedrooms Objects

Page 21: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Application: Neural Style transfer

Page 22: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Application: 3D GAN

Page 23: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

My Project: Facial Motion Retargeting

Page 24: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

My Project: Facial Motion Retargeting

Network

Page 25: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

• Compute facial landmarks:

2D-to-3D CycleGAN

Page 26: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

• Compute facial landmarks:

• Convert 3D model to 2D position map:

2D-to-3D CycleGAN

Page 27: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

3D discriminator

landmark loss

• Compute facial landmarks:

• Convert 3D model to 2D position map:

• Train CycleGAN:

generator A→B

generator B→A

2D discriminator

2D-to-3D CycleGAN

Page 28: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Input Landmark only CycleGAN

Results

Page 29: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Input Landmark only CycleGAN

Results

Page 30: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Useful links

• GAN Zoo: https://github.com/hindupuravinash/the-gan-zoo

• GAN hacks: https://github.com/soumith/ganhacks

• Code Bases:• Tensorflow: https://www.tensorflow.org/tutorials/generative/dcgan• Keras: https://github.com/eriklindernoren/Keras-GAN• Pytorch: https://github.com/pytorch/examples/tree/master/dcgan

• References:• http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture13.pdf• https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/slides/lec19.pdf• https://www.cs.cmu.edu/~bhiksha/courses/deeplearning/Fall.2015/slides/lec13.GAN.pdf

Page 31: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:

Deep Adversarial Learning for NLP - sameersingh.orgsameersingh.org/files/ppts/naacl19-advnlp-part1-william-slides.pdf · Adversarial Training: A Simple Example •Adversarial Training

Deep Adversarial Learning for NLP - sameersingh.orgsameersingh.org/files/ppts/naacl19-advnlp-part1-william-slides.pdf · Adversarial Training: A Simple Example •Adversarial Training

Adversarial Examples and Adversarial Trainingc4dm.eecs.qmul.ac.uk/horse2016/HORSE2016_Goodfellow.pdf• “Explaining and Harnessing Adversarial Examples” Goodfellow et al 2014 •

Adversarial Examples and Adversarial Trainingc4dm.eecs.qmul.ac.uk/horse2016/HORSE2016_Goodfellow.pdf• “Explaining and Harnessing Adversarial Examples” Goodfellow et al 2014 •

Generative Adversarial Text to Image Synthesisproceedings.mlr.press/v48/reed16.pdfGenerative Adversarial Text to Image Synthesis tures to synthesize a compelling image that a human

Generative Adversarial Text to Image Synthesisproceedings.mlr.press/v48/reed16.pdfGenerative Adversarial Text to Image Synthesis tures to synthesize a compelling image that a human

Hypothetical Exploration on Generative Adversarial Networks · Convolutional Neural Networks, Generative Adversarial Net- works, Laplacian Pyramid of Adversarial Networks. I. INTRODUCTION

Hypothetical Exploration on Generative Adversarial Networks · Convolutional Neural Networks, Generative Adversarial Net- works, Laplacian Pyramid of Adversarial Networks. I. INTRODUCTION

Games & Adversarial Search

Games & Adversarial Search

Deep Adversarial Learningfor NLPstatic.tongtianta.site/paper_pdf/95dc5768-97b9-11e9-8912-00163e08bb… · •Introduction: Adversarial Learning in NLP •Adversarial Generation •A

Deep Adversarial Learningfor NLPstatic.tongtianta.site/paper_pdf/95dc5768-97b9-11e9-8912-00163e08bb… · •Introduction: Adversarial Learning in NLP •Adversarial Generation •A

Generative adversarial networks

Generative adversarial networks

Adversarial search

Adversarial search

Generative Adversarial Networks - HAW Hamburgubicomp/projekte/master... · Generative Adversarial Networks ... Outline 1. Basic idea 2. Generative Adversarial ... com/2017/01/generative-adversarial-networks-hot-topic-machine-learning

Generative Adversarial Networks - HAW Hamburgubicomp/projekte/master... · Generative Adversarial Networks ... Outline 1. Basic idea 2. Generative Adversarial ... com/2017/01/generative-adversarial-networks-hot-topic-machine-learning

FAD: Fairness through Adversarial Discriminationcs229.stanford.edu/proj2018/poster/109.pdf · FAD: Fairness through Adversarial Discrimination Adversarial Model Discussion •Expand

FAD: Fairness through Adversarial Discriminationcs229.stanford.edu/proj2018/poster/109.pdf · FAD: Fairness through Adversarial Discrimination Adversarial Model Discussion •Expand

Generative Adversarial Networkscs236.stanford.edu/assets/slides/cs236_lecture9.pdfGenerative Adversarial Networks A two player minimax game between a generator and a discriminator

Generative Adversarial Networkscs236.stanford.edu/assets/slides/cs236_lecture9.pdfGenerative Adversarial Networks A two player minimax game between a generator and a discriminator

Demonstration of GestureCalc: An Eyes-Free Calculator for ...bindita/papers/gesturecalc_assets19_demo.pdfnumeric input) [2], and BrailleTap (Braille-based gesture calculator) [1].

Demonstration of GestureCalc: An Eyes-Free Calculator for ...bindita/papers/gesturecalc_assets19_demo.pdfnumeric input) [2], and BrailleTap (Braille-based gesture calculator) [1].

Adversarial Multiclass Classification: A Risk Minimization ...papers.nips.cc/paper/6088-adversarial-multiclass...Adversarial Multiclass Classification: A Risk Minimization Perspective

Adversarial Multiclass Classification: A Risk Minimization ...papers.nips.cc/paper/6088-adversarial-multiclass...Adversarial Multiclass Classification: A Risk Minimization Perspective

Lecture 9: Unsupervised, Generative & Adversarial Networks · Lecture 9: Unsupervised, Generative & Adversarial Networks ... GENERATIVE & ADVERSARIAL NETWORKS - 20 o Autoencoders

Lecture 9: Unsupervised, Generative & Adversarial Networks · Lecture 9: Unsupervised, Generative & Adversarial Networks ... GENERATIVE & ADVERSARIAL NETWORKS - 20 o Autoencoders

Adversarial Pattern Classification

Adversarial Pattern Classification

Adversarial Examples and Adversarial Training · Adversarial Examples and Adversarial Training Ian Goodfellow, OpenAI Research Scientist Guest lecture for CS 294-131, UC Berkeley,

Adversarial Examples and Adversarial Training · Adversarial Examples and Adversarial Training Ian Goodfellow, OpenAI Research Scientist Guest lecture for CS 294-131, UC Berkeley,

Dynamic Visual Sequence Prediction with Motion Flow Networksjdh/images/100.pdfgenerative adversarial network (GAN), which has greatly improved the image generation quality compared

Dynamic Visual Sequence Prediction with Motion Flow Networksjdh/images/100.pdfgenerative adversarial network (GAN), which has greatly improved the image generation quality compared

Lecture 22: Adversarial Image, Adversarial Training, Variational Autoencoders… · 2018-11-08 · Adversarial Images Adversarial Training Autoencoder VAE GAN Conclusions Lecture

Lecture 22: Adversarial Image, Adversarial Training, Variational Autoencoders… · 2018-11-08 · Adversarial Images Adversarial Training Autoencoder VAE GAN Conclusions Lecture

Adversarial Ranking Attack and Defense...Adversarial Ranking Attack and Defense 5 Therefore, adversarial ranking attack is to nd a proper adversarial pertur-bation which leads the

Adversarial Ranking Attack and Defense...Adversarial Ranking Attack and Defense 5 Therefore, adversarial ranking attack is to nd a proper adversarial pertur-bation which leads the

Multi-scale Generative Adversarial Networks for Crowd …static.tongtianta.site/paper_pdf/cee33a9e-bc31-11e9-b3c6-00163e08bb86.pdfGenerative adversarial networks [16] are commonly

Multi-scale Generative Adversarial Networks for Crowd …static.tongtianta.site/paper_pdf/cee33a9e-bc31-11e9-b3c6-00163e08bb86.pdfGenerative adversarial networks [16] are commonly