Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked...
Transcript of Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked...
![Page 1: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/1.jpg)
Generative
Adversarial Networks:
When fake never
looked so real
Evan Ntavelis1,2
Dr. Iason Kastanis1
Philipp Schmid1
{ens, iks, psd}@csem.ch
1. Robotics & Machine Learning
CSEM SA
2. Computer Vision Lab
ETH Zürich
![Page 2: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/2.jpg)
2
CSEM at a glance – Close to industry
N A L
MZ
Zürich
Muttenz
Neuchâtel
Alpnach
Landquart
83.0Turnover
(mio CHF)
450Persons
175Industrial
clients
64European
projects
![Page 4: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/4.jpg)
4
Unpaired Image-to-Image Translation using
Cycle-Consistent Adversarial Networks
Zhu et al. 2017
![Page 5: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/5.jpg)
5
AttnGAN: Fine-Grained Text to Image Generation
with Attentional Generative Adversarial Networks}
Xu et al 2018
![Page 6: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/6.jpg)
6
A Style-Based Generator Architecture
for Generative Adversarial Networks
Karras et al. 2018
![Page 7: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/7.jpg)
7
Semantic Image Synthesis with
Spatially-Adaptive Normalization
Park et al. 2019
![Page 9: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/9.jpg)
9
Few-Shot Adversarial Learning of
Realistic Neural Talking Head Models
Zakharov et al. 2019
![Page 10: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/10.jpg)
10
Generative Adversarial Nets
• Introduced in 2014 by Ian
Goodfellow
• Rapidly Adopted
• Unprecedented Generational
Quality
![Page 11: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/11.jpg)
11
Generative Adversarial Nets
• An adversarial game between
two subnets:
• The Generator
• The Discriminator
![Page 12: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/12.jpg)
12
• In the era of Fake News do highly realistic images harbor dangers to
the society?
Deep Fakes
![Page 14: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/14.jpg)
14
How can we use GANs in the industry?
The important question…
![Page 15: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/15.jpg)
15
• Gathering data is tedious and
costly
• Good quality labels require
even more effort
The Problem
![Page 16: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/16.jpg)
16
• Adversarial Domain Adaptation
• Train on a simulated data and
adapt for the use case
• Data Augmentation
• Learn how to generate new
samples to train with
• Generate images with desired
attributes
A Solution Using Adversarial Networks
Sources: CyCADA: Cycle-Consistent Adversarial Domain Adaptation
Hoffman et al. 2017,
GAN-based Synthetic Medical Image Augmentation
for increased CNN Performance
in Liver Lesion Classification
Frid-Adar et al, 2018
![Page 17: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/17.jpg)
17
• GANs are not a panacea
• Nascent technology
• Difficult to train
• Require abundance of data
• Clever schemes may reduce the
effort
• Yet, very promising results
• Worth the effort!
But…
![Page 18: Generative Adversarial Networks - SDS2019...Generative Adversarial Networks: When fake never looked so real Evan Ntavelis1,2 Dr. Iason Kastanis1 Philipp Schmid1 {ens, iks, psd}@csem.ch](https://reader033.fdocuments.in/reader033/viewer/2022053005/5f0945687e708231d4260644/html5/thumbnails/18.jpg)
18
Are you interested in being part of a highly stimulating environment
working on the latest Deep Learning Technologies?
We are hiring!
That’s all folks!