Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney...
Transcript of Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney...
![Page 1: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/1.jpg)
Mooney Faces from Photos
Tsung-Wei Ke Stella X. Yu David Whitney
VSS 2017
UC Berkeley / ICSI
![Page 2: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/2.jpg)
Faces are everywhere
[Miller, etc., 2007]
![Page 3: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/3.jpg)
![Page 4: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/4.jpg)
Source Photos are usually not available
? ? ? ? ? ?
? ? ? ? ? ?
![Page 5: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/5.jpg)
![Page 6: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/6.jpg)
[Ng, and Winkler, ICIP2014]
![Page 7: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/7.jpg)
Machine Learning
[Ng, and Winkler, ICIP2014]
![Page 8: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/8.jpg)
Machine Learning
Generate Large Scale Mooney Face Dataset
[Ng, and Winkler, ICIP2014]
![Page 9: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/9.jpg)
Machine Learning
Generate Large Scale Mooney Face Dataset
Predict Source Photos from Mooney Faces
[Ng, and Winkler, ICIP2014]
![Page 10: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/10.jpg)
Machine Learning
Generate Large Scale Mooney Face Dataset
Predict Source Photos from Mooney Faces
[Ng, and Winkler, ICIP2014]
![Page 11: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/11.jpg)
Train a Mooney Face Classifier
Mooney face classifier
![Page 12: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/12.jpg)
Train a Mooney Face Classifier
Mooney face classifier
![Page 13: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/13.jpg)
Train a Mooney Face Classifier
Mooney face classifier
![Page 14: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/14.jpg)
Generate Mooney Candidates
![Page 15: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/15.jpg)
Generate Mooney Candidates
two-tone candidates
![Page 16: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/16.jpg)
Generate Large Scale Mooney Dataset
two-tone candidates
Mooney face classifier
![Page 17: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/17.jpg)
two-tone candidates
Mooney face classifier
![Page 18: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/18.jpg)
Mooney Face Classifier: Architecture
[Krizhevsky, Sutskever, and Hinton, NIPS 2012]
[Szegedy, etc., CVPR 2015]
![Page 19: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/19.jpg)
Mooney Face Classifier: Negative Samples
original
![Page 20: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/20.jpg)
Mooney Face Classifier: Negative Samples
original dilated
![Page 21: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/21.jpg)
Mooney Face Classifier: Negative Samples
original dilated eroded
![Page 22: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/22.jpg)
Mooney Face Classifier: Negative Samples
original dilated eroded
inverted
![Page 23: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/23.jpg)
Mooney Face Classifier: Negative Samples
original dilated eroded
inverted negative
![Page 24: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/24.jpg)
Mooney Face Classifier: Negative Samples
original dilated eroded
inverted negative neg.+inv.
![Page 25: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/25.jpg)
Mooney Face Classifier: Negative ImageNet Samples
![Page 26: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/26.jpg)
two-tone candidates
Mooney face classifier
![Page 27: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/27.jpg)
Mooney Candidates
![Page 28: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/28.jpg)
Mooney Candidates
![Page 29: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/29.jpg)
Mooney Candidates
![Page 30: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/30.jpg)
Mooney Candidates
![Page 31: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/31.jpg)
Mooney Candidates
![Page 32: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/32.jpg)
Generate Large Scale Mooney Dataset
two-tone candidates
Mooney face classifier
![Page 33: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/33.jpg)
Images of Most Mooneyness
![Page 34: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/34.jpg)
Images of Least Mooneyness
![Page 35: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/35.jpg)
Machine Learning
Generate Large Scale Mooney Face Dataset
Predict Source Photos from Mooney Faces
[Ng, and Winkler, ICIP2014]
![Page 36: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/36.jpg)
We have a Large-Scale Mooney Face Dataset
![Page 37: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/37.jpg)
Can We Predict GrayScale Face from Mooney Images?
![Page 38: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/38.jpg)
Generative Adversarial Network (GAN)
[Goodfellow, etc., NIPS2016]
x sampled from data
x
![Page 39: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/39.jpg)
Generative Adversarial Network (GAN)
[Goodfellow, etc., NIPS2016]
x sampled from data
differentiable function D
D(x) tries to be 1
x
![Page 40: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/40.jpg)
Generative Adversarial Network (GAN)
[Goodfellow, etc., NIPS2016]
x sampled from data
differentiable function D
D(x) tries to be 1
x
input noise z
![Page 41: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/41.jpg)
Generative Adversarial Network (GAN)
[Goodfellow, etc., NIPS2016]
x sampled from data
differentiable function D
D(x) tries to be 1
x differentiable function G
input noise z
![Page 42: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/42.jpg)
Generative Adversarial Network (GAN)
[Goodfellow, etc., NIPS2016]
x sampled from data
differentiable function D
D(x) tries to be 1
x
G(z) sampled from model
differentiable function G
input noise z
G(z)
![Page 43: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/43.jpg)
Generative Adversarial Network (GAN)
[Goodfellow, etc., NIPS2016]
x sampled from data
differentiable function D
D(x) tries to be 1
x
G(z) sampled from model
D
D tries to make D(G(z)) near 0; G tries to make D(G(z)) near 1
differentiable function G
input noise z
G(z)G and D are playing games:G tries to generate fake images that fool DD tries to identify the fakes
![Page 44: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/44.jpg)
Generative Adversarial Network (GAN)
x sampled from data
differentiable function D
D(x) tries to be 1
x
G(z) sampled from model
D
D tries to make D(G(z)) near 0; G tries to make D(G(z)) near 1
differentiable function G
input noise z
G(z)
[Goodfellow, etc., NIPS2016]
G and D are playing games:G tries to generate fake images that fool DD tries to identify the fakes
![Page 45: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/45.jpg)
Predict GrayScale Face by GAN
Positive Examples Negative Examples
D
G
D
Real or fake pair? Real or fake pair?
[Isola, etc., CVPR2017]
Split dataset into training and testing set by identityFaces in testing are never seen to G and D
![Page 46: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/46.jpg)
Prediction Resultsinput
![Page 47: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/47.jpg)
Prediction Resultsinput CCA-1600
![Page 48: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/48.jpg)
Prediction Resultsinput CCA-1600 CGAN
![Page 49: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/49.jpg)
Prediction Resultsinput CCA-1600 CGAN Source
![Page 50: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/50.jpg)
Prediction Resultsinput CGAN input CGAN input CGAN
![Page 51: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/51.jpg)
Machine Learning
Generate Large Scale Mooney Face Dataset
Predict Source Photos from Mooney Faces
[Ng, and Winkler, ICIP2014]
![Page 52: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/52.jpg)
Now We can predict source photos
![Page 53: Mooney Faces from Photos - ICSIstellayu/publication/doc/2017...Predict Source Photos from Mooney Faces [Ng, and Winkler, ICIP2014] Now We can predict source photos Use our code and](https://reader033.fdocuments.in/reader033/viewer/2022053119/60a0dbc4f08cd977214692fd/html5/thumbnails/53.jpg)
Use our code and download dataset from https://mooney.icsi.berkeley.edu