Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1,...
Transcript of Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1,...
![Page 1: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/1.jpg)
Single Image Portrait Relighting
Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph Rhemann3, Jay Busch3,
Paul Debevec3, Ravi Ramamoorthi1
1University of California, San Diego, 2Google Research, 3Google
![Page 2: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/2.jpg)
Photography & RecordingAllowed
![Page 3: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/3.jpg)
Overview
![Page 4: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/4.jpg)
Overview
light from the back
![Page 5: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/5.jpg)
Overview
light from the back shadow on the face
![Page 6: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/6.jpg)
Overview
want to add dramatic lightinglight from the back shadow on the face
![Page 7: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/7.jpg)
Overview
want to add dramatic lightinglight from the back shadow on the face
Change the lighting of any portrait after capture using post-processing algorithm
![Page 8: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/8.jpg)
Overview
![Page 9: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/9.jpg)
Overview
![Page 10: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/10.jpg)
Portrait Relighting System
Overview
![Page 11: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/11.jpg)
Portrait Relighting System
Overview
another lighting
![Page 12: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/12.jpg)
Portrait Relighting System
Overview
another lighting
![Page 13: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/13.jpg)
Overview
Portrait Relighting System
![Page 14: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/14.jpg)
Overview
Portrait Relighting System
![Page 15: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/15.jpg)
Overview
Portrait Relighting System
I want to rotate the lighting a little bit.
![Page 16: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/16.jpg)
Overview
Portrait Relighting System
I want to rotate the lighting a little bit.
![Page 17: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/17.jpg)
Previous work
![Page 18: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/18.jpg)
Previous work• Light Stage
Debevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 19: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/19.jpg)
Previous work• Light Stage
Debevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 20: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/20.jpg)
Previous work• Light Stage
Debevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 21: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/21.jpg)
Previous work• Light Stage
Debevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
: capture ~100 images and do image-based relighting
![Page 22: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/22.jpg)
Previous work
![Page 23: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/23.jpg)
• Deep image-based relighting
Xu, Zexiang, et al. "Deep image-based relighting from optimal sparse samples." SIGGRAPH 2018
Previous work
![Page 24: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/24.jpg)
• Deep image-based relighting
Xu, Zexiang, et al. "Deep image-based relighting from optimal sparse samples." SIGGRAPH 2018
Previous work
![Page 25: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/25.jpg)
• Deep image-based relighting
Xu, Zexiang, et al. "Deep image-based relighting from optimal sparse samples." SIGGRAPH 2018
Previous work
![Page 26: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/26.jpg)
• Deep image-based relighting
Xu, Zexiang, et al. "Deep image-based relighting from optimal sparse samples." SIGGRAPH 2018
capture 5 images and do relighting via neural network
Previous work
![Page 27: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/27.jpg)
Previous work
![Page 28: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/28.jpg)
• Portrait lighting transfer
Shu, Zhixin, et al. "Portrait lighting transfer using a mass transport approach." SIGGRAPH 2018
Previous work
![Page 29: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/29.jpg)
• Portrait lighting transfer
Shu, Zhixin, et al. "Portrait lighting transfer using a mass transport approach." SIGGRAPH 2018
Previous work
![Page 30: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/30.jpg)
• Portrait lighting transfer
Shu, Zhixin, et al. "Portrait lighting transfer using a mass transport approach." SIGGRAPH 2018
transfer lighting from one portrait to another
Previous work
![Page 31: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/31.jpg)
Previous work
![Page 32: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/32.jpg)
Normal
Shading
Albedo
Relit
• SfSNet
Sengupta, Soumyadip, et al. "SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild'." CVPR 2018.
Previous work
![Page 33: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/33.jpg)
Normal
Shading
Albedo
Relit
• SfSNet
Sengupta, Soumyadip, et al. "SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild'." CVPR 2018.
: deep intrinsic decomposition mostly trained on synthetic faces
Previous work
![Page 34: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/34.jpg)
Normal
Shading
Albedo
Relit
• SfSNet
Sengupta, Soumyadip, et al. "SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild'." CVPR 2018.
: deep intrinsic decomposition mostly trained on synthetic faces
Previous work
![Page 35: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/35.jpg)
Overview
![Page 36: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/36.jpg)
• Goal: practical relighting on single portrait image
Overview
![Page 37: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/37.jpg)
• Goal: practical relighting on single portrait image
• Practical in detail:
• Robust to the pose and camera view
• Work well on natural lightings
• Adapt to high-resolution images
• Run at interactive rate
Overview
![Page 38: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/38.jpg)
• Goal: practical relighting on single portrait image
• Practical in detail:
• Robust to the pose and camera view
• Work well on natural lightings
• Adapt to high-resolution images
• Run at interactive rate
• Solution: Deep Neural Network + Real Face Data.
Overview
![Page 39: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/39.jpg)
Method
![Page 40: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/40.jpg)
Method
portrait under lighting A
![Page 41: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/41.jpg)
Method
Portrait Relighting System
(Neural Network)portrait under
lighting A
![Page 42: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/42.jpg)
Method
Portrait Relighting System
(Neural Network)portrait under
lighting Alighting B
![Page 43: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/43.jpg)
Method
Portrait Relighting System
(Neural Network)portrait under
lighting Aportrait under
lighting Blighting B
![Page 44: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/44.jpg)
Method
Portrait Relighting System
(Neural Network)portrait under
lighting Aportrait under
lighting Blighting A lighting B
![Page 45: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/45.jpg)
Method
Portrait Relighting System
(Neural Network)portrait under
lighting Aportrait under
lighting Blighting A lighting B
![Page 46: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/46.jpg)
Method
Portrait Relighting System
(Neural Network)portrait under
lighting Aportrait under
lighting Blighting A lighting B
![Page 47: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/47.jpg)
Method
Portrait Relighting System
(Neural Network)portrait under
lighting Aportrait under
lighting Blighting A lighting B
![Page 48: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/48.jpg)
Method
Portrait Relighting System
(Neural Network)portrait under
lighting Aportrait under
lighting Blighting A lighting B
How can we get the portrait pair for training?
![Page 49: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/49.jpg)
Method: Data
![Page 50: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/50.jpg)
Method: DataLight Stage
![Page 51: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/51.jpg)
Method: DataLight Stage One-Light-At-a-Time scans (OLAT)
![Page 52: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/52.jpg)
Method: Data
![Page 53: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/53.jpg)
Method: Data
Debevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 54: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/54.jpg)
Method: Data
captured OLAT captured OLATDebevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 55: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/55.jpg)
lighting
Method: Data
captured OLAT captured OLATDebevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 56: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/56.jpg)
lighting
Method: Data
captured OLAT captured OLAT
latitude-longitude representation
Debevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 57: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/57.jpg)
lighting
Method: Data
captured OLAT captured OLAT
latitude-longitude representation
Debevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 58: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/58.jpg)
lighting
Method: Data
captured OLAT captured OLAT
latitude-longitude representation
Debevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 59: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/59.jpg)
lighting
x x+ + …… =
Method: Data
captured OLAT captured OLAT
latitude-longitude representation
Debevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 60: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/60.jpg)
lighting
x x+ + …… =
Method: Data
captured OLAT captured OLAT relit image (background removed)
latitude-longitude representation
Debevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 61: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/61.jpg)
lighting
x x+ + …… =
Method: Data
captured OLAT captured OLAT relit image (background removed)
latitude-longitude representation
Wadhwa, Neal, et al. "Synthetic depth-of-field with a single-camera mobile phone." SIGGRAPH 2018Debevec, Paul, et al. "Acquiring the reflectance field of a human face." SIGGRAPH 2000.
![Page 62: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/62.jpg)
Method: Data
![Page 63: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/63.jpg)
• OLAT images• 22 people (18 training, 4 validation), each 3~5 facial expressions
Method: Data
![Page 64: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/64.jpg)
• OLAT images• 22 people (18 training, 4 validation), each 3~5 facial expressions
• Each OLAT captured with 7 cameras in 6 seconds.
Method: Data
![Page 65: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/65.jpg)
• OLAT images• 22 people (18 training, 4 validation), each 3~5 facial expressions
• Each OLAT captured with 7 cameras in 6 seconds.
• HDR lighting environments
Method: Data
![Page 66: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/66.jpg)
• OLAT images• 22 people (18 training, 4 validation), each 3~5 facial expressions
• Each OLAT captured with 7 cameras in 6 seconds.
• HDR lighting environments• ~2000 indoor HDR lighting
from Laval Dataset
Method: Data
![Page 67: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/67.jpg)
• OLAT images• 22 people (18 training, 4 validation), each 3~5 facial expressions
• Each OLAT captured with 7 cameras in 6 seconds.
• HDR lighting environments• ~2000 indoor HDR lighting
from Laval Dataset
• ~1000 outdoor HDR lighting from the web
Method: Data
![Page 68: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/68.jpg)
• OLAT images• 22 people (18 training, 4 validation), each 3~5 facial expressions
• Each OLAT captured with 7 cameras in 6 seconds.
• HDR lighting environments• ~2000 indoor HDR lighting
from Laval Dataset
• ~1000 outdoor HDR lighting from the web
Method: Data
• Total: 226,800 portrait and lighting pairs for training
![Page 69: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/69.jpg)
Method: Training
![Page 70: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/70.jpg)
Method: Training
Encoder DecoderBottleneck
![Page 71: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/71.jpg)
Method: Training
Encoder DecoderBottleneck
• Task 1: Complete relighting
![Page 72: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/72.jpg)
Method: Training
Encoder DecoderBottleneck
source image
• Task 1: Complete relighting
![Page 73: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/73.jpg)
Method: Training
Encoder DecoderBottleneck
source light
source image
• Task 1: Complete relighting
![Page 74: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/74.jpg)
Method: Training
Encoder DecoderBottleneck
source light target light
source image
• Task 1: Complete relighting
![Page 75: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/75.jpg)
Method: Training
Encoder DecoderBottleneck
source light target light
source image target image
• Task 1: Complete relighting
![Page 76: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/76.jpg)
Method: Training
Encoder DecoderBottleneck
source light target light
source image target image
• Task 1: Complete relighting L1 loss
Log L1 loss
![Page 77: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/77.jpg)
Method: Training
Encoder DecoderBottleneck
![Page 78: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/78.jpg)
Method: Training• Task 2: Illumination retargeting
Encoder DecoderBottleneck
![Page 79: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/79.jpg)
Method: Training• Task 2: Illumination retargeting
Encoder DecoderBottleneck
source image
![Page 80: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/80.jpg)
Method: Training• Task 2: Illumination retargeting
Encoder DecoderBottleneck
source light
source image
![Page 81: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/81.jpg)
Method: Training• Task 2: Illumination retargeting
Encoder DecoderBottleneck
source light
source image
Rotate
![Page 82: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/82.jpg)
Method: Training• Task 2: Illumination retargeting
Encoder DecoderBottleneck
source light target light
source image
Rotate
![Page 83: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/83.jpg)
Method: Training• Task 2: Illumination retargeting
Encoder DecoderBottleneck
source light target light
source image target image
Rotate
![Page 84: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/84.jpg)
Method: Training• Task 2: Illumination retargeting
Encoder DecoderBottleneck
source light target light
source image target image
Rotate
L1 loss
Log L1 loss
![Page 85: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/85.jpg)
Method: Training
![Page 86: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/86.jpg)
Method: Training• Network structure• U-Net
k x k conv layer concatenation
kweighted average tiling
kk-dimensional
input/label
kk-dimensional
activation
256 x 256 128 x 128 64 x 64 32 x 32 16 x 16Spatial Resolution:
loss
![Page 87: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/87.jpg)
Method: Training• Network structure• U-Net
k x k conv layer concatenation
kweighted average tiling
kk-dimensional
input/label
kk-dimensional
activation
256 x 256 128 x 128 64 x 64 32 x 32 16 x 16Spatial Resolution:
loss
SourceImage 64 643 128 1283 256 2563 512 5123 5123 5123
OutputSourceLight
16 x 32 x 33
297 16 x 32 x 13 3 3 3 33
TrueSourceLight
16x
32x3
16x
32x3
![Page 88: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/88.jpg)
Method: Training• Network structure• U-Net• Predict and feed in light at bottleneck
k x k conv layer concatenation
kweighted average tiling
kk-dimensional
input/label
kk-dimensional
activation
256 x 256 128 x 128 64 x 64 32 x 32 16 x 16Spatial Resolution:
loss
SourceImage 64 643 128 1283 256 2563 512 5123 5123 5123
OutputSourceLight
16 x 32 x 33
297 16 x 32 x 13 3 3 3 33
TrueSourceLight
16x
32x3
16x
32x3
![Page 89: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/89.jpg)
Method: Training• Network structure• U-Net• Predict and feed in light at bottleneck
confidence learning module
k x k conv layer concatenation
kweighted average tiling
kk-dimensional
input/label
kk-dimensional
activation
256 x 256 128 x 128 64 x 64 32 x 32 16 x 16Spatial Resolution:
loss
SourceImage 64 643 128 1283 256 2563 512 5123 5123 5123
OutputSourceLight
16 x 32 x 33
297 16 x 32 x 13 3 3 3 33
TrueSourceLight
16x
32x3
16x
32x3
![Page 90: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/90.jpg)
Method: Training• Network structure• U-Net• Predict and feed in light at bottleneck
confidence learning module
k x k conv layer concatenation
kweighted average tiling
kk-dimensional
input/label
kk-dimensional
activation
256 x 256 128 x 128 64 x 64 32 x 32 16 x 16Spatial Resolution:
loss
SourceImage 64 643 128 1283 256 2563 512 5123 5123 5123
OutputSourceLight
16 x 32 x 33
297 16 x 32 x 13 3 3 3 33
TrueSourceLight
16x
32x3
16x
32x3
3 512TargetLight3
32
323
512
256
512
3
512
512
256
3
256128
3
12864
3
64
51225625612812864643 3 3 3
16x
32x3
TrueTargetImage
OutputTargetImage
3
3
![Page 91: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/91.jpg)
Method: Training
![Page 92: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/92.jpg)
Method: Training• Confidence learning
![Page 93: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/93.jpg)
Method: Training• Confidence learning
![Page 94: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/94.jpg)
Method: Training
Several conv layers
• Confidence learning
![Page 95: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/95.jpg)
Method: Training
Several conv layers
• Confidence learning
![Page 96: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/96.jpg)
Method: Training
Several conv layers
resolution of the light
• Confidence learning
![Page 97: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/97.jpg)
Method: Training
Several conv layers
resolution of the light
Light prediction on each image patch
• Confidence learning
![Page 98: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/98.jpg)
Method: Training
Several conv layers
resolution of the light
Light prediction on each image patch
Confidence of prediction on each
image patch
• Confidence learning• Predict the confidence of light prediction
![Page 99: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/99.jpg)
Method: Training
Several conv layers
resolution of the light
Light prediction on each image patch
Confidence of prediction on each
image patch
* =
• Confidence learning• Predict the confidence of light prediction
Reshape
![Page 100: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/100.jpg)
Method: Training
Several conv layers
resolution of the light
Light prediction on each image patch
Confidence of prediction on each
image patch
* =
• Confidence learning• Predict the confidence of light prediction• Allow network to say “I don’t know”
Reshape
![Page 101: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/101.jpg)
Results: Validation
![Page 102: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/102.jpg)
Results: Validation➤ Relit images for complete relighting
![Page 103: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/103.jpg)
Results: Validation➤ Relit images for complete relighting
source image
![Page 104: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/104.jpg)
Results: Validation➤ Relit images for complete relighting
source image
![Page 105: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/105.jpg)
Results: Validation➤ Relit images for complete relighting
source image
![Page 106: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/106.jpg)
Results: Validation➤ Relit images for complete relighting
source image target image
![Page 107: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/107.jpg)
Results: Validation➤ Relit images for complete relighting
source image target image ours [Barron & Malik 2015][Sengupta et al. 2018] [Li et al. 2018]
![Page 108: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/108.jpg)
Results: Validation➤ Relit images for complete relighting
source image target image ours [Barron & Malik 2015][Sengupta et al. 2018] [Li et al. 2018]
![Page 109: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/109.jpg)
Results: Validation
![Page 110: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/110.jpg)
Results: Validation
![Page 111: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/111.jpg)
Results: Validation
![Page 112: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/112.jpg)
Results: Validation
➤ Relight images with predicted light as target light
Encoder DecoderBottleneck
![Page 113: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/113.jpg)
Results: Validation
➤ Relight images with predicted light as target light
source image
Encoder DecoderBottleneck
![Page 114: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/114.jpg)
Results: Validation
➤ Relight images with predicted light as target light
source image with self-supervision
withoutself-supervision
Encoder DecoderBottleneck
![Page 115: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/115.jpg)
Results: Validation
![Page 116: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/116.jpg)
Results: Validation
![Page 117: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/117.jpg)
Results: Validation
![Page 118: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/118.jpg)
Results: Validation➤ Comparison with portrait lighting transfer
![Page 119: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/119.jpg)
Results: Validation➤ Comparison with portrait lighting transfer
reference image
![Page 120: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/120.jpg)
Results: Validation➤ Comparison with portrait lighting transfer
source image
reference image
![Page 121: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/121.jpg)
Results: Validation➤ Comparison with portrait lighting transfer
source image
reference image
Extract light from reference
![Page 122: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/122.jpg)
Results: Validation➤ Comparison with portrait lighting transfer
source image
reference image
Extract light from reference
Apply to source image
![Page 123: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/123.jpg)
Results: Validation➤ Comparison with portrait lighting transfer
source image
groundtruth oursreference image
Extract light from reference
Apply to source image
![Page 124: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/124.jpg)
Results: Validation➤ Comparison with portrait lighting transfer
source image
groundtruth ours
[Shih et al. 2014] [Shu et al. 2018]
reference image
Extract light from reference
Apply to source image
![Page 125: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/125.jpg)
Results: Validation
![Page 126: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/126.jpg)
Results: Validation➤ Evaluation on lighting prediction
![Page 127: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/127.jpg)
Results: Validation➤ Evaluation on lighting prediction
source image
ground truth
![Page 128: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/128.jpg)
Results: Validation➤ Evaluation on lighting prediction
source image
ground truth
ours
ours w/o confidence learning
![Page 129: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/129.jpg)
Results: Validation➤ Evaluation on lighting prediction
source image
ground truth
ours
ours w/o confidence learning
[Barron & Malik 2015]
[Sengupta et al. 2018]
![Page 130: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/130.jpg)
Results: Images in the wild
![Page 131: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/131.jpg)
Results: Images in the wildInput Image Relit Image
![Page 132: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/132.jpg)
Results: Images in the wildInput Image Relit Image
![Page 133: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/133.jpg)
Results: Images in the wildInput Image Relit ImageInput Image Relit Image
![Page 134: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/134.jpg)
Results: Images in the wildInput Image Relit ImageInput Image Relit Image
![Page 135: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/135.jpg)
Results: Images in the wild
![Page 136: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/136.jpg)
Results: Images in the wild
![Page 137: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/137.jpg)
Results: Images in the wild
![Page 138: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/138.jpg)
Results: Images in the wild
![Page 139: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/139.jpg)
Limitations
![Page 140: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/140.jpg)
LimitationsInput Image Relit Image
• Complex shadows
• Specular highlights
• Overexposed pixels
![Page 141: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/141.jpg)
LimitationsInput Image Relit Image
• Complex shadows
• Specular highlights
• Overexposed pixels
• Over-smoothing
![Page 142: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/142.jpg)
LimitationsInput Image Relit Image
• Complex shadows
• Specular highlights
• Overexposed pixels
• Over-smoothing
• Unseen high-saturation color
![Page 143: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/143.jpg)
Conclusion
![Page 144: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/144.jpg)
Conclusion
• Learn the relighting function on portraits using Light Stage data
![Page 145: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/145.jpg)
Conclusion
• Learn the relighting function on portraits using Light Stage data
• Take home message:
![Page 146: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/146.jpg)
Conclusion
• Learn the relighting function on portraits using Light Stage data
• Take home message:
• For human faces, use real data.
![Page 147: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/147.jpg)
Conclusion
• Learn the relighting function on portraits using Light Stage data
• Take home message:
• For human faces, use real data.
• End-to-end training vs assuming models.
![Page 148: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/148.jpg)
Acknowledgement• This work was funded in part by a Jacobs Fellowship, the Ronald L.
Graham Chair, and the UC San Diego Center for Visual Computing.
• Thanks to Zhixin Shu and Yichang Shih for the help on baseline algorithms
• Thanks to Jean-François Lalonde for providing the indoor lighting dataset.
• Thanks to Peter Denny for coordinating dataset capture.
• Thanks to all the anonymous volunteers in the dataset from Google, UCSD and UCLA
![Page 149: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/149.jpg)
Also presenting in poster session today at 12:15-1:15.
Single Image Portrait Relighting
![Page 150: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/150.jpg)
Single Image Portrait Relighting
input imagegenerated portraits under new illuminations
![Page 151: Single Image Portrait Relighting™天成.pdf · Single Image Portrait Relighting Tiancheng Sun1, Jonathan T. Barron2, Yun-Ta Tsai2, Zexiang Xu1, Xueming Yu3, Graham Fyffe3, Christoph](https://reader034.fdocuments.in/reader034/viewer/2022051908/5ffb45871bf337768d28c48a/html5/thumbnails/151.jpg)
Single Image Portrait Relighting
input imagegenerated portraits under new illuminations