Interactive Images: Cuboid Proxies for Smart Image Manipulation
description
Transcript of Interactive Images: Cuboid Proxies for Smart Image Manipulation
![Page 1: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/1.jpg)
Interactive Images: Cuboid Proxies for Smart Image Manipulation
Youyi Zheng, Xiang Chen, Mingming Cheng, Kun Zhou, Shi-Min Hu, Niloy J. Mitra
Presenter: Jiapei ZHANG
![Page 2: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/2.jpg)
Outline
• Coupled 2D-3D Analysis 2D pixels 3D proxies• Image Decomposition Background + Textured Proxies • Interactive Editing
![Page 3: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/3.jpg)
Coupled 2D-3D Analysis
![Page 4: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/4.jpg)
Initial camera calibration( )i iR CH R Grab Cut Six corners
iR
Corner points[ | ]j j
ip K R t P
6 correspondence pairsj j
iP pLinear program
1 2, , , ,K R t l lCamera calibration
![Page 5: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/5.jpg)
Initial Scene EstimationTriplet of adjacent edges Candidate cuboid
For each candidate cuboid in the set {Pi}
Joint Optimization minimize the total fitting error
![Page 6: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/6.jpg)
Stacking & Occlusion
BAA B
![Page 7: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/7.jpg)
Resolving Ambiguities
( , )
( ) ( , )l i i i i j ji i j
E u R l c R l R l
MRF formulation
![Page 8: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/8.jpg)
Non-convex objectsCutting plane identification Redundancy removal
Greedy Algorithm
Choose the one covers the largest
area of Ri
![Page 9: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/9.jpg)
Image Decomposition
![Page 10: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/10.jpg)
Light & Shadow
User-marked shadow correspondence
Estimate a single light position (in a least squares sense)such that
,i i ifor each R l S
( , )i p if P l S
![Page 11: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/11.jpg)
Decomposition• Remove projected proxy and shadows• Image completion• For each proxy: (i) shadow effects based on light position (ii) symmetric regions to fill in missing texture for invisible faces
![Page 12: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/12.jpg)
Proxy-Based Edits
![Page 13: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/13.jpg)
Basic Operation
• Extracted proxies + Relations• Relations:
• Operations:
placement, repetition, alignment, coplanarity, repetition, parallelism
translate, rotate, scale, stack
![Page 14: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/14.jpg)
User-Indicated Relation
hinge joint
slide inside mark
![Page 15: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/15.jpg)
Repeated Edit
![Page 16: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/16.jpg)
Virtual Shopping
![Page 17: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/17.jpg)
User Study
Original Images Editing ResultsV.S.
Users recognized: 63.2% real images 44.5% fake images
![Page 18: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/18.jpg)
Conclusion
![Page 19: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/19.jpg)
2D 3D 2D
![Page 20: Interactive Images: Cuboid Proxies for Smart Image Manipulation](https://reader035.fdocuments.in/reader035/viewer/2022062814/568166fb550346895ddb5db5/html5/thumbnails/20.jpg)
Thank You