Drag-and-drop Pasting
description
Transcript of Drag-and-drop Pasting
![Page 1: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/1.jpg)
Drag-and-drop Pasting
By Chui Sung Him, Gary
Supervised by Prof. Chi-keung Tang
![Page 2: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/2.jpg)
Outline
Background Objectives Techniques Results & extended application Demo
![Page 3: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/3.jpg)
Background
Seamless object cloning Traditional method
– User interaction– Time– Expertise
![Page 4: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/4.jpg)
Objectives
Reduce user-interaction Suppress unnatural look automatically Optimize boundary to achieve the above
objectives
![Page 5: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/5.jpg)
Techniques
User provide rough region of interest (RoI)– Contiaining object of interest (OoI)– Drag-and-drop to the target
Optimization problem
Euler-Lagrange equation Poisson equation
|*| with min
2fff
fv
|*| with ,over div fff v
Ω
Ωobj
f*
![Page 6: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/6.jpg)
Problem
![Page 7: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/7.jpg)
Objectives
Reduce user-interaction Suppress unnatural look automatically Optimize boundary
![Page 8: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/8.jpg)
User provides only rough RoI Assume v=∇g and let f’=f – g, reformulate opti
mization problem
Poisson equation becomes Laplace equation
Approach zero when (f*-g) = constant– find an optimal boundary to satisfy this
Techniques (Cont’d)
|)*(|' with 'min
2
'gfff
f
|*|' with ,over 0' gfff
![Page 9: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/9.jpg)
Techniques (Cont’d)
To find the optimal boundary– Inside the RoI– Outside the OoI
Define an energy function– Total color variance–
Minimize it
Ω
Ωobj
f*
objp
kpgpfkE
\ s.t. )))()(*((, 2
![Page 10: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/10.jpg)
Iterative minimization Initialize ∂Ω as boundary of RoI Given new ∂Ω, optimize E w.r.t. k
Given new k, optimize E with new ∂Ω– Shortest path problem
Until convergence reached
0),(
k
kE
p
pgpfk *1
objp
kpgpfkE
\ s.t. )))()(*((, 2
![Page 11: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/11.jpg)
Shortest path problem?
Cost of each pixel = its color variance w.r.t. new k
Path to find in closed band Ω\Ωobj
– Not a usual shortest path
A shortest closed-path problem
Ω
Ωobj
f*
![Page 12: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/12.jpg)
Shortest closed-path
Break the band with a cut– Not closed now
![Page 13: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/13.jpg)
Shortest closed-path
Perform usual shortest path algorithm on a yellow pixel– Dijkstra O(NlogN)
![Page 14: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/14.jpg)
Shortest closed-path
Perform on M yellow pixels– O(MNlogN)
![Page 15: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/15.jpg)
Selecting the cut
With minimum length M
Reduce probability of twisting path– Not to pass the cut more than once
Reduce running time (MNlogN)
![Page 16: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/16.jpg)
Results
![Page 17: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/17.jpg)
Results
![Page 18: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/18.jpg)
Result
![Page 19: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/19.jpg)
Result
![Page 20: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/20.jpg)
Extended Application
Seamless image completion A hole in an image S Another image D provided by user
– Semantically correct
Auto complete the hole
![Page 21: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/21.jpg)
Seamless Image Completion
D and S semantically agreed– Color– Scene objects
Selecting region on D to complete the hole– Sum of Squared Difference (SSD) of color– Distance to the hole on S
![Page 22: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/22.jpg)
Seamless Image completion Result
![Page 23: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/23.jpg)
Seamless Image completion Result
![Page 24: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/24.jpg)
Live Demo
![Page 25: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/25.jpg)
Q&A
![Page 26: Drag-and-drop Pasting](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815930550346895dc664be/html5/thumbnails/26.jpg)
THE END