IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I...
Transcript of IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I...
![Page 1: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/1.jpg)
IMAGE COMPLETIONdeceivingly simple, impressive results
![Page 2: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/2.jpg)
WHAT IS IT?
‣ fill in a missing part of an image
not an actual result
![Page 3: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/3.jpg)
MANY NAMES
‣ image completion
‣ in-painting
‣ object removal
‣ image restoration
‣ filling-in
‣ error concealment
‣ image extrapolation
‣ we'll see why in later examples
![Page 4: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/4.jpg)
TEXTURE SYNTHESISa detour?
![Page 5: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/5.jpg)
WHAT IS IT?
‣ produce a similar texture starting from a sample
not an actual result
![Page 6: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/6.jpg)
![Page 7: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/7.jpg)
INSPIRATION
‣ A Mathematical Theory of Communication – Shannon 1948
‣ build n-grams probabilities table from a book
‣ eg:
- go is often followed by home
- how, are, you are frequently encountered together
‣ sample repeatedly to create sentences
![Page 8: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/8.jpg)
GENERATE SENTENCES
bake cake go home will
bake - 71 - - -
cake - - - 12 -
go 6 - 3 49 -
home 7 24 - - -
will 53 - 14 - -
![Page 9: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/9.jpg)
GENERATE SENTENCES
bake cake go home will
bake - 71 - - -
cake - - - 12 -
go 6 - 3 49 -
home 7 24 - - -
will 53 - 14 - -
will was followed by bake 53 times
![Page 10: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/10.jpg)
GENERATE SENTENCES
bake cake go home will
bake - 71 - - -
cake - - - 12 -
go 6 - 3 49 -
home 7 24 - - -
will 53 - 14 - -
will was followed by bake 53 times
will was followed by go 14 times
![Page 11: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/11.jpg)
GENERATE SENTENCES
‣ finish this: Today I will ...
‣
bake cake go home will
bake - 71 - - -
cake - - - 12 -
go 6 - 3 49 -
home 7 24 - - -
will 53 - 14 - -
will was followed by bake 17 times
will was followed by go 64 times
![Page 12: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/12.jpg)
GENERATE SENTENCES
‣ finish this: Today I will ...
‣
bake cake go home will
bake - 71 - - -
cake - - - 12 -
go 6 - 3 49 -
home 7 24 - - -
will 53 - 14 - -
will was followed by bake 17 times
will was followed by go 64 times
![Page 13: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/13.jpg)
GENERATE SENTENCES
‣ finish this: Today I will ...
‣ bake
bake cake go home will
bake - 71 - - -
cake - - - 12 -
go 6 - 3 49 -
home 7 24 - - -
will 53 - 14 - -
will was followed by bake 17 times
will was followed by go 64 times
![Page 14: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/14.jpg)
GENERATE SENTENCES
‣ finish this: Today I will ...
‣ bake
‣
bake cake go home will
bake - 71 - - -
cake - - - 12 -
go 6 - 3 49 -
home 7 24 - - -
will 53 - 14 - -
will was followed by bake 17 times
will was followed by go 64 times
![Page 15: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/15.jpg)
GENERATE SENTENCES
‣ finish this: Today I will ...
‣ bake
‣ cake
bake cake go home will
bake - 71 - - -
cake - - - 12 -
go 6 - 3 49 -
home 7 24 - - -
will 53 - 14 - -
will was followed by bake 17 times
will was followed by go 64 times
![Page 16: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/16.jpg)
GENERATE SENTENCES
‣ finish this: Today I will ...
‣ bake
‣ cake
‣
bake cake go home will
bake - 71 - - -
cake - - - 12 -
go 6 - 3 49 -
home 7 24 - - -
will 53 - 14 - -
will was followed by bake 17 times
will was followed by go 64 times
![Page 17: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/17.jpg)
GENERATE SENTENCES
‣ finish this: Today I will ...
‣ bake
‣ cake
‣ home
bake cake go home will
bake - 71 - - -
cake - - - 12 -
go 6 - 3 49 -
home 7 24 - - -
will 53 - 14 - -
will was followed by bake 17 times
will was followed by go 64 times
![Page 18: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/18.jpg)
GENERATE SENTENCES
‣ finish this: Today I will ...
‣ bake
‣ cake
‣ home
‣ Today I will bake a cake at home.
‣ alternative: Today I will go home.
bake cake go home will
bake - 71 - - -
cake - - - 12 -
go 6 - 3 49 -
home 7 24 - - -
will 53 - 14 - -
will was followed by bake 17 times
will was followed by go 64 times
![Page 19: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/19.jpg)
‣ generalize to 2D
- unit: letter ~ pixel
- context: n-gram ~ texel
‣ eg:
- red often follows red
- grey seldom after red
‣ find missing pixel color in probability table, based on neighbors
CORE IDEA
![Page 20: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/20.jpg)
CONTINUE THE TEXTURE
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 21: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/21.jpg)
CONTINUE THE TEXTURE
X X
X
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 22: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/22.jpg)
CONTINUE THE TEXTURE
X X X
X
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 23: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/23.jpg)
CONTINUE THE TEXTURE
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 24: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/24.jpg)
CONTINUE THE TEXTURE
?
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 25: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/25.jpg)
CONTINUE THE TEXTURE
?
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 26: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/26.jpg)
CONTINUE THE TEXTURE
!
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 27: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/27.jpg)
CONTINUE THE TEXTURE
?
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 28: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/28.jpg)
CONTINUE THE TEXTURE
?
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 29: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/29.jpg)
CONTINUE THE TEXTURE
!
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 30: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/30.jpg)
CONTINUE THE TEXTURE
?
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 31: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/31.jpg)
CONTINUE THE TEXTURE
?
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 32: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/32.jpg)
CONTINUE THE TEXTURE
!
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 33: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/33.jpg)
CONTINUE THE TEXTURE
?
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 34: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/34.jpg)
CONTINUE THE TEXTURE
?
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 35: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/35.jpg)
CONTINUE THE TEXTURE
!
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
with some stochasticity
![Page 36: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/36.jpg)
CONTINUE THE TEXTURE
?
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 37: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/37.jpg)
CONTINUE THE TEXTURE
?
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
![Page 38: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/38.jpg)
CONTINUE THE TEXTURE
!
red grey
left red, above red 3 0
left red, above grey 3 4
left grey, above red 2 4
left grey, above grey 1 2
small mistake
![Page 39: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/39.jpg)
BUT THERE'S A PROBLEM!
‣ words are categorical (finite & unordered), pixels are continuous
- discretization leads to millions of colors
- info about red should also (partially) apply to dark red
‣ building a probability table is unfeasible
- gets exponentially larger when considering more neighbors
- one image is not exhaustive, exact match might not be present
‣ solution: mimic table by looking up directly in the image
- don't store anything, search every time
![Page 40: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/40.jpg)
ALGORITHM
‣ input: image and hole position
1. pick a pixel on the hole's edge empty pixels having at least one filled neighbor
2. select a patch around it
3. find the most similar patch to it using sum of squared distances
4. replace pixel with found patch center
5. repeat until the hole is filled
‣output: filled image
![Page 41: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/41.jpg)
ALGORITHM
source: Texture Synthesis Paper presentation
![Page 42: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/42.jpg)
ONE LAST CONSIDERATION
‣ texture types:
- regular (repeating)
- stochastic
- combination
‣ add stochasticity by not always picking the closest match
![Page 43: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/43.jpg)
ALGORITHM PARAMETERS
‣ patch size: texel size (how often texture repeats)
‣ selection std: texture randomness (picking most similar)
LARG
E
SMAL
LLIT
TLE
A LOT
![Page 44: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/44.jpg)
EXAMPLES
not run by me
![Page 45: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/45.jpg)
![Page 46: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/46.jpg)
![Page 47: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/47.jpg)
![Page 48: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/48.jpg)
DEMO
![Page 49: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/49.jpg)
PRACTICAL CONSIDERATIONS
‣ to speed up computation:
- replace the whole patch instead of the center pixel
- restrict searching from whole image to a smaller area
![Page 50: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/50.jpg)
›
![Page 51: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/51.jpg)
ORIGINAL
![Page 52: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/52.jpg)
REMOVED
![Page 53: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/53.jpg)
THAT WAS NOT A TEXTURE!
‣ turns out texture synthesis was not a detour
‣ photos are made up of many "textures"
‣ the method can be used for many tasks
![Page 54: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/54.jpg)
MORE EXAMPLES
not run by me, various methods
![Page 55: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/55.jpg)
![Page 56: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/56.jpg)
NOT ACTUAL WORDS!
WORDS
![Page 57: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/57.jpg)
ORIGINAL
![Page 58: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/58.jpg)
EXPANDED
![Page 59: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/59.jpg)
IMAGE RESTORATION
DAMAGED MASK RESTORED
![Page 60: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/60.jpg)
WRINKLE REMOVAL
ORIGINAL MASK REMOVED
![Page 61: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/61.jpg)
OBJECT REMOVAL
ORIGINAL MASK REMOVED
![Page 62: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/62.jpg)
OBJECT REMOVAL
ORIGINAL MASK REMOVED
![Page 63: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/63.jpg)
OBJECT REMOVAL
![Page 64: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/64.jpg)
![Page 65: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/65.jpg)
TEXT REMOVAL
![Page 66: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/66.jpg)
METHOD COMPARISON
![Page 67: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/67.jpg)
ORIGINAL
![Page 68: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/68.jpg)
REMOVED
![Page 69: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/69.jpg)
CONTENT-AWARE RESIZEsource: FMI UB CV lecture 4
![Page 70: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/70.jpg)
IN-PAINTING RESIZING
![Page 71: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/71.jpg)
LIMITATIONS
![Page 72: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/72.jpg)
ORIGINAL
![Page 73: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/73.jpg)
FAILURE
![Page 74: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/74.jpg)
SUMMARY
![Page 75: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/75.jpg)
TEXTURE SYNTHESIS
‣ fundamentally sound
‣ conceptually simple
‣ widely applicable
‣ not very robust
‣ pretty slow
![Page 76: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/76.jpg)
REFERENCES
‣ Texture Synthesis by Non-Parametric Sampling, A. Efros & T. Leung
‣ Inpainting Methods Survey, M. Bertalmio et al
‣ Assignment 4 – CPSC 425: Computer Vision, University of British Colombia
‣ many published example runs on various image inputs
![Page 77: IMAGE COMPLETION - stefann.eu completion.pdf · GENERATE SENTENCES ‣ finish this: Today I will... ‣ bake ‣ cake ‣ home ‣ Today I will bake a cake at home. ‣ alternative:](https://reader033.fdocuments.in/reader033/viewer/2022050608/5fafb14dd4dd4a06b43ef6d0/html5/thumbnails/77.jpg)
THANK YOU!