Inverse Texture Synthesis
29
Inverse Texture Synthesis Li-Yi Wei 1 Jianwei Han 2 Kun Zhou 1,2 Hujun Bao 2 Baining Guo 1 Harry Shum 1 1 Microsoft 2 Zhejiang University
description
Inverse Texture Synthesis. Li-Yi Wei 1 Jianwei Han 2 Kun Zhou 1 , 2 Hujun Bao 2 Baining Guo 1 Harry Shum 1 1 Microsoft 2 Zhejiang University. Example-based texture synthesis. For a small input texture produce an arbitrarily large output with similar look - PowerPoint PPT Presentation
Transcript of Inverse Texture Synthesis
Inverse Texture SynthesisInverse Texture Synthesis
Li-Yi Wei1 Jianwei Han2 Kun Zhou1,2
Hujun Bao2 Baining Guo1 Harry Shum1
1Microsoft 2Zhejiang University
Example-based texture synthesisExample-based texture synthesis
For a small input texture
produce an arbitrarily large output with similar look
Why? may not possible to obtain large input
input
output
texture synthesis
Inverse texture synthesisInverse texture synthesis
From a large input texture
produce a small output that best summarizes input
input
output
inverse texture synthesis
Yale University Columbia UniversityMSR Asia
Why?Why?
Textures are getting large
Advances in scanning technology
High dimensionality: time-varying, BRDF
Expensive to store, transmit, compute
OverviewOverview
input(large)
inverse texture synthesisoutput(small)
texturing(slow)
texturing(fast)
similar quality
Related work: image compressionRelated work: image compression
compress
inverse synth texture synthinput
decompress
pixel-wiseidentical
perceptualsimilar
Related work: epitomeRelated work: epitome
Epitome [Jojic et al. 2003]
Jigsaw [Kannan et al. 2007]
Major source of inspiration for us
For general images, not just textures
We provide better quality
Bidirectional similarity [Simakov et al. 2008]
Factoring repeated content [Wang et al. 2008]
Related work: manual cropRelated work: manual crop
our resultmanual croporiginal
globallyvarying
stationary
Globally-varying texturesGlobally-varying textures
Markov Random Field (MRF) textures
local & stationary
Globally-varying textures
local, but not necessarily stationary
MRF globally varying
Globally varying texturesPrevious workGlobally varying texturesPrevious work
MRF input → globally varying output
texture-by-numbers in Image analogies [Hertzmann et al. 2001]
progressively variant textures [Zhang et al. 2003]
texture design and morphing [Matusik et al. 2005]
Globally varying input
appearance manifold [Wang et al. 2006]
spatially & time varying BRDF [Gu et al. 2006]
context-aware texture [Lu et al. 2007]
Globally varying texturesDefinitionGlobally varying texturesDefinition
texture + control maps Examples of control maps
user-specified colors [Hertzmann et al. 2001]
spatially-varying parameters [Gu et al. 2006]
weathering degree-map [Wang et al. 2006]
context information [Lu et al. 2007]
texture (paint crack)
control map (paint thickness)
Globally varying texturesGlobally varying textures
Including time-varying textures as well
Large data size!
time-varying BRDF[Gu et al. 2006]512 x 512 x 33, 288 MB
context-aware texture[Lu et al. 2007]1226 x 978 x 50, 35 MB
Inverse texture synthesisInverse texture synthesis
Compacting globally varying textures
including both texture + control map
input
texture control map
output compaction
inverse synthesis
texture control
Compaction as summary of originalCompaction as summary of original
Re-synthesis with user control map
compactionuser control
+
re-synthesisfromcompaction
forwardsynthesis
re-synthesisfromoriginal
faster slower
Basic formulationBasic formulation
Inspired by texture optimization [Kwatra et al. 2005]
† †
2 2† †
1( ; ) | | | |
| | | |p p q qp X q Z
E x z x z x zX Z
inverse term (New!) forward term [Kwatra et al. 2005]
X (input)
Z (output)zq
xq
best match
xp Zpbest match
Energy plotEnergy plot
compaction size
ener
gy
† †
2 2† †
1( ; ) | | | |
| | | |p p q qp X q Z
E x z x z x zX Z
original
Why both terms?Why both terms?
inverse term preserves all input features
forward term avoids artifacts in compaction
† †
2 2† †
1( ; ) | | | |
| | | |p p q qp X q Z
E x z x z x zX Z
inverse
f-onlymissing feature
forward
both both bothi-onlygarbage
i-onlydiscontinuity
Comparing with epitome [Jojic et al. 2003]Comparing with epitome [Jojic et al. 2003]
Similar to our method but only inverse term
blur, discontinuity
original original
epitome
our
epitome
our
Comparing with epitome [Jojic et al. 2003]Re-synthesis Comparing with epitome [Jojic et al. 2003]Re-synthesis
original original
epitome
our
epitome
our
SolverSolver
How to solve this?
Texture optimization [Kwatra et al. 2005]
Discrete solver [Han et al. 2006]
† †
2 2† †
1( ; ) | | | |
| | | |p p q qp X q Z
E x z x z x zX Z
Optimization [Kwatra et al. 2005]Optimization [Kwatra et al. 2005]
† †
2 2† †
1( ; ) | | | |
| | | |p p q qp X q Z
E x z x z x zX Z
X
Z
NO inverse term forward term [Kwatra et al. 2005]
zq
xq
xq Zq
E-stepfix xq
argminz E(x,z)least square
M-stepfix Zargminxq
|xq-zq|2
search
fix xq
argminxq |xq-zq|2
Our solverOur solver
† †
2 2† †
1( ; ) | | | |
| | | |p p q qp X q Z
E x z x z x zX Z
X
Z
inverse term forward term [Kwatra et al. 2005]E-step
fix xq
argminz E(x,z)least square
M-step (forward)fix Zargminxq
|xq-zq|2
search
zq
xq
xp
M-step (inverse)fix xp
argminzp |xp-zp|2
discrete solver
xp
zpxq Zq
discrete solver [Han et al. 2006]
discrete solverargminxq
|xq-zq|2
ResultsResults
GPU synthesis – small texture betterExtension from [Lefebvre & Hoppe 2005]
GPU synthesis – small texture betterExtension from [Lefebvre & Hoppe 2005]
original
cheesemold
1214 x 1212
dirt
271x481
compaction
1282
1282
3 fps, original6 fps, compact
3.5 fps, original7.0 fps, compact
Limitation:Correlation between texture & controlLimitation:Correlation between texture & control
original
compaction
reconstruction
texture control
Orientation field for anisotropic texturesOrientation field for anisotropic textures
Orientation field w as part of energy function
E(x, z) → E(x, z; w)
Good orientation field yields better solution
original orientation field
comp.no w
comp.with w
Future workFuture work
Higher dimensional textures
e.g. video
General images, not just textures
Bidirectional similarity [Simakov et al. CVPR 2008]
Image compression
AcknowledgementsAcknowledgements
Yale graphics group
Columbia graphics group
Sylvain Lefebvre
Hughes Hoppe
Matusik et al. 2005
Mayang.com
Jiaping Wang
Xin Tong
Jian Sun
Frank Yu
Bennett Wilburn
Eric Stollnitz
Dwight Daniels
Reviewers
Dinesh Manocha
Ming Lin
Chas Boyd
Brandon Lloyd
Avneesh Sud
Billy Chen
Thank You!Thank You!
