Fast Texture Synthesis using Tree-structured Vector Quantization
description
Transcript of Fast Texture Synthesis using Tree-structured Vector Quantization
![Page 1: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/1.jpg)
Fast Texture Synthesis using Fast Texture Synthesis using Tree-structured Vector Tree-structured Vector
QuantizationQuantization
Li-Yi Wei Marc Levoy
Computer Graphics GroupStanford University
![Page 2: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/2.jpg)
IntroductionIntroduction
Texture Synthesis
Input
Result
![Page 3: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/3.jpg)
Desirable PropertiesDesirable Properties
• Result looks like the input
• Efficient
• General
• Easy to use
• Extensible
![Page 4: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/4.jpg)
Previous WorkPrevious Work
• Procedural Synthesis– Perlin 85, Witkin 91, Worley 96
• Statistical Feature Matching– Heeger 95, De Bonet 97, Simoncelli 98
• Markov Random Fields– Popat 93, Efros 99
![Page 5: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/5.jpg)
OutlineOutline
• Basic algorithm
• Multi-resolution algorithm
• Acceleration
• Applications
![Page 6: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/6.jpg)
Texture ModelTexture Model
• Textures are– local– stationary
• Model textures by – local spatial neighborhoods
![Page 7: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/7.jpg)
Basic AlgorithmBasic Algorithm
• Exhaustively search neighborhoods
![Page 8: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/8.jpg)
NeighborhoodNeighborhood
• Use causal neighborhoods
Causal Non-causal
Input
Noise
![Page 9: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/9.jpg)
NeighborhoodNeighborhood
• Neighborhood size determines the quality & cost
33 55 77
99 1111 4141
423 s 528 s 739 s
1020 s 1445 s 24350 s
![Page 10: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/10.jpg)
Multi-resolution PyramidMulti-resolution Pyramid
High resolution Low resolution
![Page 11: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/11.jpg)
Multi-resolution Multi-resolution AlgorithmAlgorithm
![Page 12: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/12.jpg)
BenefitBenefit
• Better image quality & faster computation
1 level55
3 levels55
1 level1111
![Page 13: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/13.jpg)
ResultsResults
Random Oriented
Regular Semi-regular
![Page 14: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/14.jpg)
FailuresFailures
• Non-planar structures
• Global information
![Page 15: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/15.jpg)
ComparisonComparison
Heeger 95 De Bonet 97 Efros 99 Our method
Input
1941 secs 503 secs
12 secs
![Page 16: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/16.jpg)
Acceleration Acceleration
• Computation bottleneck: neighborhood search
![Page 17: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/17.jpg)
Nearest Point SearchNearest Point Search
• Treat neighborhoods as high dimensional points
1 2 3 4 5
6 7 8 9 10
11 12
Neighborhood
1 2 3 4 5 6 7 8 9 10 11 12
High dimensional point/vector
![Page 18: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/18.jpg)
AccelerationAcceleration
• Nearest point search in high dimensions– [Nene 97]
• Cluster-based model for textures– [Popat 93]
• Tree-structured Vector Quantization– [Gersho 92]
![Page 19: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/19.jpg)
Tree-structured Tree-structured Vector QuantizationVector Quantization
![Page 20: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/20.jpg)
TimingTiming
• Time complexity : O(log N) instead of O(N)– 2 orders of magnitude speedup for non-trivial images
1941 secs 503 secs 12 secs
Efros 99 Full searching TSVQ
![Page 21: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/21.jpg)
Results: Results: Brodatz TexturesBrodatz Textures
Input Exhaustive: 360 secs TSVQ: 7.5 secs
D103
D20
![Page 22: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/22.jpg)
Application 1: Application 1: Constrained SynthesisConstrained Synthesis
?
![Page 23: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/23.jpg)
Possible SolutionPossible Solution
• Multi-resolution blending [Burt & Adelson 83]– produce visible boundaries
![Page 24: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/24.jpg)
Possible SolutionPossible Solution
• Original raster-scan algorithm– discontinuities at right and bottom boundaries
![Page 25: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/25.jpg)
Possible SolutionPossible Solution
• Adaptive neighborhoods [Efros 99]– Hard to accelerate
![Page 26: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/26.jpg)
Modifications Modifications
• Need to use a single symmetric neighborhood
• 2 pass algorithm with extrapolation
• Spiral order synthesis
![Page 27: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/27.jpg)
ResultResult
![Page 28: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/28.jpg)
ResultResult
• Extrapolation
??
??
![Page 29: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/29.jpg)
ResultResult
• Image editing by texture replacement
![Page 30: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/30.jpg)
Application 2:Application 2:Temporal TextureTemporal Texture
• Indeterminate motions both in space and time– fire, smoke, ocean waves
• How to synthesize?– extend our 2D algorithm to 3D
![Page 31: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/31.jpg)
Temporal TextureTemporal Texture
Fire Smoke Waves
Input
Result
![Page 32: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/32.jpg)
Future WorkFuture Work
• More general “textures”– light fields, solid textures– motion signals– displacement maps
• Real time texture synthesis
![Page 33: Fast Texture Synthesis using Tree-structured Vector Quantization](https://reader035.fdocuments.in/reader035/viewer/2022062803/56814652550346895db365cb/html5/thumbnails/33.jpg)
AcknowledgmentAcknowledgment
• Kris Popat• Alyosha Efros• Stanford Graphics Group• Intel, Interval, Sony
More information
http://graphics.stanford.edu/projects/texture/