Image-Guided Weathering:Image-Guided Weathering:A New Approach Applied to Flow A New Approach Applied to Flow

PhenomenaPhenomenaC. BoschC. Bosch11, P. Y. Laffont, H. Rushmeier, , P. Y. Laffont, H. Rushmeier,

J. Dorsey, G. DrettakisJ. Dorsey, G. Drettakis

Yale University – REVES/INRIA Sophia AntipolisYale University – REVES/INRIA Sophia Antipolis1 1 Currently at ViRVIG, University of GironaCurrently at ViRVIG, University of Girona

Aging and WeatheringAging and Weathering

Essential for modeling urban environments Governed by physical, chemical and biological processes

Flow effectsFlow effects

Particularly complex Flow over the scene (global effect)

Material properties (local effect)

Aging and Weathering in CGAging and Weathering in CG

Physically-based simulation Difficult to get the desired effect

Texture synthesis Restricted by input information

Global effects particularly hard

MotivationMotivation

Physically-based simulation More flexible, allows global effects

Two main difficulties Choosing appropriate parameters to achieve a given effect

Obtaining realistic visual detail

Image-Guided WeatheringImage-Guided Weathering

Use images to guide simulation Flow stains as a representative case

Exemplar

New simulation

Overview (I)Overview (I)

Extract data from exemplars Color information

Simulation parameters

High frequency details

Si = 1.301 rt = 0.252kS = 0.0201 at = 0.404kD = 0.0807 T = 803ka,t = 0.021

Exemplar

Data

Overview (II)Overview (II)

Simulate new effects on scenes

Si = 1.301 rt = 0.252kS = 0.0201 at = 0.404kD = 0.0807 T = 803ka,t = 0.021

Data

Related WorkRelated Work

Simulation Phenomenon-specific [Merillou08]

Flow stains [Dorsey96; Chen05; Endo10]

Capture-and-transfer (synthesis) Single image [Wang06; Xue08]

Acquisition systems [Gu06; Mertens06; Sun07; Lu07]

Inverse procedural textures [Bourque04; Lefebvre00]

Flow modelFlow model

Particle-based simulation [Dorsey96] Absorption, solubility and deposition

Stain concentration maps

Parameters Particles: mass (m), Si

Stain material: kS, kD

Target materials: a, ka, roughness (r)

Simulation: time (t), particle rate (N)

Extracting StainsExtracting Stains

Based on Appearance Manifolds [Wang06]

ExemplarAppearance Manifold

Degree Map

Degree map = Stain concentration map

Error Simulation

Parameter FittingParameter FittingInput stain Degree map

Si = 1kS = 0.04kD = 0.04rt = 0.2at = 0.3ka,t = 0.05T = 300

Si = 1kS = 0.04kD = 0.04rt = 0.2at = 0.3ka,t = 0.05T = 300

Initialparameters

New parameters

Si = 1.3kS = 0.02kD = 0.08rt = 0.25at = 0.4ka,t = 0.02T = 803

Si = 1.3kS = 0.02kD = 0.08rt = 0.25at = 0.4ka,t = 0.02T = 803

Error < threshold or max. iterations

StopStop

Proxy geometry

target

(Levenberg-Marquardt) [Lourakis04]

image plane

source

Improving FittingImproving Fitting

1. Stain distribution along the source Accumulate degree from bottom to top

Improving Fitting (II)Improving Fitting (II)

2. Flow deflection along the target Compute local degree distribution (~vector field)

Error Simulation

Input stain Degree map

Si = 1kS = 0.04kD = 0.04rt = 0.2at = 0.3ka,t = 0.05T = 300

Si = 1kS = 0.04kD = 0.04rt = 0.2at = 0.3ka,t = 0.05T = 300

Initialparameters

New parameters

Si = 1.3kS = 0.02kD = 0.08rt = 0.25at = 0.4ka,t = 0.02T = 803

Si = 1.3kS = 0.02kD = 0.08rt = 0.25at = 0.4ka,t = 0.02T = 803

Error < threshold or max. iterations

StopStop

Proxy geometry

target

(Levenberg-Marquardt) [Lourakis04]

image plane

source

Parameter Fitting (II)Parameter Fitting (II)

Vector field

Stain distribution

Fitting Results (w/o vector field)Fitting Results (w/o vector field)

Exemplar Degree Map SimulationUsing source distribution

Fitting Results (w/o vector field)Fitting Results (w/o vector field)

Exemplar Degree Map Simulation

Fitting Results (w/ vector field)Fitting Results (w/ vector field)

Exemplar

Degree Map Simulation w/o vfield

Fitting Results (w/ vector field)Fitting Results (w/ vector field)Exemplar Degree Map Simulation

Fitting Results (w/ vector field)Fitting Results (w/ vector field)Exemplar Degree Map Simulation

Fitting Results (Complex Targets)Fitting Results (Complex Targets)

Exemplar Degree Map Simulation

Stain DetailStain Detail

Simulation lacks spatial variations (high-frequency detail)

Degree Map Simulation

Exemplar

Detail MapsDetail Maps

Extract detail by image difference Use guided texture synthesis [Lefebvre05] Detail maps will modify stain adhesion

Degree Map Simulation Difference

Detail Map

Simulating New StainsSimulating New Stains

Link data to stain sources and targets Parameters, detail maps, color

Use 1D texture synthesis for distributions

Run flow simulation Flow deflected by target geometry (+ disp. map)

Color TransferColor Transfer

Transfer stain color from input image Background mixed with stain everywhere

Non-linear relationship between color and degree

Use per-pixel warping

background color

target background

fully stained

ResultsResults

Results (II)Results (II)

Results (III)Results (III)

Results (IV)Results (IV)

PerformancePerformance

Preprocessing Degree map: 1-3 minutes

Fitting: 30-60 minutes (500 iter., ~256x512)

Detail synthesis: 1-2 minutes (1024x1024)

Final simulation Stain simulation: 2-5 minutes/stain

Color warping: 5-8 seconds/stain (1024x1024)

LimitationsLimitations

Good extraction from background Fitting: Not true physical estimations Detail maps: Depend on appropriate fit Computation time

ConclusionsConclusions

New approach to acquire simulation data from photographs Solves parameter estimation from images

Combines simulation with data-driven methods Appearance manifold, texture synthesis, …

Fills the gap between data-driven and simulation

Easy to use

Natural variations (including global effects)

Future workFuture work

Extend to other weathering phenomena Deal with large scale scenes

Fast simulation, global effects, …

AcknowledgementsAcknowledgements

Visiting grant U.Girona ANR project (ANR-06-MDCA-004-01) ERCIM “Alain Bensoussan” Fellowship Autodesk (Maya/MentalRay) Coding help: Li-Ying, Su Xue Scene treatment: S. Close and F. Andrade-Cabral

Thank youThank you