Visual Perception of Surface Materials Roland W. Fleming Max Planck Institute for Biological...

Visual Perception of Surface Visual Perception of Surface

Materials Materials

Roland W. FlemingRoland W. FlemingMax Planck InstituteMax Planck Institute

for Biological Cyberneticsfor Biological Cybernetics

3

Different materials, such as quartz, satin and chocolate have distinctive visual appearances

Without touching an object we usually have a clear impression of what it would feel like: hard or soft; wet or dry; rough or smooth

Visual Perception of Material Visual Perception of Material QualitiesQualities

4

The appearance of “stuff” was a major preoccupation in C17th Dutch art: Sensual quality of things Stasis, and

contemplation in Still Life

Contrast with Italian Renaissance: Perspective and the

spatial arrangement of things

Drama, events and dynamism.

Willem Kalfdetail from

Still Life with Silver JugRijksmuseum, Amsterdam

5

Material-ismMaterial-ism

6

Everyday lifeEveryday life

7

The state of thingsThe state of things

8

EdibilityEdibility

9

EdibilityEdibility

10

UsabilityUsability

11

DungDung

12

Materials in Materials in Computer GraphicsComputer Graphics

Hollywood and the games industry know that photorealism means getting materials right

Henrik Wann Jensen, Steve Marschner and Pat Hanrahan won a technical Oscar in 2004 for their work on Sub-surface scattering

Henrik Wann Jensen and Craig Donner

13

What gives a material its characteristic appearance?

How does the human brain identify materials across a wide variety of viewing conditions?

Research QuestionsResearch Questions

14

Visual estimation of surface reflectance properties: gloss

Perception of materials that transmit light Refraction Sub-surface scattering

Exploiting the assumptions of the visual system to edit the material appearance of objects in photographs

OutlineOutline

15

Surface ReflectanceSurface Reflectance

These spheres look different because they have different surface reflectance properties.

Everyday language: colour, gloss, lustre, etc.

Images: Ron Dror

16

Surface ReflectanceSurface Reflectance

Visual system’s goal: Estimate the BRDF

f(i, i; r, r)BRDF

Images: Ron Dror

17

Confounding EffectsConfounding Effectsof Illuminationof Illumination

Identical materials can lead to very different images

Different materials can lead to very similar images

18

Ambiguity betweenAmbiguity betweenReflectance and IlluminationReflectance and Illumination

a(f) = 1 / f

= 0 = 0.4 = 0.8 = 1.2

= 1.6 = 2.0 = 4.0 = 8.0

19

HypothesisHypothesis

Humans exploit statistical regularities of real-world

illumination in order to eliminate unlikely image interpretations

20

Real-world IlluminationReal-world Illumination

Directly from luminous sources

Indirectly, reflected from other surfaces

Illumination at a point in space: amount of light arriving from every direction.

21

Photographically capturedPhotographically capturedlight probeslight probes (Debevec (Debevec et alet al., 2000)., 2000)

Panoramic projectionPanoramic projection

22

Statistics of typicalStatistics of typicalIlluminationIllumination (Dror et al. 2004) (Dror et al. 2004)

Intensity histogram is heavily skewed Few direct light sources

23

Statistics of typicalStatistics of typicalIlluminationIllumination (Dror et al. 2004) (Dror et al. 2004)

Typical 1/f amplitude spectrum

24

HypothesisHypothesis

Humans exploit statistical regularities of real-world

illumination in order to eliminate unlikely image interpretations

25

Dismissing unlikelyDismissing unlikelyinterpretationsinterpretations

Blurry featureBlurry feature

2 interpretations:

Sharp reflection, blurry world Blurry reflection, sharp worldBut the world usually isn’t blurry!

Therefore it is probably a blurry reflection

26

Dismissing unlikelyDismissing unlikelyinterpretationsinterpretations

In practice, unlikely image interpretations do not need to be explicitly entertained

Under normal illumination conditions, different materials yield to diagnostic image features that are responsible for their ‘look’

The brain doesn’t need to The brain doesn’t need to model the physics, it just model the physics, it just needs to look out for tell-needs to look out for tell-tale image featurestale image features

27

Context has surprisingly little effect on apparent gloss

ObservationsObservations

28

FindingsFindings

Subjects are good at judging surface reflectance across real-world illuminations: ‘gloss constancy’

BeachBeachLight probeLight probe

CampusCampusLight probeLight probe

Single pointSingle point sourcesource

GaussianGaussian1/f Noise1/f Noise

Subjects are poorer at estimating gloss under illuminations with atypical statistics.

29

Illuminations haveIlluminations haveskewed intensity histogramsskewed intensity histograms

Intensity, I

N(I

)

0 200

>103

Intensity, I

N(I

)

400

>102

30

OriginalOriginal

Illumination distributionIllumination distributionis importantis important

Campus

Pink noise

ModifiedModified

Intensity, I

N(I

)

0 200

>103

Intensity, I

N(I

)

0 200

>103

Intensity, I

N(I

)

400

>102

Intensity, I

N(I

)

400

>102

OriginalOriginal

ModifiedModified

31

Histograms aren’t everythingHistograms aren’t everything

White noise White noise withwithhistogram ofhistogram of Campus CampusCampus originalCampus original

32

Heeger-Bergen texture Heeger-Bergen texture synthesissynthesis

Input textureInput texture Synthesized textureSynthesized texture

Treat illumination maps as if they are stochastic texture

Taken from Pyramid-Based Texture Analysis/Synthesis

33

Wavelet StatisticsWavelet Statistics

BeachBeach BuildingBuilding CampusCampus EucalyptusEucalyptus

UffiziUffiziSt. Peter’sSt. Peter’sKitchenKitchenGraceGrace

Synthetic illuminations with same wavelet statistics as real-world illuminations

34




OutlineOutline

35

Previous work onPrevious work onmaterials that transmit lightmaterials that transmit light

Adelson E.H. (1993). Perceptual organization and the judgment of brightness. Science. 262(5142): 2042-2024.Adelson, E. H. (1999). Lightness perception and lightness illusions. In In The New Cognitive Neurosciences. 2nd edn. (ed. Gazzaniga, M.S.) 339—351 (MIT Press, Cambridge, Massachusetts, 1999).Adelson, E. H., and Anandan, P. (1990). Ordinal characteristics of transparency. Proceedings of the AAAI-90 Workshop on Qualitative Vision, 77- 81.Anderson B.L. (1997). A theory of illusory lightness and transparency in monocular and binocular images: the role of contour junctions. Perception. 26(4): 419-453.Anderson, B.L. (1999). Stereoscopic surface perception. Neuron. 24: 991-928.Anderson, B.L. (2001). Contrasting theories of White's illusion. Perception, 30: 1499-1501.Anderson, B.L. (2003). The Role of Occlusion in the Perception of Depth, Lightness, and Opacity. Psychological Review, 110(4): 785-801.Anderson B.L. (2003). Perceptual organization and White's illusion. Perception. 32(3): 269-84.Beck J. (1978). Additive and subtractive color mixture in color transparency. Percept Psychophys. 23(3): 265-267. Beck J, Prazdny K, Ivry R. (1984). The perception of transparency with achromatic colors. Percept Psychophys. 35(5): 407-422.Beck J, Ivry R. (1988). On the role of figural organization in perceptual transparency. Percept Psychophys. 44(6): 585-594.Chen, J.V. & D'Zmura, M. (1998). Test of a convergence model for color transparency perception. Perception, 27: 595-608.D'Zmura, M., Colantoni, P., Knoblauch, K. & Laget, B. (1997). Color transparency, Perception, 26, 471-492.D Zmura, M., Rinner, O., & Gegenfurtner, K. R. (2000). The colors seen behind transparent filters. Perception, 29, 911-926.Gerbino W., Stultiens C.I., Troost J.M. and de Weert C.M. (1990). Transparent layer constancy. J Exp Psychol Hum Percept Perform. 16(1): 3-20. Gerbino, W. (1994). Achromatic transparency. In A.L. Gilchrist, Ed. Lightness, Brightness and Transparency. (pp 215-255). Hove, England: Lawrence Erlbaum.Heider, G. M. (1933). New studies in transparency, form and color. Psychologische Forschung. 17: 13—56.Kersten, D. (1991) Transparency and the cooperative computation of scene attributes. In Computational Models of Visual Processing, M.I.T. Press, Landy M and Movshon A., Eds..Kersten, D. and Bülthoff, H. (1991) Transparency affects perception of structure from motion. Massachusetts Institute of Technology, Center for Biological Information Processing Tech. Memo, 36.Kersten, D., Bülthoff, H. H., Schwartz, B., & Kurtz, K. (1992). Interaction between transparency and structure from motion. Neural Computation, 4(4), 573-589.Khang BG, Zaidi Q. (2002). Cues and strategies for color constancy: perceptual scission, image junctions and transformational color matching. Vision Res. 42(2):211-26. Erratum in: Vision Res.

2002. 42(24): 2729.Khang, B.-G., & Zaidi, Q. (2002). Accuracy of color scission for spectral transparencies. Journal of Vision, 2(6), 451-466, doi:10.1167/2.6.3.Kanizsa, G. (1955). Condiziono ed effetti della transparenza fenomica. Revista di Psicologia. 49: 3—19.Koffka, K. (1935). Principles of Gestalt Psychology. Cleveland: Harcourt, Brace and World.Lindsey DT, Todd JT. (1996). On the relative contributions of motion energy and transparency to the perception of moving plaids. Vision Res. 36(2): 207-22.Metelli, F. (1970). An algebraic development of the theory of perceptual transparency. Ergonomics, 13(1): 59-66.Metelli, F. (1974a). Achromatic color conditions in the perception of transparency. In Eds. Macleod, R. B., & Pick, H. L. (Eds.) Perception (pp. 95—116). Ithaca, NY: Cornell University Press.Metelli F. (1974b). The perception of transparency. Scientific American, 230(4): 90-98.Metelli, F. (1985). The simulation and perception of transparency. Psychological Research, 47(4): 185-202.Metelli, F., Da Pos, O. and Cavedon, A. (1985). Balanced and unbalanced, complete and partial transparency. Perception & Psychophysics, 38(4): 354-66.Masin SC. (1999). Color scission and phenomenal transparency. Percept Mot Skills. 89(3 Pt 1): 815-23.Masin SC. (1999) Phenomenal transparency in achromatic checkerboards. Percept Mot Skills. 88(2): 685-92. Masin SC. (1997). The luminance conditions of transparency. Perception. 26(1): 39-50.Nakayama, K., & Shimojo, S. (1992). Experiencing and perceiving visual surfaces. Science. 257: 1357-1363.Nakayama, K., Shimojo, S., & Ramachandran, V. (1990). Transparency: Relation to depth, subjective contours, luminance, and neon color spreading. Perception, 19: 497-513.Plummer DJ, Ramachandran VS. (1993). Perception of transparency in stationary and moving images. Spat Vis. 7(2): 113-23. Robilotto R, Khang BG & Zaidi Q. (2002). Sensory and physical determinants of perceived achromatic transparency. JOV. 2(5): 388-403.Robilotto, R., & Zaidi, Q. (2004). Perceived transparency of neutral density filters across dissimilar backgrounds. Journal of Vision, 4(3), 183-195, doi:10.1167/4.3.5.Singh M, Anderson BL. (2002). Perceptual assignment of opacity to translucent surfaces: the role of image blur. Perception. 31(5): 531-52. Singh M, Anderson BL. (2002). Toward a perceptual theory of transparency. Psychol Rev. 109(3): 492-519.Somers, D. C., and Adelson, E. H. (1997). Junctions, Transparency, and Brightness. Investigative Ophthalmology and Vision Science (supplement), 38, S453.Stoner GR, Albright TD, Ramachandran VS. (1990). Transparency and coherence in human motion perception. Nature. 344(6262): 153-155. Tommasi M. (1999) A ratio model of perceptual transparency. Percept Mot Skills. 89(3 Pt 1): 891-7. Watanabe, T. & Cavanagh, P. (1992). The role of transparency in perceptual grouping and pattern recognition. Perception, 21,131-139.Watanabe, T. & Cavanagh, P. (1993). Transparent surfaces defined by implicit X junctions. Vision Research, 33: 2339-2346. Watanabe, T. & Cavanagh, P. (1993). Surface decomposition accompanying the perception of transparency. Spatial Vision, 7, 95-111.

36

Transparent MaterialsTransparent Materials

Metelli’s episcotister

37

Real transparent objectsReal transparent objects

38

Real transparent objectsReal transparent objects … are not ideal infinitesimal

films

… obey Fresnel’s equations: Specular reflections Refraction

… can appear vividly transparent without containing the image cues traditionally believed to be important for the perception of “transparency”.

39


Questions:

What image cues do we use to tell that something is transparent?

How do we estimate and represent the refractive index of transparent bodies?

40

Refractive IndexRefractive Index Possibly the most important property that

distinguishes real chunks of transparent stuff from Metelli-type materials

Snell’s Law:

sin(i)

sin(t)

n2

n1

=

41

Refractive IndexRefractive Index

Varying the refractive index can lead to the distinct impression that the object is made out a different material

1.51.2 2.3

refractive index

42


43


44


45


46


47


48


49


50


51


52

Refraction and image distortionRefraction and image distortion

convex concave

Low RI

53

Refraction and image distortionRefraction and image distortion

convex concave

High RI

54

Displacement FieldDisplacement Field

Measures the way the transparent object displaces the position of refracted features in the image.

d = prefracted - pactual

The perturbation of the texture caused by refraction can be captured by the displacement field.

55

vector field

Displacement FieldDisplacement Field

56

Distortion FieldsDistortion Fields

But, to compute the displacement field the visual system would need to know the positions of non-displaced features on the backplane.

Let us assume, instead, that the visual system can estimate the relative distortions of the texture (compression or expansion of texture)

D = div( d )

57


distortion fieldimage

Red = magnification Green = minification

58

Distance to backplaneDistance to backplane

59


60


61


62


63


64


65


66


67


68


convex concave

near

69


convex concave

far

70

Object thicknessObject thickness

71


72


73


74


75


76


77


78


79


80

Asymmetric Matching taskAsymmetric Matching task

The distortions in the image depend not only on the RI but also on: Geometry of object (curvatures, thickness) Distance between object and background Distance between viewer and object

So, if the observers base their judgments of RI primarily on the pattern of distortions (rather than correctly estimating the physics), then they should show systematic errors in their estimates when these irrelevant scene factors vary.

81


Subject adjusts RI of standard probe stimulus to match the appearance of the other stimulus.

One scene variable (thickness, distance to back-plane) is clamped at a different value for test stimulus.

Measures ability of observer to ‘ignore’ or ‘discount’ the differences that are caused by irrelevant scene variables

probetest

82


distance to backplane thickness of pebble

0.25

0.625

1.375

1.75

0.5

0.75

1.25

1.5

Refractive index

Perc

eiv

ed R

efr

act

ive in

dex

84

Observers don’t appear to estimate the physics correctly.

Instead, they probably use some heuristic image measurements (e.g. distortion fields) that are affected by refractive index, but also by other scene factors.

This can lead to illusions (mis-perceptions)

ObservationsObservations

85




OutlineOutline

86

Image-based Material EditingImage-based Material EditingKahn, Reinhart, Fleming & Bülthoff, SIGGRAPH Kahn, Reinhart, Fleming & Bülthoff, SIGGRAPH 06.06. inputinput outputoutput

87

Image based material editingImage based material editing

Goal: given single (HDR) photo as input, change appearance of object to completely different material

Physically accurate solution would require fully reconstructing illumination and 3D geometry. Beyond state-of-the-art computer vision capabilities

input output

88

Image based material editingImage based material editing

Alternative: “Perceptually accurate” solution Series of simple, highly approximate manipulations, each of which is provably

incorrect, but whose ensemble effect is visually compelling.

input output

89

texture mapping

output image

bilateralfilter

holefilling

environment

depth map

Processing PipelineProcessing Pipeline

alpha matte

segmentation

90

Alpha MattesAlpha Mattes By restricting our image modifications to the area within the

boundary of the object, we can create the illusion of a transformed material.

Assumption: addition of new non-local effects (e.g. additional reflexes, caustics, etc.) is not crucial. Approximate shadows and reflections are already in place in original

scene

91

Automatic alpha-mattingAutomatic alpha-matting

Bayesian pixel classification methods Chuang, Curless, Salesin & Szeliski (2001) Apostoloff & Fitzgibbon (2004)

Methods using multiple images with different planes of focus McGuire, Matusik, Pfister, Hughes & Durand

(2005) Kahn & Reinhard (2005)

Although we extract our alpha-mattes manually (rotoscoping), there has been a recent wave of automatic / semi-automatic procedures.

92


texture mapping

output image

bilateralfilter

holefilling

environment

depth map

alpha matte

segmentation

93


texture mapping

output image

bilateralfilter

holefilling

environment

depth map

alpha matte

segmentation

94

How How notnot to do to doshape-from-shadingshape-from-shading

Try using the state-of-the-art algorithms and you will generally be disappointed!

inputinput

reconstructionreconstruction

95


inputinput



96


inputinput



97

What is the alternative ?What is the alternative ?

We use a simple but suprisingly effective heuristic:

Dark is DeepDark is DeepIn other words …

z(x, y) = L(x, y)z(x, y) = L(x, y)

initial depth estimateinitial depth estimateinputinput

ComplicateComplicated shape d shape

from from shading shading

algorithmalgorithm

WHAT ?WHAT ?

98

Bilateral FilterBilateral Filter

The ‘recovered depths’ are conditioned using a bilateral filter (Tomasi & Manduchi, 1998; Durand & Dorsey, 2002).

Simple non-linear edge-preserving filter with kernels in space and intensity domains.

Note: 2 user parameters.

99

Bilateral FilterBilateral Filter

spacespace

intensityintensity

Normally use log L, but this is unbounded. We prefer to use a sigmoid (based on human vision)

100

Bilateral Filter: 3 main functionsBilateral Filter: 3 main functions

1. De-noising depth-map Intuition: depths are generally smoother than intensities in the

real world.

2. Selectively enhance or remove textures for embossed effect

101

Bilateral Filter: 3 main functionsBilateral Filter: 3 main functions

3. Shape-from-silhouette, like level-sets shape ‘inflation’ (e.g. Williams, 1998) Intuition: values outside object are set to zero, so blurring

across boundary makes recovered depths smooth and convex.

102

Forgiving caseForgiving case

Diffuse surface reflectance leads to clear shading pattern Silhouette provides good constraints

originaloriginal reconstructed depthsreconstructed depths

103

Difficult caseDifficult case Strong highlights create large spurious depth peaks Silhouette is relatively uninformative


104

Light from the sideLight from the side Shadows and intensity gradient leads to substantial distortions of the

face


105

Importance of viewpointImportance of viewpoint

correct viewpointcorrect viewpoint

Substantial errors in depth reconstruction are not visible in transformed image

transformed imagetransformed image

106


107


108


109


110


111


112

Why does it work ?Why does it work ? Generic viewpoint assumption (Koenderink & van

Doorn, 1979; Binford, 1981; Freeman, 1994)

113

Why does it work ?Why does it work ?

Bas-relief ambiguity Belhumeur, Kriegman & Yuille (1999)

Affine transformed versions of scene are indistinguishable

Particularly important for objects illuminated from the side

114

Why does it work ?Why does it work ?

frontfront sideside

115

Why does it work ?Why does it work ? Masking effect of patterns that are subsequently placed on surface

(e.g. highlights).

116

From depths to surface normalsFrom depths to surface normals

Note: 1 user parameter (number of iterations)

Take the derivates. Reshape them recursively.

Cross product of derivates gives surface normal

117


texture mapping

output image

bilateralfilter

holefilling

environment

depth map

alpha matte

segmentation

118


texture mapping

output image

bilateralfilter

holefilling

environment

depth map

alpha matte

segmentation

119

Re-texturing the objectRe-texturing the object

Use recovered surface normals as indices into a texture map The most important trick: blend original intensities back into

image, for correct shading and highlights

120


Use recovered surface normals as indices into a texture map The most important trick: blend original intensities back into

image, for correct shading and highlights

121


The most important trick: blend original intensities back into image, for correct shading and highlights

TextureShop (Fang & Hart, 2004) tacitly uses the same trick:

122

Other material transformationsOther material transformations

To create the illusion of transparent or translucent materials, we map a modified version of the background image onto the surface.

To apply arbitrary BRDFs, we use the recovered surface normals, and an approximate reconstruction of the environment

to evaluate the BRDF.

123


texture mapping

output image

bilateralfilter

holefilling

environment

depth map

alpha matte

segmentation

124


texture mapping

output image

bilateralfilter

holefilling

environment

depth map

alpha matte

segmentation

125

Hole filling the easy wayHole filling the easy way A number of sophisticated algorithms exist for object removal (e.g.

Bertalmio et al. 2000; Drori et al. 2003; Sun et al. 2005) Crude but fast alternative: cut and paste!

126



127



128

Fake transparencyFake transparency The human visual system appears does not recognize

transparency by correctly using inverse optics. Instead, it seems to rely on the consistency of the image

statistics and patterns of distortion.

129

Fake translucencyFake translucency

To create frosted glass, simply blur the background before mapping it onto the surface

130


For translucency, blur and colour correct Reinhard et al. (2001) colour transfer

131


132


133

Environment mapEnvironment map

Background image is used to generate full HDR light probe for image-based lighting

134

Arbitrary BRDFsArbitrary BRDFs Given surface normals and complete HDR light probe, we can

evaluate empirical or parametric BRDFs, as in standard image based lighting (local effects only).

We used Matusik’s BRDFs.

originaloriginal blue metallicblue metallic nickelnickel

135

Arbitrary BRDFsArbitrary BRDFs Given surface normals and complete HDR light probe, we can

evaluate empirical or parametric BRDFs, as in standard image based lighting (local effects only).

We used Matusik’s BRDFs.

originaloriginal nickelnickel

136

How important is HDR ?How important is HDR ?

HDR makes the procedure much more stable Depth estimates are superior Easier to identify (and selectively manipulate) highlights

137

How important is HDR ?How important is HDR ?

HDR makes the procedure much more stable Depth estimates are superior Easier to identify (and selectively manipulate) highlights

originaloriginal boostedboosted suppressedsuppressed

138

General PrinciplesGeneral Principles Complementary heuristics. Normally, errors accumulate

as one approximation is added to another. The key to our approach is choosing heuristics such that the errors of one approximation visually compensate for the errors of another.

Exploit visual tolerance for ambiguities to achieve solutions that are perceptually acceptable even though they are physically wrong.

139

Using shape toUsing shape toinfer material propertiesinfer material properties

140

Using shape toUsing shape toinfer material propertiesinfer material properties

Photo: Ted Adelson

141

Thank YouThank You

Funding: DFG FL 624/1-1

Some renderings generated using Henrik Wann Jensen’s DALI

Co-authors:

Ted Adelson, Ron Dror, Frank Jäkel, Larry Maloney, Erum Kahn, Erik Reinhard, Heinrich Bülthoff

142

Stability across extrinsic scene Stability across extrinsic scene variablesvariables

Physical RI

Perc

ep

tual sc

ale

distance tobackground

pebblethickness

143

Contrast as a cue to RIContrast as a cue to RI

Refractive index

Variance

Mean

144

Contrast as a cue to RIContrast as a cue to RI

Scene variable

Variance

Mean

thickness

backplane distance

view distance

145

Variability across Variability across subjectssubjectsdistance to backplane thickness of pebble

Subject 1

Subject 2

146

Binocular cuesBinocular cues

Matched features generally do not specify the correct stereo depth of the surface: specularities and features visible through object

Disparity field can contain large spurious vertical disparities and unmatchable features, leading to rivalry

Extent of rivalry could in fact be a cue to RI (cf. binocular ‘lustre’).

147

Binocular cuesBinocular cues

Matched features generally do not specify the correct stereo depth of the surface: specularities and features visible through object

Disparity field can contain large spurious vertical disparities and unmatchable features, leading to rivalry

Extent of rivalry could in fact be a cue to RI (cf. binocular ‘lustre’).

148

Refraction vs. ReflectionRefraction vs. Reflectionglassmirror

Both reflection and refraction work by mapping features from the surroundings into the image

But: different mapping rules

149

displacement field


Measures the way the transparent object displaces the position of refracted features in the image.

150

Real transparent objectsReal transparent objects … are not ideal infinitesimal

films

… obey Fresnel’s equations: Specular reflections Refraction

… can appear vividly transparent without containing the image cues traditionally believed to be important for the perception of “transparency”.

151


Questions:

What image cues do we use to tell that something is transparent?

How do we estimate and represent the refractive index of transparent bodies?

152

The visual brainThe visual brain

More than a third of the brain contains cells that process visual signals.

Literally billions of neurons are dedicated to the task of seeing. Why? Is vision really so complicated?

154

Statistics of typicalStatistics of typicalIlluminationIllumination (Dror et al. 2004) (Dror et al. 2004) Properties based on raw luminance values

High dynamic range Pixel histogram heavily skewed towards low intensities

Quasi-local and Non-local properties Nearby pixels are correlated in intensity (roughly 1/f amplitude

spectrum) Distributions of wavelet coefficients are highly kurtotic (i.e.

significant wavelet coefficients are sparse) Approximate scale invariance (i.e. distributions of wavelet

coefficients are similar at different scales)

Global and Non-stationary properties Dominant direction of illumination Presence of recognizable objects such objects and trees Cardinal axes (due to ground plane and perpendicular structures

erected thereupon).

155

What is the perceptual scale of refractive index ?

??

Physical RI

Perc

eiv

ed R

I

156

Maximum Likelihood Difference Maximum Likelihood Difference ScalingScaling

A B

157


nCk image permutations: n=10 k=4

nCk=210 quadruplesestimate scale that best accountsfor similarity judgments

Refractive Index

21 4 10

A B

158


Physical RI

Perc

ep

tual sc

ale

159


Physical RI

Perc

ep

tual sc

ale

RMS imagedifferences

perceptual scale

160

Physical RI

Dis

tort

ion M

ag

nit

ude


161

Physical RI

Perc

ep

tual sc

ale

RMS imagedifferences

perceptual scale


mean distortion

162

Surface Reflectance MatchingSurface Reflectance Matching

TestTest MatchMatch

METHODMETHOD

163

Ward modelWard model

Specular Reflectance matte to glossy

Roughness crisp to blurred

Axes rescaled to form a psychophysically uniform space (Pellacini et al. 2000)

Rou

gh

nes

s

Specular Reflectance

164

Real-world IlluminationsReal-world Illuminations

Illuminations downloaded from: http//graphics3.isi.edu/~debevec/Probes

BeachBeach BuildingBuilding CampusCampus EucalyptusEucalyptus

UffiziUffiziSt. Peter’sSt. Peter’sKitchenKitchenGraceGrace

165

Match IlluminationMatch Illumination

GalileoGalileo

166

Artificial IlluminationsArtificial Illuminations

Single point sourceSingle point source Multiple point sourcesMultiple point sources Extended sourceExtended source

Gaussian White NoiseGaussian White Noise Gaussian Pink NoiseGaussian Pink Noise

167

Subject: RF. (110 observations)Illumination: “St. Peter’s”.

Specular reflectance

Surface roughness

Subject: MS. (110 observations)Illumination: “Grace”.

Subject: RA. (110 observations)Illumination: “Eucalyptus”.

Specular reflectance Specular reflectance

Surface roughnessSurface roughness

Mat

chm

atte

shin

y

matte shinyTest

Mat

chm

atte

shin

y

Mat

chm

atte

shin

y

matte shinyTest

matte shinyTest

smooth roughTest smooth rough

Test

smooth roughTest

Mat

chsm

ooth

roug

h

Mat

chsm

ooth

roug

h

Mat

chsm

ooth

roug

h

Subjects can matchSubjects can match surface reflectance surface reflectance

168

Specular reflectance Surface roughness

roughsmoothmatte shiny

Data pooled across all subjects and all real-world illuminations

Test

Mat

ch

Test

Mat

chSubjects can matchSubjects can match surface reflectance surface reflectance

169

Real-world Real-world vsvs Artificial IlluminationArtificial Illumination

0

5

10

15

20

25

30

35

40

45

Beach BuildingCampusEucalyptus

GraceKitchenSt. Peter's

UffiziPoint Multi

ExtendedWhite NoisePink Noise

0

5

10

15

20

25

30

Beach BuildingCampusEucalyptus

GraceKitchenSt. Peter's

UffiziPoint Multi

ExtendedWhite NoisePink Noise

erro

r

erro

r

Illumination map Illumination map

Specular reflectanceSpecular reflectance RoughnessRoughness

170

Noise is unlikeNoise is unlikereal-world illuminationreal-world illumination

UffiziUffizi White NoiseWhite Noise Pink NoisePink Noise

mat

ch

mat

ch

mat

chSpecular contrast

Least accurate of all real-world illuminations

Specular contrast Specular contrast

171

What are the relevant statistics?What are the relevant statistics?

Real Extended 1/f Noise

172

Real-world IlluminationsReal-world Illuminations

BeachBeach CampusCampus

Single point sourceSingle point source

Gaussian Pink NoiseGaussian Pink Noise

173

Material-ismMaterial-ism

174

Materials in Materials in Computer GraphicsComputer Graphics

Nvidia advanced skin rendering demo

Hollywood and the games industry know that photorealism means getting materials right

No subsurface scatteringNo subsurface scattering With subsurface scatteringWith subsurface scattering

175

EdibilityEdibility

176

EdibilityEdibility

177

EdibilityEdibility

178

EdibilityEdibility

179

algaealgae

Visual Perception of Surface Materials Roland W. Fleming Max Planck Institute for Biological...

Documents

Transcript of Visual Perception of Surface Materials Roland W. Fleming Max Planck Institute for Biological...