1

Probability density function (pdf) estimation using isocontours/isosurfaces

Application to Image Registration Application to Image Filtering Circular/spherical density estimation in

Euclidean space

2

3

Histograms Kernel density

estimate

Mixture model

Parameter selection:bin-width/bandwidth/number

of componentsBias/variance tradeoff:

large bandwidth: high bias, low bandwidth: high variance)

Sample-based methods

Do not treat a signal as a

signal

4

Continuous image representation: using some interpolant.

Trace out isocontours of the intensity function I(x,y) at several intensity values.

αIntensity at Curves Level ααα ∆+ andIntensity at Curves Level regionbrown of area )( ∝∆+<< ααα IP

6

Assume a uniform density on (x,y)

Random variable transformation from

(x,y) to (I,u)

Integrate out u to get the density of intensity I

Every point in the image domain contributes to

the density.

22),(

),(

||1

||1)(

yxyxI

yxII

II

du

dxdyddp

+Ω=

Ω=

∫

∫

=

≤

α

αα

α

Published in CVPR 2006, PAMI 2009.

u = direction along the level set

(dummy variable)

7

8

)y,x()y,x()y,x(I,)y,x(I|)y,x(C

|))y,x(sin()y,x(I)y,x(I|1

||1),(p

2211

21C21I,I 21

at gradients between angle=θα=α==

θ∇∇Ω=αα ∑

2211 Iin and Iin Intensity at Curves Level αα2222

1111

Iin ),( and Iin ) ,(Intensity at Curves Level

αααααα

∆+∆+

regionblack of area ),( 22221111 ∝∆+<<∆+<< αααααα IIP

Relationships between geometric and probabilistic

entities.

Similar density estimator developed by Kadir and Brady (BMVC 2005) independently of us.

Similar idea: several differences in implementation, motivation, derivation of results and applications.

9

10

)y,x(I,)y,x(I|)y,x(C

|))y,x(sin()y,x(I)y,x(I|1

||1),(p

2211

21C21I,I 21

α=α==

θ∇∇Ω=αα ∑

|),(|||1)(

),(yxI

dupyxI

I ∇Ω= ∫

= α

α

Densities (derivatives of the cumulative) do not exist where image gradients are zero, or where image

gradients run parallel.

Compute cumulative interval measures. )( ∆+<< αα IP

11

12

Standard histograms Isocontour Method

32 bins64 bins128 bins256 bins512 bins1024 bins

13

Randomized/digital approximation to area calculation.

Strict lower bound on the accuracy of the isocontour method, for a fixed interpolant.

Computationally more expensive than the isocontour method.

14

15

128 x 128 bins

Simplest one: linear interpolant to each half-pixel (level curves are segments).

Low-order polynomial interpolants: high bias, low variance.

High-order polynomial interpolants: low bias, high variance.

16

17

Polynomial Interpolant

Accuracy of estimated density improves as signal is sampled with

finer resolution.

Assumptions on signal: better interpolant

Bandlimited analog signal, Nyquist-sampled digital

signal:Accurate reconstruction by

sincinterpolant!(Whitaker-Shannon Sampling Theorem)

Probability density function (pdf) estimation using isocontours

Application to Image Registration Application to Image Filtering Circular/spherical density estimation in

Euclidean space

18

19

• Given two images of an object, to find the geometric transformation that “best” aligns one with the other, w.r.t. some image similarity measure.

Mutual Information: Well known image similarity measure Viola and Wells (IJCV 1995) and Maes et al (TMI 1997).

Insensitive to illumination changes: useful in multimodality image registration

20

Marginal entropy

Joint entropy

Conditional entropy

)( 1IH

),( 21 IIH

)|( 21 IIH

),(log),(),(

)(log)()(

)(log)()(

2112211221

22222

11111

1 2

2

1

αααα

αα

αα

α α

α

α

ppIIH

ppIH

ppIH

∑ ∑

∑

∑

−=

−=

−=

),()()( )|()(),(

2121

21121

IIHIHIHIIHIHIIMI−+=

−=

)(p 2112 ,ααJoint Probability

)()(

22

11

αα

pp

Marginal Probabilities

Functions of Geometric Transformation

j)(ip ,12 ),( 21 IIH ),(MI 21 II

1I 2I

Hypothesis: If the alignment between images is optimal then Mutual information is maximum.

21

2205.0=σ 2.0=σ 7.0=σ

23

05.0=σ 32 bins128 bins2.0=σ 7.0=σPVI=partial volume interpolation (Maes

et al, TMI 1997)

24

PD slice T2 sliceWarped T2 sliceWarped and Noisy T2 slice

Brute force search for themaximum of MI

25

MI with standardhistograms

MI with our method

7.0=σPar. of affine transf.

0,3.0,30 =−=== ϕθ ts

26

Method Error in Theta(avg., var.)

Error in s(avg.,var.)

Error in t(avg., var.)

Histograms (bilinear)

3.7,18.1 0.7,0 0.43,0.08

Isocontours 0,0.06 0,0 0,0

PVI 1.9, 8.5 0.56,0.08 0.49,0.1

Histograms (cubic)

0.3,49.4 0.7,0 0.2,0

2DPointProb 0.3,0.22 0,0 0,0

32 BINS

Probability density function (pdf) estimation using isocontours

Application to Image Registration Application to Image Filtering Circular/spherical density estimation in

Euclidean space

27

Anisotropic neighborhood filters (Kernel density based filters): Grayscale images

28

∑∑

∈

∈

−

−=

),(),(

),(),(

));,(),((

),());,(),((),(

baNyx

baNyx

baIyxIK

yxIbaIyxIKbaI

σ

σ

Central Pixel (a,b):Neighborhood N(a,b) around

(a,b)

K: a decreasing function (typically Gaussian)

Parameter σ controls the degree of anisotropicity of the smoothing

Anisotropic Neighborhood filters: Problems

29

Sensitivity to the parameter

Sensitivity to the SIZE of the Neighborhood

Does not account for

gradient information

σ

Anisotropic Neighborhood filters: Problems

30

Treat pixels as independent samples

Continuous Image Representation

31

∑∑

∈

∈

−

−

=

),(),(

),(),(

));,(),((

),());,(),((

),(

baNyx

baNyx

baIyxIK

yxIbaIyxIK

baIσ

σ

∫ ∫

∫ ∫−

−=

),(

),(

));,(),((

));,(),((),(),(

baN

baN

dxdybaIyxIK

dxdybaIyxIKyxIbaI

σ

σ

Interpolate in between the pixel values

Areas between isocontours at

intensity α and α+Δ (divided by area of neighborhood)=

Pr(α<Intensity <α+Δ|N(a,b))

Continuous Image Representation

32

∫ ∫

∫ ∫−

−=

),(

),(

));,(),((

));,(),((),(),(

baN

baN

dxdybaIyxIK

dxdybaIyxIKyxIbaI

σ

σ

∑ −×∆+<<→∆

∑ −××∆+<<→∆=

ασααα

ασαααα

));,(()),(|(Area0

));,(()),(|(Area0),(

baIKbaNILim

baIKbaNILimbaI

∑ −×∆+<<→∆

∑ −××∆+<<→∆=

ασααα

ασαααα

));,(()),(|(Pr0

));,(()),(|(Pr0),(

baIKbaNILim

baIKbaNILimbaI

Areas between isocontours: contribute to weights for

averaging.

33

Published in EMMCVPR 2009

Extension to RGB images

34

Joint Probability of R,G,B = Area of overlap of isocontour pairs from R, G, B images

));,,(()),,(Pr(

));,,(()),,(Pr(

),(),,(),,(

)(

)(

σααα

σαααα

α

α

BGRKBGR

BGRKBGR

baBbaGbaR

−∆+<<

−∆+<<=

∑∑

Mean-shift framework

• A clustering method developed by Fukunaga& Hostetler (IEEE Trans. Inf. Theory, 1975).

• Applied to image filtering by Comaniciu and Meer (PAMI 2003).

• Involves independent update of each pixel by maximization of local estimate of probability density of joint spatial and intensity parameters.

35

Mean-shift framework• One step of mean-shift update around (a,b,c) where c=I(a,b).

36.cb)I(a,Set 4.

c)b,(a, (2)(1) 3.)c,b,a(c)b,(a, 2.

σc)y)(I(x,

σb)(y

σa)(xexp:y)w(x,

c , )b,a(

y)w(x,

y)w(x,y))I(x,y,(x,

:)c,b,a( 1.

2

2r

2

2s

2

2s

b)N(a,y)(x,

b)N(a,y)(x,

⇐

⇐

−−−−−−=

≡≡

=∑

∑

∈

∈

changing. stopstill and stepsRepeat

valueintensity updatedodneighborho the of center updated

Our Method in Mean-shift Setting

37

I(x,y) X(x,y)=x Y(x,y)=y

Our Method in Mean-shift Setting

38

);)),(),,(),,((),,(K(

);)),(),,(),,((),,(K(),,(

)),(),,(),,((

),(

),(

σ

σ

baIbaYbaXIYX

baIbaYbaXIYXIYX

baIbaYbaXkkk

baNk

kkkbaNk

kkk

−

−

=∑

∑

∈

∈

k

k

A

A

Facets of tessellation induced by isocontours and the pixel grid

= Centroid of Facet #k. = Intensity (from interpolated image) at . = Area of Facet #k.

),( kk YX

),( kk YXkI

kA

Experimental Setup: Grayscale Images

• Piecewise-linear interpolation used for our method in all experiments.

• For our method, Kernel K = pillbox kernel, i.e.

• For discrete mean-shift, Kernel K = Gaussian.• Parameters used: neighborhood radius ρ=3, σ=3.• Noise model: Gaussian noise of variance 0.003

(scale of 0 to 1).

39

1);( =σzK

0);( =σzK

If |z| <= σ

If |z| >σ

Original Image Noisy Image

Denoised (Isocontour Mean Shift)

Denoised (Gaussian Kernel Mean Shift)

40

Original Image

Noisy ImageDenoised (Isocontour Mean Shift)

Denoised (Std.Mean Shift)

41

Experiments on color images

• Use of pillbox kernels for our method.• Use of Gaussian kernels for discrete mean

shift.• Parameters used: neighborhood radius ρ= 6,

σ = 6.• Noise model: Independent Gaussian noise on

each channel with variance 0.003 (on a scale of 0 to 1).

42

Experiments on color images

• Independent piecewise-linear interpolation on R,G,B channels in our method.

• Smoothing of R, G, B values done by coupled updates using joint probabilities.

43

Original Image

Noisy Image

Denoised (Isocontour Mean Shift)

Denoised (Gaussian Kernel Mean Shift)

44

Original Image

Noisy ImageDenoised (Isocontour Mean Shift)

Denoised (Gaussian Kernel Mean Shift)

45

Observations

• Discrete kernel mean shift performs poorly with small neighborhoods and small values of σ.

• Why? Small sample-size problem for kernel density estimation.

• Isocontour based method performs well even in this scenario (number of isocontours/facets >> number of pixels).

• Large σ or large neighborhood values not always necessary for smoothing.

46

Observations

• Superior behavior observed when comparing isocontour-based neighborhood filters with standard neighborhood filters for the same parameter set and the same number of iterations.

47

Probability density function (pdf) estimation using isocontours

Application to Image Registration Application to Image Filtering Circular/spherical density estimation in

Euclidean space

48

Examples of unit vector data:1. Chromaticity vectors of color values:

2. Hue (from the HSI color scheme) obtained from the RGB values.

49

222

),,(),,(BGR

BGRbgr++

=

−−

−=BGR

BG2

)(3arctanθ

50

222

),,(),,(

iii

iiiiiii

BGR

BGRbgrv++

==Convert RGB values to unit vectors

Estimate density of unit vectors ,;1)(

1∑

=

=N

ii )vK(v

Nvp κ

constantion normalizat)(pCparameterion concentrat

K

uTve)(pC)u,;v(K

=κ=κ=

κκ=κ

Kernel Fisher Mises-von

voMF mixture

modelsBanerjee et al (JMLR

2005)

Other popular kernels: Watson,

cosine.

51

Estimate density of RGB using KDE/Mixture models

Density of (magnitude,chromaticity)

using random-variable transformation

222

2 ),,(),(

BGRm

BGRpmvmp

++=

=

222

0

2 ),,( )(

BGRm

dmBGRpmvpm

++=

= ∫∞

=Density of chromaticity

(integrate out magnitude)

∑=

−+−+−−=

N

iiBBiGGiRR

NBGRp

1 2

2)(2)(2)(exp1),,(

σ

Projected normal estimator:Watson,”Statistics on spheres”,

1983,Small,”The statistical theory of

shape”, 1995

Density of chromaticity: conditioning on m=1.

( ) ( )

2

1

,,

exp2

11)(

σκ

κκπ

ii

i

iii

iT

i

N

i io

mmwvwm

vvIN

vp

===

= ∑=

Variable bandwidth voMF KDE:Bishop, “Neural networks for

pattern recognition” 2006.

What’s new? The notion that all estimation can proceed in

Euclidean space.

52

Estimate density of RGB using KDE/Mixture models

Use random variable transformation to get density of HSI (hue, saturation,intensity)

Integrate out S,I to get density of hue

Consistency between densities of Euclidean and unit vector data (in terms of random variable transformation/conditioning).

Potential to use the large body of literature available for statistics of Euclidean data (example: Fast Gauss Transform Greengard et al (SIAM Sci. Computing 1991), Duraiswami et al (IJCV 2003).

Model selection can be done in Euclidean space.

53