Multiscale Analysis for Intensity and Density

Estimation

Rebecca Willett’s MS Defense

Thanks to Rob Nowak, Mike Orchard,Don Johnson, and Rich Baraniuk

Eric Kolaczyk and Tycho Hoogland

Poisson and Multinomial Processes

Why study Poisson Processes?

Astrophysics

Network analysis

Medical Imaging

Examining data at different resolutions (e.g., seeing the forest, the trees, the leaves, or the dew)yields different information about the structure of the data.

Multiresolution analysis is effective because it sees the forest (the overall structure of the data)without losing sight of the trees (data singularities)

Multiresolution Analysis

Beyond Wavelets

Multiresolution analysis is a powerful tool, but what about…

Edges?Nongaussian noise?Inverse problems?

Piecewise polynomial- and platelet- based methods address these issues.

Non-Gaussian problems?Image Edges?

Inverse problems?

Computational Harmonic Analysis

I. Define Class of Functions to Model SignalA. Piecewise PolynomialsB. Platelets

II. Derive basis or other representationIII. Threshold or prune small

coefficientsIV. Demonstrate near-optimality

Approximating Besov Functions with Piecewise

Polynomials

Nr

Nd

Nd

O :Decay Error Discrete

dO:RateDecay Error23r-

r2

Approximation with Platelets

50 100 150 200 250

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Consider approximating this image:

E.g. Haar analysis

Terms = 2068, Params = 206850 100 150 200 250

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Wedgelets

Original Image Haar Wavelet Partition Wedgelet Partition

E.g. Haar analysis with wedgelets

Terms = 1164, Params = 116450 100 150 200 250

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

E.g. Platelets

Terms = 510, Params = 77450 100 150 200 250

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Error Decay

Platelet Approximation Theory

Error decay rates:• Fourier: O(m-1/2)• Wavelets: O(m-1)• Wedgelets: O(m-1)• Platelets: O(m-min(,))

)C(lderoH 2,

)C(lderoH 1,

4,0

0,0

1,0 1,1 1,11,0

0,0

3,0

2,0

1,0

0,0

1,1

2,1 2,2 2,3

3,1 3,2 3,3 3,4 3,5 3,6 3,7

4,1 4,2 4,3 4,4 4,5 4,6 4,7 4,8 4,9 4,10 4,11 4,12 4,13 4,14 4,15

A Piecewise Constant Tree

0,0

1,0 1,0

0,0

2,0

1,0

0,0

1,1

2,1

4,8 = 4,9 = 4,10 = 4,11 = 4,12 = 4,13 = 4,14 = 4,154,4 = 4,5 = 4,6 = 4,74,0 = 4,1 = 4,2 = 4,3

A Piecewise Linear Tree

0,0

1,0 1,0

0,0

2,0

1,0

0,0

1,1

2,1

4,k = a0 + k a14,4 = 4,5 = 4,6 = 4,74,0 = 4,1 = 4,2 = 4,3

Maximum Penalized Likelihood Estimation

Goal: Maximize the penalized likelihoodGoal: Maximize the penalized likelihood

So the MPLE isSo the MPLE is

)ˆ(ˆ

))((Lmaxargˆ

θμμ

θμθθ

θθ

μ|x

θμ|xμ

in parameters of number the is }{#

parameter smoothing a is

likelihood the is )(p

where

}{#)(plog)(L

The Algorithm

Data

Const Estimate

Wedge Estimate

Platelet Estimate

Wedged Platelet Estimate

Inherit from finer scale

Algorithm in Action

Penalty ParameterPenalty parameter balances between fidelity to the data (likelihood) and

complexity (penalty).

= 0 Estimate is MLE: Estimate is a constant:

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

Variance

Bia

s2

Different values

xμˆxμˆ

Nlogc Choose

Penalization

10-2 10-1 100 10110-4

10-3

10-2

10-1

100

/ log(# of photon counts)

Mea

n M

SE

Bowl

Ave. Counts/Pixel = 1Ave. Counts/Pixel = 10Ave. Counts/Pixel = 100

Density Estimation - Blocks

Density Estimation - Heavisine

Density Estimation - Bumps

Density Estimation Simulation

Medical Imaging Results

Inverse Problems

Goal: estimate from observationsx ~ Poisson(P)

EM algorithm (Nowak and Kolaczyk, ’00):

),(Qmax:Step-M

]x|)(L[E),(Q:StepE

i

ii

Confocal Microscopy: An Inverse Problem

Platelet Performance

2 4 6 8 10 12 14

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

Iteration

Err

or

MLE (min error level shown in dots)Piecewise ConstantPlatelet

Confocal Microscopy: Real Data

Hellinger Loss

• Upper bound for affinity

(like squared error)

• Relates expected error to Lp approximation bounds

2

''2 )x(p)x(pp,pH

2/1)x(q)x(p)q,p(A

Bound on Hellinger Risk

)'(penp,pKLminp,pHE

satisfiesˆ of riskthen,1eIf

''

ˆ2

'

)'(pen

KL distanceApproximation error

Estimationerror

(follows from Li & Barron ’99)

Bounding the KL

• We can show:

• Recall approximation result:

• Choose optimal d

2

2'

cN2

Nn

p,pKLN1

μμμ'μ

Nr

Nd

Nd

O

23r-2

2

μ'-μ

Near-optimal Risk

• Maximum risk within logarithmic factor of minimum risk

• Penalty structure effective:

1r2

r22

ˆ21r2

r2

N)N(log

O)p,p(HN1

N1

O μμ

Nlogc

Conclusions

CHA with Piecewise Polynomials or Platelets

• Effectively describe Poisson or multinomial data

• Strong approximation capabilites• Fast MPLE algorithms for

estimation and reconstruction• Near-optimal characteristics

Future Work

• Risk analysis for piecewise polynomials

• Platelet representations and approximation theory

• Shift-invariant methods

• Fast algorithms for wedgelets and platelets

• Risk Analysis for platelets

Major Contributions

Approximation Theory Results

.ˆ

22

ˆ

).(),(

)1(),(),(),(

),min(,

2

1,2,

)}({2)}({1

mKff

m

Jmf

ClderoHHClderoHf

IyxfIyxfyxf

L

J

i

xHyxHy

then , so ion,approximat

platelet resolution scale,- term,- the is If

and where

functions of class the Consider

Why don’t we just find the MLE?

xλ

λ|xλ

λx

ˆ

01))(plog(

!x

λe )|p(

i

xi

λ- ii

i i

i

i

x

MPLE Algorithm (1D)

Multiscale Likelihood Factorization

Probabilistic analogue to orthonormal wavelet decomposition

Parameters wavelet coefficients Allow MPLE framework, where

penalization based on complexity of underlying partition

Poisson Processes

• Goal: Estimate spatially varying function, (i,j), from observations of Poisson random variables x(i,j) with intensities (i,j)

• MLE of would simply equal x. We will use complexity regularization to yield smoother estimate.

Accurate Model

Parsimonious ModelComplexity

Regularization

Penalty for each constant region results in fewer splits

Bigger penalty for each polynomial or platelet region more degrees of freedom, so more efficient to store constant if likely

Astronomical Imaging