Post on 25-Jan-2016
description
MIT AI Lab / LIDS
Artificial Intelligence Laboratory &Laboratory for Information and Decision SystemsMassachusetts Institute of Technology
A Unified Multiresolution Framework for
Automatic Target Recognition
Eric Grimson, Alan Willsky, Paul Viola, Jeremy S. De Bonet, and John Fisher
MIT AI Lab / LIDS
Outline
• Review Multiresolution Analysis Models– MAR (Multiresolution Auto-Regressive)
– MNP (Multi-scale Nonparametric)
• Applications of MNP Models– Classification/Recognition
– Segmentation and Multi-Look Registration
– Synthesis and Super-Resolution
• Continuing Efforts
MIT AI Lab / LIDS
MAR Processes for SAR
Pyramid Residuals
Irving,Willsky &Novak
MIT AI Lab / LIDS
Intuition: Construct a Model for the Scale-to-scale Dependency in SAR imagery
V(x,y)={ }
Parent Vector
),(,
2,
2,...,
2,
2,
2,
2, 01111 yxI
yxI
yxI
yxIyxV
NNNNNN
L0
L1
L2L3
L4
MIT AI Lab / LIDS
Build a Model for Observed Distribution
yxVP ,
IWN: Conditionally Gaussian
MIT AI Lab / LIDS
MIT AI Lab / LIDS
Multi-scale Non-parametric Models
• Two key insights:– Alternative multi-scale representation
• Sub-band oriented representations (Wavelets, Gabor Filters)
– Non-parametric models of conditional dependence
De Bonet & Viola (1997)
MIT AI Lab / LIDS
Freeman and Simoncelli
Steerable Pyramids
),( yxFl ),( yxFl
MIT AI Lab / LIDS
…for a SAR image
MIT AI Lab / LIDS
V(x,y)={ }
coarse fine
Parent Vector
Multiresolution parent vector
yxFyxFyxF
yxF
yxF
yxF
yxF
yxF
yxFyxV
MNNN
M
NNM
NNNNNNN
,,,,,,
,2
,2
,,2
,2
,2
,2
,2
,2
,,2
,2
,2
,2
,
10
11
10
1
10
MIT AI Lab / LIDS
Build a Model for Observed Distribution
yxVP ,
DB: Non-parametricDistribution
MIT AI Lab / LIDS
Probabilistic Model
)...,(),,(|),(
),(}{|),(
21 yxVyxVyxVP
yxVFeaturesyxVP
lll
ll
)...,(),,(|),(
),(}{|),(
21 yxVyxVyxVP
yxVFeaturesyxVP
lll
ll
...),,|(
),|(
)|()()()(
2103
102
010
VVVVP
VVVP
VVPVPfeaturesPIP
...),,|(
),|(
)|()()()(
2103
102
010
VVVVP
VVVP
VVPVPfeaturesPIP
yx
llll yxVyxVyxVPVP,
21 ...),(),,(|),( yx
llll yxVyxVyxVPVP,
21 ...),(),,(|),(
Markov
ConditionallyIndependent
SuccessiveConditioning
MIT AI Lab / LIDS
Estimating Conditional Distributions
...),(),,(
...),(),,(),,(
...),(),,(
...),(),,(),,(
...),(),,(|),(
21*
21*
21
21
21
yxVyxVP
yxVyxVyxVP
yxVyxVP
yxVyxVyxVP
yxVyxVyxVP
ll
lll
ll
lll
lll
...),(),,(
...),(),,(),,(
...),(),,(
...),(),,(),,(
...),(),,(|),(
21*
21*
21
21
21
yxVyxVP
yxVyxVyxVP
yxVyxVP
yxVyxVyxVP
yxVyxVyxVP
ll
lll
ll
lll
lll
i
ixxRxP )()(* i
ixxRxP )()(*• Non-parametrically
MIT AI Lab / LIDS
distribution Similarity
sample
Likelihood
distribution condition
exampleimage
synthesis
discrimination
registration
segmentation
denoising
super resolution
Outline
MIT AI Lab / LIDS
Capturing Structure (Texture Perspective)
MIT AI Lab / LIDS
Synthesis Results
MIT AI Lab / LIDS
Synthesis Results
MIT AI Lab / LIDS
MIT AI Lab / LIDS
Alternative 1: Gaussian Distribution: GMRF
2
221
||
|)(|
),,()(
WWI
I
e
e
INIP
2
221
||
|)(|
),,()(
WWI
I
e
e
INIP
W
W
W
21
W
W
W
21
Chellappa and Chattergee
MIT AI Lab / LIDS
MIT AI Lab / LIDS
Alternative 2: Statistical Wavelet Models
yxlll
yxllyxl
yxFP
yxFPObsP
,,,,
,,,,,,
),(
),()(
yxlll
yxllyxl
yxFP
yxFPObsP
,,,,
,,,,,,
),(
),()(
(.):
)},({:
:
,
lPModel
yxFObserve
WGiven
l
(.):
)},({:
:
,
lPModel
yxFObserve
WGiven
l
Bergen and Heeger
Donoho
Simoncelli and Adelson
MIT AI Lab / LIDS
Heeger and Bergen Texture Synthesis Model
MIT AI Lab / LIDS
Heeger and Bergen Texture Synthesis Model
MIT AI Lab / LIDS
orig
inal
text
ure
patc
h
synt
hesi
zed
text
ure
patc
h
Sam
plin
g P
roce
dure
Analysis Synthesis
MIT AI Lab / LIDS
Not quite right...
Very similar to a Gaussian Model(i.e. no phase alignment)
MIT AI Lab / LIDS
Wavelet Representation of Edges
WaveletTransform
MIT AI Lab / LIDS
Pyramid Representation
MIT AI Lab / LIDS
Conditional Distributions
WaveletTransform
MIT AI Lab / LIDS
orig
inal
text
ure
patc
h
Sam
plin
g P
roce
dure
Analysis Synthesis
synt
hesi
zed
text
ure
patc
h
MIT AI Lab / LIDS
Multiresolution progression
MIT AI Lab / LIDS
MIT AI Lab / LIDS
Joint feature occurrence across resolution
MIT AI Lab / LIDS
Joint feature occurrence across resolution
MIT AI Lab / LIDS
MIT AI Lab / LIDS
MIT AI Lab / LIDS
Texture Synthesis Results
MIT AI Lab / LIDS
BMP2-C21 BTR70-C71 T72-132
Models
Models for target vehicles were generated from example images:
• generated from vehicles with different numbers from the target vehicles• only 10 examples, evenly distributed in heading angle• measured at a depression angle of 17 degrees (targets were at 15 degrees)
MIT AI Lab / LIDS
BMP2-9563 BMP2-9566
BTR70-C71
T72-812 T72-S7
Target vehicles
• Five target vehicles were used.
• Vehicles which differed from the target class were included as confusion targets.
• There were roughly 200 images in each class.
MIT AI Lab / LIDS
2S1 BRDM2 D7 T62
ZIL131 ZSU23
Confusion vehicles
Six additional confusion vehicles were used as well.
MIT AI Lab / LIDS
BMP2-C21 BTR70-C71 T72-132
Flexible Histograms
Template Matching
De Bonet, Fisher and Viola
MIT AI Lab / LIDS
Rtie-point
Rtest
parent structure
B (x,y)= 8
Measuring Visual Structure : Flexible Histogram II
MIT AI Lab / LIDS
Rlandmark
B(,x,y)= 8
2= (B-B’)2/B
B’(x,y)= 3
Rtie-point
Rtest
Measuring Visual Structure : Flexible Histogram III
MIT AI Lab / LIDS
Registration pipeline
Tie-pointdetermination
Multiresolution texture match:flexible histograms
Multiresolutionalignmentsearch
MIT AI Lab / LIDS
Tie-point determination
MIT AI Lab / LIDS
Here, only vehicles provide distinct landmarks.
When present, roads and buildings provide useful landmarks as well.
Tie-point examples
MIT AI Lab / LIDS
Coarse Fine
Coarse to fine alignment
MIT AI Lab / LIDS
Example Registration
MIT AI Lab / LIDS
Example Registration
MIT AI Lab / LIDS
Example Registration
MIT AI Lab / LIDS
MIT AI Lab / LIDS
MIT AI Lab / LIDS
MIT AI Lab / LIDS
distribution | | condition
Prior beliefs about natural images
image analysis
Image + Noise
maximum likelihood sample
Blind Image Denoising
MIT AI Lab / LIDS
MIT AI Lab / LIDS
MIT AI Lab / LIDS