Background Subtraction for Temporally Irregular Dynamic ...
Transcript of Background Subtraction for Temporally Irregular Dynamic ...
Background Subtraction for Temporally Irregular
Dynamic TexturesGerald Dalley, Joshua Migdal,
and W. Eric L. Grimson
Workshop on Applications of Computer Vision (WACV) 2008
8 Jan. 2008 2
Key Prior Approaches
l MoG – learn modes for all pixels & track changesl Works as long as dynamic textures are regular
l Per-pixel kernels (Mittal & Paragios)l Can pay attention to higher-level features like optical flowl How many kernels to keep?
l Spatial kernels (Sheikh & Shah)l Handles temporal irregularity in dynamic texturesl How many kernels to keep?
8 Jan. 2008 3
Results with Mittal & Paragios Traffic Clip
Key:l bboxes from
ground truthl True
positivel False
positivel False
negative
l No ground truth available
8 Jan. 2008 4
Our Goals
l Compactnessl Speedl Complexity growth
l Grow with scene complexity, not with the amount of time to see the complexity§ If a leaf blows in front of a pixel once a minute, we don’t
want to have to keep 1800 kernels
l Usable in MoG frameworksl MoG variants are very widely used (original
Stauffer & Grimson paper has 1000+ citations)
8 Jan. 2008 5
Pixels Affected by a GivenMixture ComponentOur Model
l Mixture of Gaussians (MoG)
Standard MoG
Ours
p(ci|Φ) ∝∑
j∈NiwjN (ci;µj ,Σj)
ci the observed color at pixel i
Φ the model {wj , µj ,Σj}j
Ni neighborhood of pixel i
p(ci|Φ) ∝∑
j∈NiwjN (ci;µj ,Σj)
ci the observed color at pixel i
Φ the model {wj , µj ,Σj}j
Ni neighborhood of pixel i
8 Jan. 2008 6
Foreground/Background Classification
l Find best matching background Gaussian, jl Use neighborhood
l Squared Mahalanobis Distance
-
=
input
d2ij
mixture model
Φ1 Φ2 Φ3
dij = (ci − µj)TΣ−1
j (ci − µj)dij = (ci − µj)TΣ−1
j (ci − µj)
8 Jan. 2008 7
hard Stauffer-Grimson
soft Stauffer-Grimson
best match only
update all matches
Model Update OptionsHard UpdatesSoft Updates
Loca
lR
egio
nal
8 Jan. 2008 8
Results on a Classic Dataset: Wallflower WavingTrees
Ground
Truth
Toyam
aet al.
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
8 Jan. 2008 9
ROC (Wallflower)
8 Jan. 2008 10
Traffic #410G
round T
ruthM
ittal &
Paragios
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
8 Jan. 2008 11
Traffic #150G
round T
ruthM
ittal &
Paragios
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
8 Jan. 2008 12
Pushing the Limits:Ducks
Ground
Truth
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
8 Jan. 2008 13
Pushing the Limits:Jug
Ground
Truth
Zhong
&
Sclaroff
Best
MoG
Ours
Final ResultAfter MRFMahalanobisDistances
8 Jan. 2008 14
Parameter Sensitivity
0 1000 2000 3000 4000 5000 6000 70000
0.2
0.4
0.6
0.8
1
min number of mistakes
freq
uenc
y
ws=7ws=5ws=3ws=1
traffic
0 500 1000 1500 20000
0.05
0.1
0.15
min number of mistakes
freq
uenc
y
ws=7ws=5ws=3ws=1
wallflower
max max
8 Jan. 2008 15
Summary
l Model representationl Same as MoG
l FG/BG Classifier Inputl Add local neighborhood loop
l Updatel Several options, including standard MoG updates
l Resultsl Reduces foreground false positivesl Reduces need for exotic input features (e.g. optical flow)l Cost window size
8 Jan. 2008 16
Also in the Paper…
l Analysisl Timing experimentsl More ROC analysis
l Implementation Detailsl MRF choicel Update formulas
8 Jan. 2008 17
Thank You
Questions...
8 Jan. 2008 18
Backup Slides
8 Jan. 2008 19
PureSoft SoftLocal PureHard HardLocal0
10
20
30
40
50
60R
elat
ive
Tim
e C
ost
Model Update Computational Costs
W=12
W=32
W=52
W=72
8 Jan. 2008 208 Jan. 2008 20
PureSoft SoftLocal PureHard HardLocal0
20
40
60
80
100R
elat
ive
Tim
e C
ost
Mahalanobis Map Computational Costs
W=12
W=32
W=52
W=72
8 Jan. 2008 21
1
2
3
4
5
6
7
8
1 2 3 4 1 2 3 4 1 2 3 41 2 3 4
Background
Model
Observed
Image
9x9
Matching
3x3
Matching
Small Windows are Still Useful
8 Jan. 2008 22
c(t−1)ic(t−1)ic(t−1)i c
(t)ic(t)ic(t)i
φ(t)jφ(t)j
z(t−1)iz(t−1)i
l(t−1)jl(t−1)jl(t−1)j l
(t)jl(t)jl(t)j
JJ
II
z(t)iz(t)i
φ(t−1)jφ(t−1)j
Generative Model
Particle locations
Particle appearances
Pixel-to-particle assignments
Pixel appearance
8 Jan. 2008 23
Independent Assignments
l(t−1)1
l(t−1)2
l(t−1)3
z(t)i = 2
l(t)1
l(t)2
l(t)3
l(t−1)1l(t−1)1l(t−1)1
l(t−1)2l(t−1)2l(t−1)2
l(t−1)3l(t−1)3l(t−1)3
z(t)i = 2z(t)i = 2z(t)i = 2
l(t)1l(t)1l(t)1
l(t)2l(t)2l(t)2
l(t)3l(t)3l(t)3