ICPR 2012
-
Upload
bouwmans-thierry -
Category
Technology
-
view
288 -
download
1
description
Transcript of ICPR 2012
Foreground Detection via Robust Low Rank MatrixFactorization including Spatial Constraint with Iterative
Reweighted Regression
C. Guyon, T. Bouwmans and E. [email protected]
MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, FrancePresenter: Muriel Visani (L3i lab - University of la Rochelle)
—ICPR2012, Tsukuba, Japan
November 14, 2012
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 1 / 25
Summary
1 Introduction and motivation on IRLS
2 Temporal constraint with an adapted norm
3 Diagram flow and spatial constraint
4 Experimental Results
5 Conclusion
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 2 / 25
Introduction and motivation
Purpose
Foreground detection : Segmentation of moving objects in video sequenceacquired by a fixed camera.
Background modeling : Modelization of all that is not moving object.
Involved applicationsSurveillance cameraMotion capture
On the importanceCrucial Task : Often the first step of a full video surveillance system.
Strategy usedEigenbackground decomposition.
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 3 / 25
Eigenbackgrounds
Find an « ideal » subspace of the video sequence, which describes the bestas possible the (dynamic) background.
Fig.1 The common process of background subtraction via PCA (Principal ComponentAnalysis). At the final step, an adaptative threshold is used to get a binary image.
Without a robust framework, the moving object may be absorbed in the model !C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 4 / 25
Data Structure Transformation
First, we consider a video sequence as a matrix A ∈ Rn×m
n is the amount of pixels in a frame (∼ 106)m is the number of frames considered (∼ 200)
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 5 / 25
IRLS : Vector version (1)
The usual IRLS (Iteratively Reweighted Least Squares) scheme for solveargmin
x||Ax− b||α is given by :
D(i) = diag((ε+ |b − Ax (i)|)α−2)x (i+1) = (AtD(i)A)−1AtD(i)b
(1)
This IRLS method is convergent for 1 ≤ α < 3. An more suitableformulation is :
r (i) = b − Ax (i)
D = diag((ε+ |r (i)|)α−2)y (i) = (A′DA)−1A′Dr (i)
x (i+1) = x (i) + (1+ λopt)y(i)
(2)
for λopt computed in a second inner loop. It is convergent for 1 ≤ α < +∞
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 6 / 25
IRLS : Vector version (2)
For spatio/temporal RPCA, it needs to solve the following general problem :
argminx
||Ax − b||α + λ||Cx − d ||β (3)
By derivation, the associated IRLS scheme is,
r1 = b − Ax (i), r2 = d − Cx (i), e1 = ε+ |r1|, e2 = ε+ |r2|D1 = (
∑eα1 )
1α−1diag(eα−21 ),D2 = λ(
∑eβ2 )
1β−1diag(eβ−22 )
y (i) = (A′D1A + C ′D2C)−1(A′D1r1 + C ′D2r2)
x (i+1) = x (i) + (1 + λopt)y(i)
(4)
Good news : Just few lines in Matlab !
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 7 / 25
IRLS : Matrix Version
More generally, we consider the following matrix regression problem withtwo parameters norm (α, β) and a weighted matrix W ,
minX||AX − B||α,β
W
with ||Mij ||α,βW
= (n∑
i=1
(m∑j=1
Wij |Mij |β)αβ )
1α (5)
The problem is solved in the same manner on matrices with a reweightedregression strategy,
Until X is stable, repeat on each k-columnR ← B − AXS ← ε+ |R|Dk ← diag(Sβ−2ik ◦ (
∑j(S
βij ◦Wij))
αβ−1 ◦Wik)k
Xik ← Xik +(1+Λ(max(α, β)))(AtDkA)−1AtDkRik
(6)
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 8 / 25
Various RPCA formulation (only for α = 1)
PCA with a fixed rank is : minL,S
||S ||Fs.t. Rank(L) = k
A = L + S
(7)
R(obust)PCA is (Non convex and NP-hard) :
minL,S
||σ(L)||0 + λ||S ||0s.t. A = L + S
(8)
Convex relaxed problem of (8) is RPCA-PCP proposed by Candès et al. [1] :
minL,S
||σ(L)||1 + λ||S ||1s.t. A = L + S
(9)
where σ(L) means singular values of L.A mix is Stable PCP of Zhou et al. [2] (both entry-wise and sparse noise) :
minL,S
||σ(L)||1 + λ||S ||1s.t. ||A− L− S ||F < δ
(10)
All of them could be solved by Augmented Lagrangian Multipliers (ALM).
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 9 / 25
Video examples
Some examples, temporal RPCA and ideal RPCA with ground truth fitting.
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 10 / 25
Summary
1 Introduction and motivation on IRLS
2 Temporal constraint with an adapted norm
3 Diagram flow and spatial constraint
4 Experimental Results
5 Conclusion
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 11 / 25
Sparse solution
In RPCA, residual error is sparse.Using the RPCA decomposition on a synthetic low-rank random matrixplus noise, the error looks like :
Same principle with video. Sparse noise (or outliers) are the moving objects.
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 12 / 25
Let’s play with norms
Varying the α, β norm → Different kind of recovering pattern error.
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 13 / 25
Let’s play with norms...(2)
Some issuesWhat is the best specific norm for temporal constrain ?Initial assumption is ||.||2,1. Confirmed experimentally ?
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 14 / 25
Validation
If ideal eigenbakgrounds are that, best norm should be ...
Let us denote Lopt , the ideal low-rank subspace which outliers do not contribute to PCA
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 15 / 25
Experimental validation
Let us denote Lα,β , the low-rankrecovered matrix with a ||.||α,β-PCA.The plot shows the error between||Lopt − Lα,β||F for parameters αand β chosen freely. The darkestvalue means that the error is thesmallest here.
||S ||2,1 is not optimal, but for convenience we use it.The benefit of the ad hoc block-sparse hypothesis is confirmed bytesting its efficiency directly on video dataset.
Experimentation done on dynamic category of dataset change detectionworkshop 2012 : http://www.changedetection.net/
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 16 / 25
Summary
1 Introduction and motivation on IRLS
2 Temporal constraint with an adapted norm
3 Diagram flow and spatial constraint
4 Experimental Results
5 Conclusion
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 17 / 25
Overview & addition of a spatial constraint via TV
Figure: Overview of the learning and evaluation process. Learning process needsGT (Ground Truth) for better fits the eigenbackground components.
Spatial Constraint via TVSuppose A = L+ S where L and S are computed via some kind ofRPCA techniques with the addition of Total Variation penalty on S .This penalty increases connected (or connexe) shapes.
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 18 / 25
Exemple with a synthetic 1-D signal
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 19 / 25
Summary
1 Introduction and motivation on IRLS
2 Temporal constraint with an adapted norm
3 Diagram flow and spatial constraint
4 Experimental Results
5 Conclusion
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 20 / 25
Experimental Protocol
RPCA-IRLS is compared for the following four recent robust methods :
Low-Rank Block sparse Decomposition (LBD, 2011) [3]
Low-Rank Representation (LRR, 2011) [4]
Symmetric Alternating Direction Augmented Lagrangian (SADAL, 2011) [5]
Grassmannian Robust Adaptive Subspace Tracking Algorithm (GRASTA, 2012) [6]
References[1] E. Candes, X. Li, Y. Ma, and J. Wright, Robust principal component analysis, International Journal of ACM,
vol. 58, no. 3, May 2011.
[2] Z. Zhou, X. Li, J. Wright, E. Candes, and Y. Ma, Stable principal component pursuit,IEEE ISITProceedings, pp. 1518-1522, Jun. 2010.
[3] G. Tang and A. Nehorai, Robust principal component analysis based on low-rank and block-sparse matrixdecomposition, CISS 2011, 2011.
[4] Z. Lin, R. Liu, and Z. Su. Linearized alternating direction method with adaptive penalty for low-rankrepresentation. NIPS 2011, Dec. 2011.
[5] S. Ma. Algorithms for sparse and low-rank optimization : Convergence, complexity and applications. Thesis,2011.
[6] J. He, L. Balzano, and A. Szlam. Incremental gradient on the grassmannian for online foreground andbackground separation in subsampled video. Conference on Computer Vision and Pattern Recognition(CVPR), June 2012.
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 21 / 25
Experimental Protocol & Quantitative Results
Optimal threshold is chosen for maximizing F-measure criterionwhich is based 2× 2 histogram of True/false/positive/negative :
DR =TP
TP + FN, Prec =
TP
TP + FP, F =
2 DR Prec
DR + Prec
Good performance is then obtained when the F-measure is closed to 1Time consumption is not take into account in the evaluation process.
Figure: F-Measure on the Wallflower and I2R dataset.
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 22 / 25
Quantitative Results
Here, we show other experimental results on the real dataset of BMC 2012,
Video Recall Precision F-measure PSNR Visual Results1 0.9139 0.7170 0.8036 38.24252 0.8785 0.8656 0.8720 26.77213 0.9658 0.8120 0.8822 37.70534 0.9550 0.7187 0.8202 39.36995 0.9102 0.5589 0.6925 30.58766 0.9002 0.7727 0.8316 29.99947 0.9116 0.8401 0.8744 26.83508 0.8651 0.6710 0.7558 30.50409 0.9309 0.8239 0.8741 55.1163
Table: Quantitative results with common criterions. Last column : sample of theoriginal video, GT and our results of the first four real video sequences.
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 23 / 25
Summary
1 Introduction and motivation on IRLS
2 Temporal constraint with an adapted norm
3 Diagram flow and spatial constraint
4 Experimental Results
5 Conclusion
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 24 / 25
Conclusion
Advantages
Experiments on video surveillance datasets show that this approach is morerobust than other recent RPCA formulation in presence of dynamicbackgrounds (DC) and illumination changes (IC).
Well suited for video with spatially spread and temporarily sparse outliers.
Disadvantages
Small local motions, like « waving trees » are not (yet) well modelized bythis kind of global PCA. For example, IC needs few eigenBackground andDC needs more with the risk to integrate moving objects into the model.
Future WorksLack in computation time : Further research consists in developping anincremental version to update the model at every frame and to achieve thereal-time requirements.
C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 25 / 25