OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22...
Transcript of OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22...
![Page 1: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/1.jpg)
OPPA European Social FundPrague & EU: We invest in your future.
![Page 2: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/2.jpg)
� density propagation
� importance sampling
� efficient 3D head tracking by particle filter
� 2D tracking
Particle Filteringaka Condensation, Sequential Monte Carlo
(SMC), . . .
Tomáš Svoboda, [email protected] Technical University in Prague, Center for Machine Perception
http://cmp.felk.cvut.czLast update: April 12, 2010
![Page 3: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/3.jpg)
2/22What is tracking?
� At a certain time we need decide about one state (position) of thetarget object.
� Inner state representation can be arbitrary.
� Let represent the state of the object by probability density.
� Representing of the probability density by particles is one of theeffective choices.
![Page 4: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/4.jpg)
2/22What is tracking?
� At a certain time we need decide about one state (position) of thetarget object.
� Inner state representation can be arbitrary.
� Let represent the state of the object by probability density.
� Representing of the probability density by particles is one of theeffective choices.
Particle filter: Particles a the input, measurements, update, . . . , particles atthe output.
![Page 5: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/5.jpg)
3/22Particle filter in computer vision
� technique known outside computer vision for long
� popularized under the acronym Condensation in 1996 [4]
� Condensation stands for CONditional DENSity propagATION
� simple, easy to implement, robust . . .
� frequently used in many algorithms
� comprehensive overview [2]
![Page 7: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/7.jpg)
5/22Particle filtering
Input: St−1 ={(s(t−1)i, π(t−1)i)
}, i = 1, 2, . . . , N .
Output: St and object state (position) if required
Workflow for time t
1. Resample data St−1 by using importance sampling.
2. Predict s̃(t)i, think about position and velocity model.
3. Uncertainty in the state change → noisify the predicted states.
4. Measure how well the predicted states fit the observation, and updateweights πt.
5. If needed compute the mean state (where is the target, actually?).
6. Update the prediction model if used.
![Page 9: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/9.jpg)
7/22Importance sampling
Input: set of samples with associated probabilities
Ouput: new set of samples where the frequency depends proportionally ontheir probabilities
1 2 3 4 5 6 7 8 9 100
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x variable
rela
tive
freq
uenc
y
1 2 3 4 5 6 7 8 9 100
5
10
15
20
25
30
35
xnew
variable
freq
uenc
y
![Page 10: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/10.jpg)
8/22Importance sampling
video: importance sampling
![Page 11: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/11.jpg)
9/22Example: 1–D tracking
video: 1–D tracking
![Page 12: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/12.jpg)
10/22Example: 1–D tracking, closer look
10 20 30 40 50 60 70 80 90 100−4
−3
−2
−1
0
1
2
3
4
time step
stat
e va
lue
state of the systemestimated valuesamples
−4 −3 −2 −1 0 1 2 3 40
0.5
1step 13
measurement functiontrue state
−4 −3 −2 −1 0 1 2 3 40
0.01
0.02
weighted samplesestimated state
−4 −3 −2 −1 0 1 2 3 40
0.5
1step 53
measurement functiontrue state
−4 −3 −2 −1 0 1 2 3 40
0.02
0.04
weighted samplesestimated state
![Page 13: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/13.jpg)
11/22
Application: 3D head tracking in multicamerasystem
video
![Page 14: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/14.jpg)
12/22
3D head tracking in multicamerasystem—essentials
Assume calibrated system, Pj, and motion segmentated projections
video
� Head modeled as ellipsoid
� State comprises position, orientation, velocity vector . . .
� Ellipsoid project as ellipses into cameras
� We measure how far are the ellipses from contours
We will go step by step . . .
![Page 15: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/15.jpg)
13/22Ellipsoid and its 2D projection
Quadric surface Q
X>QX = 0
project to a (line) conic
C∗ = PQ∗P>
point conic C which is dual to C∗
u>Cu = 0
3
3Image from [3]
![Page 16: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/16.jpg)
14/22Measurement in (multiple) images
Remeber, we can efficiently project outline of the ellipsoid to images.
video: segmented data video: distance map to edges
Chamfer distance
� distance map computed just once per image
� measuring samples is just reading out values from a table
![Page 17: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/17.jpg)
15/22Head 3D tracking — results
video: 3D localization results video: example of particles convergence
Problem: 3D position only, no orientation . . .
![Page 18: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/18.jpg)
16/22Learning appearance
video: Learning head appearance
� Combines stereo and gradient based localization.
� Explanation of the principle [PDF; www4]. More in [6].
4http://cmp.felk.cvut.cz/projects/multicam/Demos/3Dtracking.html
![Page 20: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/20.jpg)
18/223D tracking — similarity measure
Oponent colors
a =12(R−G) , b =
14(2B −R−G) , a, b ∈ 〈−128, 127〉.
Histogram of oponent colors
0
5
10
15
20
0
5
10
15
20
0
0.05
0.1
0.15
0.2
a channel of opponent colorsb channel of opponent colors
Bhattacharya distance
bhatta(I,M) =∑k,l
√Ik,l · Mk,l .
![Page 21: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/21.jpg)
19/223D tracking — Results
video: 3D tracking including orientation
No post-processing, no smoothing applied.
![Page 22: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/22.jpg)
20/222D tracking — object modeled by color histogram
video
![Page 23: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/23.jpg)
21/22References
[1] Andrew Blake and Michael Isard. Active Contours : The Application of Techniques form Graphics, Vision,Control Theory and Statistics to Visual Tracking of Shapes in Motion. Springer, London, Great Britain,1998. On-line available at http://www.robots.ox.ac.uk/˜contours/.
[2] Arnaud Doucet, Nando De Freitas, and Neil Gordon. Sequential Monte Carlo Methods in Practice.Springer, 2001.
[3] R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press,Cambridge, UK, 2000. On-line resources at:http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook1.html.
[4] Michael Isard and Blake Andrew. Contour tracking by stochastic propagation of conditional density. InProceedings of European Conference on Computer Vision, pages 343–356, 1996. Demos, code, and moredetatailed info available at http://www.robots.ox.ac.uk/~misard/condensation.html.
[5] Petr Lhotský. Detection and tracking objects using sequential monte carlo method. MSc ThesisK333–24/07, CTU–CMP–2007–01, Department of Cybernetics, Faculty of Electrical Engineering CzechTechnical University, Prague, Czech Republic, January 2007.
[6] Karel Zimmermann, Tomáš Svoboda, and Jiří Matas. Multiview 3D tracking with an incrementallyconstructed 3D model. In Third International Symposium on 3D Data Processing, Visualization andTransmission, Chapel Hill, USA, June 2006. University of North Carolina.
![Page 25: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/25.jpg)
![Page 26: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/26.jpg)
![Page 27: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/27.jpg)
1 2 3 4 5 6 7 8 9 100
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x variable
rela
tive
freq
uenc
y
![Page 28: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/28.jpg)
1 2 3 4 5 6 7 8 9 100
5
10
15
20
25
30
35
xnew
variable
freq
uenc
y
![Page 29: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/29.jpg)
10 20 30 40 50 60 70 80 90 100−4
−3
−2
−1
0
1
2
3
4
time step
stat
e va
lue
state of the systemestimated valuesamples
![Page 30: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/30.jpg)
−4 −3 −2 −1 0 1 2 3 40
0.5
1step 13
measurement functiontrue state
−4 −3 −2 −1 0 1 2 3 40
0.01
0.02
weighted samplesestimated state
−4 −3 −2 −1 0 1 2 3 40
0.5
1step 53
measurement functiontrue state
−4 −3 −2 −1 0 1 2 3 40
0.02
0.04
weighted samplesestimated state
![Page 31: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/31.jpg)
![Page 32: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/32.jpg)
![Page 33: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/33.jpg)
![Page 34: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/34.jpg)
0
5
10
15
20
0
5
10
15
20
0
0.05
0.1
0.15
0.2
a channel of opponent colorsb channel of opponent colors
![Page 35: OPPA European Social Fund Prague & EU: We invest in your future. · 2013. 12. 20. · 10/22 Example: 1–Dtracking,closerlook 10 20 30 40 50 60 70 80 90 100-4-3-2-1 0 1 2 3 4 time](https://reader034.fdocuments.in/reader034/viewer/2022052006/601abb5182611216fb7d03a8/html5/thumbnails/35.jpg)
OPPA European Social FundPrague & EU: We invest in your future.