Object Tracking
description
Transcript of Object Tracking
![Page 1: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/1.jpg)
1
Object Tracking
Babol university of technology
ECE Dep.
Machine Vision
Prof: M. Ezoji
Presentation: Alireza Asvadi
Winter 2012
![Page 2: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/2.jpg)
2
What is tracking?
Estimating the trajectory of an object over time by locating its position in every frame.
“Tracking by detection” Vs “Tracking with dynamics”
Tracking by Detection: we have a strong model of the object, we detect the object independently in each frame and can record its position over time.
Tracking with dynamics: we use object position estimated by measurement but also incorporate the position predicted by dynamics.
![Page 3: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/3.jpg)
3
Time t Time t+1 Time t+n
1. detect the object independently in each frame
2. Link up the instances and we have a track
Tracking by detection:
Ref: Master Thesis, Alireza Asvadi, "Object Tracking from Video Sequence (using color and texture information and RBF Networks)"
![Page 4: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/4.jpg)
4
Methods for detection of object :
Point detectors Background ModelingBackground Subtraction
Supervised ClassifiersContour evaluation….
Other Methods:
Template and density Based appearance models
Shape models
Ref: Master Thesis, Alireza Asvadi, "Object Tracking from Video Sequence (using color and texture information and RBF Networks)"
![Page 5: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/5.jpg)
5
Point detectors:
Point detectors are used to find interest points in images which have an expressivetexture in their respective localities. The object is represented by points.commonly used interest point detectors include Moravec, Harris and SIFT detector.
Start
Take new image
Select Good Features
Track FeaturesIf lost
featuresReplace features
If images remaining
Store ImageReplace features
Stop
Yes
No
Yes
No
KLT control graph:
Ref: Ankit Gupta (1999183) Vikas Nair (1999219) Supervisor Prof M. Balakrishnan Electrical Engineering Department IIT Delhi
![Page 6: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/6.jpg)
6Ref: A. Yilmaz, O. Javed, and M. Shah, “Object tracking: A survey,” ACM Computing Surveys, Vol. 38, No. 4, pp. 1–45, December 2006.
Ref: M. Sonka,V. Hlavac, R.Boyle, “Image Processing, Analysis, and Machine Vision,” 3rd Edition, 2008.
![Page 7: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/7.jpg)
7
x,y
Template Image
Input Image
I(x,y) O(x,y)
Output Image
x,yCorrelation
Template Matching:
Template matching is a brute force method of searching the image for a region similar to the object template.The position of the template in the current image is computed by a similarity measure, for example:
NCC=normalized cross-correlation SSD=Sum of square differences SAD=Sum of absolute differences
Ref: Ankit Gupta (1999183) Vikas Nair (1999219) Supervisor Prof M. Balakrishnan Electrical Engineering Department IIT Delhi
![Page 8: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/8.jpg)
8
1
0
1
0
22
1
0)(
N
i
N
iii
i
N
i i
yyxx
yyxxcorx is the template gray level image
x is the average grey level in the templatey is the source image sectiony is the average grey level in the source imageN is the number of pixels in the section image (N= template image size = columns * rows)The value cor is between –1 and +1, with larger values representing a stronger relationship between the two images.
NCC:
Template Matching:
In Matlab: C = normxcorr2(template, A)
Affine Transformations could be use to Confirm a Match.
Ref: M. Sonka,V. Hlavac, R.Boyle, “Image Processing, Analysis, and Machine Vision,” 3rd Edition, 2008.
![Page 9: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/9.jpg)
9
NCC
SAD=sum(abs(TEMP(:)- IMGBLK(:)))SSD=sum((TEMP(:)-IMGBLK(:)).^2)
Frame
Template
![Page 10: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/10.jpg)
10
Shape Matching:
Shape matching can be performed similar to tracking based on template matching.
shape models are usually in the form of edge maps.
Ref: A. Yilmaz, O. Javed, and M. Shah, “Object tracking: A survey,” ACM Computing Surveys, Vol. 38, No. 4, pp. 1–45, December 2006.
![Page 11: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/11.jpg)
11
Background subtraction is often a good enough detector in applications where the background is known and all trackable objects look different from the background. The most important limitation of background subtraction is the requirement of stationary cameras.
Background Subtraction:
Ref: M. Sonka,V. Hlavac, R.Boyle, “Image Processing, Analysis, and Machine Vision,” 3rd Edition, 2008.
![Page 12: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/12.jpg)
12
Supervised Classifiers:
During testing, the classifier gives a score to the test data indicating the degree of membership of the test data to the positive class.
Maximum classification score over image regions estimate the position of the object.
Object detection can be performed by learning different object views automatically from a set of examples by means of a supervised learning mechanism.
Ref: A. Yilmaz, O. Javed, and M. Shah, “Object tracking: A survey,” ACM Computing Surveys, Vol. 38, No. 4, pp. 1–45, December 2006.
![Page 13: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/13.jpg)
13
See Details inMean Shift Slides
![Page 14: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/14.jpg)
14
Mean Shift algorithm:
![Page 15: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/15.jpg)
15
Mean Shift Widely used, various enhancements (e.g. Robert Collins):
![Page 16: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/16.jpg)
16
![Page 17: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/17.jpg)
17
Problems with Tracking by detection Methods:
Occlusions:
Time
Similar Objects:
![Page 18: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/18.jpg)
18
NCC
NCC map in First Frame
Problems with Tracking by detection Methods:
Or if the detector fails to detect object
(A template matching Example)
![Page 19: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/19.jpg)
19
Tracking with dynamics:
Key idea: Given a model of expected motion, predict where objects will occur in next frame.
Observation (Detected object) + Dynamics
- Prediction vs. correction
- If the observation model is too strong, tracking is reduced to repeated detection
- If the dynamics model is too strong, will end up ignoring the data
![Page 20: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/20.jpg)
20
Filtering Problem:
kzkx
kxa bc
Estimate of c based on
Prediction a and measurement b
In kalman Filter they are correspond with:
![Page 21: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/21.jpg)
Kalman filter assumptions:
The Kalman filter model assumes that the state of a system at a time t evolved from the prior state at time t-1 according to:
Measurements of the system can be performed, according to the model:
AA
To have similarnotation
R. Faragher , “Understanding the Basis of the Kalman Filter Via a Simple and Intuitive Derivation,” IEEE Signal Processing Magazine, September 2012.
![Page 22: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/22.jpg)
The Kalman filter:
Control information:(force)
state vector:(position & velocity)
The relationship between the forceDuring the time period Δt and position and velocity: In matrix form:
A
R. Faragher , “Understanding the Basis of the Kalman Filter Via a Simple and Intuitive Derivation,” IEEE Signal Processing Magazine, September 2012.
![Page 23: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/23.jpg)
23
The Kalman filter:
The information from the predictions and measurements are combined to provide the best possible estimate of the location of the train.
THE PRODUCT OF TWO GAUSSIAN FUNCTIONS IS ANOTHER GAUSSIAN FUNCTION
R. Faragher , “Understanding the Basis of the Kalman Filter Via a Simple and Intuitive Derivation,” IEEE Signal Processing Magazine, September 2012.
![Page 24: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/24.jpg)
24
The Kalman filter:
R. Faragher , “Understanding the Basis of the Kalman Filter Via a Simple and Intuitive Derivation,” IEEE Signal Processing Magazine, September 2012.
![Page 25: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/25.jpg)
25
The Kalman filter:
R. Faragher , “Understanding the Basis of the Kalman Filter Via a Simple and Intuitive Derivation,” IEEE Signal Processing Magazine, September 2012.
![Page 26: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/26.jpg)
26
Estimate(a posteriori)
Best Prediction prior to Zk(a priori)
Optimal Weighting(Kalman Gain)
Residual
Read details in References process noise
covariance
G. Welch, G. Bishop , “An Introduction to the Kalman Filter,” ,1995.
![Page 27: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/27.jpg)
27
kkkkkk zHxHKkk
1|0
10 ˆlimlim
RR
1||00 ˆˆlim0lim kkkkk xxKkk PP
1 1ˆ ˆ ˆk | k k | k k k k k | kx x K z H x
Small prediction error:
the dynamics model is too strong, will end up ignoring the data
Small measurement error:
The observation model is too strong, tracking is reduced to repeated detection
The Kalman filter:
![Page 28: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/28.jpg)
28
The Kalman filter:
kz
kx
G. Welch, G. Bishop , “An Introduction to the Kalman Filter,” ,1995.
![Page 29: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/29.jpg)
29
The Kalman filter:
![Page 30: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/30.jpg)
30
Problems: Yet it is not good enough
Kalman filter Estimate Is combination of Prediction and Measurement.
Result by applying The kalman filter
Till now we considered the entire measurements But what if we omit uninformative observations or highly unlikely measurements? It is Data Association.
NCC
NCC+Kalman
![Page 31: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/31.jpg)
31
So far, we’ve assumed the entire measurement to be relevant to determining the state.
In reality, there may be uninformative measurements or measurements that are not necessarily resulted from the target of interest or may belong to different tracked objects.
Data association:
Data association: task of determining which measurements go with which tracks.
Simple strategy: only pay attention to the measurement that is “closest” to the prediction.
![Page 32: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/32.jpg)
32
![Page 33: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/33.jpg)
33
![Page 34: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/34.jpg)
34
![Page 35: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/35.jpg)
35
![Page 36: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/36.jpg)
36
![Page 37: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/37.jpg)
37
Tracking: Detection(observation)+ dynamics +Data association
Notice Predicted Values By Kalman Filter (Green)Applying Data association (Gating)
DA(Gating):Omit MeasurementsOutside the gate (say a circle with radius 50)
NCC
NCC+Kalman
NCC+Kalman+DA
![Page 38: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/38.jpg)
38
Tracking by detectiondetect the object independently in each frame
tracking=Detection Detection Methods:
Tracking with dynamicsincorporate object dynamics to tracking Methods:tracking=Detection(observation)+dynamics
Applying Data associationEliminate highly unlikely measurementstracking=Detection(observation)+ dynamics +Data association
Summary:
Point detectorsTemplate matchingdensity Based appearance modelsShape modelsBackground Subtraction…
Filtering MethodsKalman filter…
Methods:Tracking MatchingGating…
![Page 39: Object Tracking](https://reader035.fdocuments.in/reader035/viewer/2022081512/56814514550346895db1d752/html5/thumbnails/39.jpg)
39
Reference:
D. A. Forsyth, J. Ponce, “Computer Vision: A Modern Approach,” Prentice Hall,2nd Edition, 2012.
M. Sonka,V. Hlavac, R.Boyle, “Image Processing, Analysis, and Machine Vision,” 3rd Edition, 2008.
A. Yilmaz, O. Javed, and M. Shah, “Object tracking: A survey,” ACM Computing Surveys, Vol. 38, No. 4, pp. 1–45, December 2006.
R. Faragher , “Understanding the Basis of the Kalman Filter Via a Simple and Intuitive Derivation,” IEEE Signal Processing Magazine, September 2012.
G. Welch, G. Bishop , “An Introduction to the Kalman Filter,” ,1995.