Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia...
-
Upload
marybeth-hardy -
Category
Documents
-
view
215 -
download
1
Transcript of Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia...
![Page 1: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/1.jpg)
Object detection, tracking and event recognition: the ETISEO experience
Andrea Cavallaro
Multimedia and Vision Lab
Queen Mary, University of London
![Page 2: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/2.jpg)
Outline
• QMUL’s object tracking and event recognition• Change detection and object tracking• Event recognition
• ETISEO• Evaluation: protocol, data, ground truth• Impact• Improvements of future evaluation campaigns
• Conclusions
• … and an advert
![Page 3: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/3.jpg)
Outline
• QMUL’s object tracking and event recognition• Change detection and object tracking• Event recognition
• ETISEO• Evaluation: protocol, data, ground truth• Impact• Improvements of future evaluation campaigns
• Conclusions
• … and an advert
![Page 4: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/4.jpg)
Prior system for event detection
http://www.elec.qmul.ac.uk/staffinfo/andrea/CREDS-help.html
RATP/CREDS
![Page 5: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/5.jpg)
Introduction
• QMUL Detection, Tracking, Event Recognition (Q-DTE)• initially designed for Event Detection and Tracking in metro
stations• modified to respond to ETISEO • components:
• Moving object detection
• Background subtraction with noise modeling
• Object tracking
• Graph matching
• Composite target distance based on multiple object features
• Event recognition
M. Taj, E. Maggio, A. Cavallaro“Multi-feature graph-based object tracking”Proc. of CLEAR Workshop - LNCS 4122, 2006
![Page 6: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/6.jpg)
Object detection and tracking
• Change detection• Statistical change detection
• Gaussians on colour components
• Noise filtering• Contrast enhancement
• Problem: data association after object detection• Appearance/disappearance of objects• False detections due to clutter and noisy observations
![Page 7: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/7.jpg)
Moving object segmentation
• Motion detection through frame difference
• Problem• D = {d k}, dk 0 even if there is no structural change in k
current frame reference frame difference frame D
![Page 8: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/8.jpg)
02 / HtP k
0Hdp k
2
2
2 2exp
2
1
kd
Adaptive threshold for change detection
• Noise modelling
• Test statistics
• Significance test
Hyp. H0: “no changes in k”, camera noise N(0, )
kw
kk d 22
02 Hp k ondistributi2
![Page 9: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/9.jpg)
Tracking
• Graph matching using weighted features• Data association verified throughout several frames
to validate the correctness of the tracks • Support track recovery in occlusion scenarios • Features
• centre of mass
• velocity
• bounding box
• colour
).1(),(.),(.),(.),(.),(
H]h,w, , y,[x, = X
4321
..
ijXXgXXgXXgXXgXXg
yx
bj
ai
bj
ai
bj
ai
bj
ai
bj
ai
velocity
appearance
size
position
![Page 10: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/10.jpg)
Graph matching: full graph
v(x11)
v(x21)
v(x31)
v(x41)
v(x13)
v(x23)
v(x33)
v(x43)
v(x12)
v(x22)
v(x32)
V1 V3
v(x11)
v(x21)
v(x31)
v(x41)
v(x13)
v(x23)
v(x33)
v(x43)
V2
v(x12)
v(x22)
v(x32)
![Page 11: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/11.jpg)
v(x11)
v(x21)
v(x31)
v(x41)
v(x13)
v(x23)
v(x33)
v(x43)
v(x12)
v(x22)
v(x32)
Graph matching: max path cover
V1 V3V2
![Page 12: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/12.jpg)
Experimental framework
• Key parameters• noise variance: 1.8• kernel size: 3x3 • feature weights
• position α = 0.40
• velocity β = 0.30
• appearance γ = 0.15
• size δ = 0.15
• Determined using CLEAR dataset/metrics• Moving object detection accuracy / precision (MODA / MODP)• Moving object tracking accuracy / precision (MOTA / MOTP)
![Page 13: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/13.jpg)
Event recognition
![Page 14: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/14.jpg)
Event recognition
![Page 15: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/15.jpg)
Event recognition
![Page 16: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/16.jpg)
Event recognition
![Page 17: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/17.jpg)
![Page 18: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/18.jpg)
Outline
• QMUL’s object tracking and event recognition• Change detection and object tracking• Event recognition
• ETISEO• Evaluation: protocol, data, ground truth• Impact• Improvements of future evaluation campaigns
• Conclusions
• … and an advert
![Page 19: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/19.jpg)
ETISEO
• Impact• Promote evaluation
• Formal and objective evaluation is (urgently) needed
• Data collection and distribution• time consuming!
• common ground for research
• Priority sequences • Use of an existing XML schema • Discussion forum
• Choice of performance measures and experimental data is not obvious
![Page 20: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/20.jpg)
Improvements
• Involve stakeholders at earlier stages• More input from end users
• what do they want / need?
• costs / weights of errors
• Involve (more) researchers from the beginning • Facilitate understanding of the protocol
• Fix errors / ambiguities early
• Use training/testing dataset• see i-Lids and CLEAR
• Maybe private dataset too
• Give meaning to measures • what is the “value” of these numbers?
• e.g., compare with a naïve result
• what is the “value” of a difference of (e.g.) 0.1?
![Page 21: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/21.jpg)
• Improvements of future evaluation campaigns• Are we evaluating too many things simultaneously?
• Too many variables
• Do we need so many measures?• remove redundant measures
• Is the ground truth really “truth”?• statistical analysis / more annotators / confidence level
• Should we distribute the evaluation tool / ground-truth earlier? • Are we happy with the current demarcation of regions / definition of
events?• Do we want to evaluate all the event types together?
• should we focus on subsets of events and move on progressively
• Is the dataset too heterogeneous? • Can we generalize the results obtained so far?
Questions
![Page 22: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/22.jpg)
Conclusions
• Conclusions• QMUL submission
• Statistical colour change detection
• Multi-feature weighted graph matching
• Event recognition module: evolution from CREDS 2005.
• Next: extend to 3D
• Feedback on ETISEO• Evaluation + discussion
• Extend the community / do not duplicate efforts
• Metrics
More information
http://www.elec.qmul.ac.uk/staffinfo/andrea
… and an advert
![Page 23: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/23.jpg)
IEEE International Conference on
Advanced Video and Signal based Surveillance
IEEE AVSS 2007London (UK)
5-7 September 2007
Paper submission: 28 February 2007
![Page 24: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/24.jpg)
• Acknowledgments• Murtaza Taj• Emilio Maggio
![Page 25: Object detection, tracking and event recognition: the ETISEO experience Andrea Cavallaro Multimedia and Vision Lab Queen Mary, University of London andrea.cavallaro@elec.qmul.ac.uk.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649e165503460f94b00865/html5/thumbnails/25.jpg)
Evaluation metric
http://www.elec.qmul.ac.uk/staffinfo/andrea/CREDS-help.html
S
DAt
B
Maximum Delay
Maximum Score
Accepted anticipation
Unaccepted anticipation