Hanning Zhou (hanzhou@amazon) and Don Kimber (kimber@fxpal )
1
Experimental Results Trained on one week’s video recorde d in FXPAL Tested on another week’s video usual events: pass by, pick up print outs unusual events: distribute mails, open mul tiple drawers to look for stationery, open the cabinet, multiple people Quantitative experiments on Terrasc ope data 48 segments in 4 scenarios usual events: group meeting, natural video seq uence unusal events: Hanning Zhou ([email protected]) and Don Kimber ([email protected] m ) Unusual Event Detection via Multi-camera Video Mining Previous Work General Event Detection specific event with well-defined model supervised statistical learning (DTW, HMM, factor graph) Unusual Event Detection unsupervised [Zhong & Shi ‘04] semi-supervised [Zhang et al. ‘05] All above are from single stream Key Idea Collaborative mining of multiple streams Sensor network is prevailing Events from different sensors are related Two-Stage Training Unusual events are rare Manual labeling is intractable Introduction Goal: detecting unusual events from a large amount of multiple stream video. Challenges: multi-stream video lack of labeled data Approach: semi-supervised learning Applications Online Detect unusual events alarm in surveillance system Offline Detect highlights from sport videos Analyze business process Other data streams audio, text and multimodal data streams Step 1: Temporal segmentation ………………………… …… ……… Segment k Segment k + 1 static scene Step 2: Feature extraction Feature: size and location of the motion blobs clustered into GMMs Advantage: higher spatial resolution than motion hist ogram [Zhang ‘05] [Zhong ‘04] New problem: spatial alignment Solved with approximate KL-divergence [Goldberger ‘03] Step 4: Detecting usual event Evaluating the likelihood with forward-backward algorithm [Brand ‘97] Step 3: Training a Model for us ual event 1st Stage: Bootstrap Clustering: keep the large clusters User feedback: exam the small cluster s 2nd Stage: Train CHMM for usual even ts Inference in CHMM is efficient O(T(CN)^2) vs O(TN^(2C)) statistic model to handle variant duration s, noisy observation and asynchrony The hidden state depends on 3D location of the objects The observations are 2D projection of the objects CHMM as a loose stereo Dependent chains The 3D location inferred from different cameras are RELATED Examples of unusual events Examples of usual events ROC curve of HMM vs CHMM on Terrascope data Exact stereo does not work well, because: gaps between the views wide baseline, few correspondences Mailroom camera setup Terrascope dataset camera setup Courtesy of Christopher Jaynes ………………… ……
description
Unusual Event Detection via Multi-camera Video Mining. Step 1: Temporal segmentation. Introduction Goal : detecting unusual events from a large amount of multiple stream video. Challenges : multi-stream video lack of labeled data Approach : semi-supervised learning. - PowerPoint PPT Presentation
Transcript of Hanning Zhou (hanzhou@amazon) and Don Kimber (kimber@fxpal )
Experimental Results Trained on one week’s video recorded in FXPAL Tested on another week’s video usual events: pass by, pick up print outs unusual events: distribute mails, open multiple drawers
to look for stationery, open the cabinet, multiple people
Quantitative experiments on Terrascope data 48 segments in 4 scenarios usual events:
group meeting, natural video sequence unusal events:
group exit, intruder, theft, suspicious behavior
Hanning Zhou ([email protected]) and Don Kimber ([email protected] )
Unusual Event Detection via Multi-camera Video Mining
Previous Work General Event Detection specific event with well-defined model supervised statistical learning (DTW, HMM, factor graph) Unusual Event Detection unsupervised [Zhong & Shi ‘04] semi-supervised [Zhang et al. ‘05] All above are from single stream
Key Idea Collaborative mining of multiple streams Sensor network is prevailing Events from different sensors are related Two-Stage Training Unusual events are rare Manual labeling is intractable
Introduction Goal: detecting unusual events from a large
amount of multiple stream video. Challenges: multi-stream video
lack of labeled data Approach: semi-supervised learning
Applications Online Detect unusual events alarm in surveillance system Offline Detect highlights from sport videos Analyze business process Other data streams audio, text and multimodal data streams
Step 1: Temporal segmentation
…………………………
……
………
Segment k Segment k + 1
static scene
Step 2: Feature extraction Feature: size and location of the motion blobs clustered into GMMs Advantage: higher spatial resolution than motion histogram
[Zhang ‘05] [Zhong ‘04] New problem: spatial alignment Solved with approximate KL-divergence
[Goldberger ‘03]
Step 4: Detecting usual event Evaluating the likelihood with forward-backward
algorithm [Brand ‘97]
Step 3: Training a Model for usual event 1st Stage: Bootstrap
Clustering: keep the large clusters User feedback: exam the small clusters
2nd Stage: Train CHMM for usual events
Inference in CHMM is efficient O(T(CN)^2) vs O(TN^(2C))
statistic model to handle variant durations, noisy observation and asynchrony
The hidden state depends on 3D location of the objects The observations are 2D projection of the objects CHMM as a loose stereo Dependent chains
The 3D location inferred from different cameras are RELATED
Examples of unusual events
Examples of usual events
ROC curve of HMM vs CHMM on Terrascope data
Exact stereo does not work well, because:
gaps between the views wide baseline, few correspondences
Mailroom camera setupTerrascope dataset camera setup
Courtesy of Christopher Jaynes
…………………
……
