Retrieving Actions in Group Contexts Tian Lan, Yang Wang, Greg Mori, Stephen Robinovitch Simon...
-
date post
19-Dec-2015 -
Category
Documents
-
view
215 -
download
0
Transcript of Retrieving Actions in Group Contexts Tian Lan, Yang Wang, Greg Mori, Stephen Robinovitch Simon...
Retrieving Actions in Group Contexts
Tian Lan, Yang Wang, Greg Mori, Stephen Robinovitch Simon Fraser University
Sept. 11, 2010
Outline
• Action Retrieval as Ranking
• Results and Future Work
• Contextual Representation of Actions
Nursing Home
• Fall analysis in nursing home surveillance videos– a system automatically rank the videos according
to the relevance to fall action is expected
Actions in Group Context
• Motivation– human actions are rarely performed in isolation,
the actions of individuals in a group can serve as context for each other.
• Goal– explore the benefit of contextual information in
action retrieval in challenging real-world applications
Action Context Descriptor
Feature Descriptor
Multi-class SVM
action class
scor
e
action class
scor
e
…action class
scor
e
max
action classsc
ore
e.g. HOG by Dalal & Triggs
Outline
• Action Retrieval as Ranking
• Results and Future Work
• Contextual Representation of Actions
Classification or Retrieval
• Previous Work–Most work in human action understanding
focuses on action classification.
Classification or Retrieval • Most surveillance tasks are typical retrieval
tasks– retrieve a small video segment contains a
particular action from thousands of hours of videos.
• The “action of interest” is rare event– Extremely imbalanced classes
Outline
• Action Retrieval as Ranking
• Results and Future Work
• Contextual Representation of Actions
Dataset
• Nursing Home Dataset • 5 action categories: walking, standing, sitting, bending
and falling. (per person)• 18 video clips.• Query: fall
• Collective Activity Dataset (Choi et al. VS. 09)
• 5 action categories: crossing, waiting, queuing, walking, talking. (per person)
• 44 video clips.• Query: each of the five actions
System Overview
Person
DetectorPerson
DescriptorVideo
u
v
RankSVM
• Pedestrian Detection by Felzenszwalb et al.• Background Subtraction
• HOG by Dalal & Triggs • LST by Loy et al. at cvpr 09
Baselines
• Context vs No Context– Action Context Descriptor– Original feature descriptors, e.g. HOG (Dalal & Triggs at CVPR
05), LST (Loy et al. at CVPR 09)
• RankSVM vs SVM
• Methods– Context + RankSVM (our method)– Context + SVM– No Context + RankSVM– No Context + SVM
Conclusion
• A new contextual feature descriptor to represent actions– action context (AC) descriptor
• Formulate our problem as a retrieval task.
Future Work
• Contextual Feature Descriptors– How to only encode useful context?
• Rank-SVM loss, optimize the NDCG score