Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic...
-
Upload
spencer-lester -
Category
Documents
-
view
221 -
download
0
Transcript of Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic...
![Page 1: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/1.jpg)
Sparse Granger Causality Graphs for Human Action Classification
Saehoon Yi and Vladimir Pavlovic
Rutgers, The State University of New Jersey
![Page 2: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/2.jpg)
Outline Objective and challenges Previous work Sparse Granger Causality Graph Model Analysis and result Conclusion
![Page 3: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/3.jpg)
Objective Classify human action time series data
Challenges High dimensional time series data
Dimensionality reduction Difficulties in interpretation
Idiosyncratic patterns of same action Need to find commonality within an action
![Page 4: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/4.jpg)
Previous work Learning dynamics of joints
Each action is modeled as Linear Dynamic System C. Bregler, CVPR 97
Align time series data Dynamic Time Warping Canonical Time Warping
F. Zhou and F. De la Torre, NIPS 2009 Need to tune parameter for each pair of sequence
Isotonic Canonical Correlation Analysis S. Shariat and V. Pavlovic, ICCV 2011
![Page 5: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/5.jpg)
Our approach Robust representation of continuous joint
movements using micro event point processes.
Models salient and sparse temporal relations among skeletal joints movements
![Page 6: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/6.jpg)
Overview
![Page 7: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/7.jpg)
Step 1: Generate micro event point processes
Continuous time series
Joint angles on knees
Detect maximal/minimalextreme points as events
Micro event point processes
![Page 8: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/8.jpg)
Step 2: Estimate Granger Causality GraphGranger causality in time
Given two AR time series X, Y
Granger causality
![Page 9: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/9.jpg)
Granger causality in frequency Given two point processes ,
Estimate power spectrum
Decompose spectrum using Wilson’s algorithm
Granger causality
[A. Nedungadi, G. Rangarajan, N. Jain, and M. Ding ’09]
![Page 10: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/10.jpg)
Granger causality graph representation Estimate Granger causality
for each pair of micro events f frequencies → summarized to 4 bands
Granger causality in 128 freq Causality summarized by 4 bands
![Page 11: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/11.jpg)
Step 3: Learn L1 regularized regression Input : 16M2 Granger causality features
Output : action category label Sparse regression coefficient W for each action
Common causality pattern within each class Positive coefficient Wij
edge i → j have high causality
Negative coefficient Wij
edge i → j have low causality
![Page 12: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/12.jpg)
Experiments HDM05 dataset
Motion capture sequence of 29 skeletal joints Each action is performed by 5 subjects 8 action classes are chosen
Deposit on the floor w/ R hand
Punch front w/ L hand
Jumping Jack Punch front w/ R hand
Kick front w/ R leg Squat
Kick side w/ R leg Walk two steps
![Page 13: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/13.jpg)
Experiment settings Two different cross validation settings
Cut 1 Randomly partition training / testing across all subjects
Cut 2 Test set subjects different from training subjects To show classification accuracy on unseen data
![Page 14: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/14.jpg)
Example of Sparse Granger Causality Graph Model
DEPOSIT FLOOR RIGHT HAND KICK RIGHT SIDE
![Page 15: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/15.jpg)
Example of Sparse Granger Causality Graph Model
PUNCH LEFT FRONT PUNCH RIGHT FRONT
![Page 16: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/16.jpg)
Comparative result
Confusion matrix of SGCGM
![Page 17: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/17.jpg)
Conclusion
Learn common structure within an action The sparse regression model chose which
pairwise relationship is important for the action
Interpretability of the model Granger causal graph describes temporal
relationship between two joints
![Page 18: Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey.](https://reader034.fdocuments.in/reader034/viewer/2022051019/5697c01b1a28abf838ccf685/html5/thumbnails/18.jpg)
Thanks you.
Q & A