Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko...
-
Upload
melvyn-phillips -
Category
Documents
-
view
219 -
download
1
Transcript of Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko...
![Page 1: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/1.jpg)
Real-Time Classification of Dance Gestures from Skeleton
Animation
Michalis Raptis, UCLA CSDDarko Kirovski, Hugues Hoppe,
Wei-ge Chen, Microsoft ResearchJoel Pritchett, Tom Fuller, Chuck Noble,
XBOX Microsoft Game Studios
![Page 2: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/2.jpg)
Absolute motion Relative motion
![Page 3: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/3.jpg)
1
1 11
1 1
1
1
1
Ora
cles
Oracle vs. Random Dancer
![Page 4: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/4.jpg)
Head
NeckLeft
shoulder
Right shoulder
Right elbow
Left elbow
Spine
Tail
Left hipRight hip
Right knee
Left knee
Left foot
Right foot
(0,0,0)
Left hand
Right hand
TORSO
1st degree joint
2nd degree joint
![Page 5: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/5.jpg)
TORSO
ur t
torso PCA
LS
t
u
r
q
jLEp
R
LHP
LErP q
j
b
R
S
![Page 6: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/6.jpg)
0 20 40 60 80 100 1200
2
4Timing analysis - Elbows
left, y
right, y
0 20 40 60 80 100 1200
0.5
1Timing analysis - Knees
left, y
right, y
0 20 40 60 80 100 1200
2
4Timing analysis - Hands
left, y
right, y
0 20 40 60 80 100 1200
2
4Timing analysis - Feet
left, y
right, y
![Page 7: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/7.jpg)
10 20 30 40 50 60 70 80 90 100 110 120-pi/2
0
pi/2Torso angles
face
ski turn
ski downhill
0 20 40 60 80 100 1201
2
3Timing analysis - Elbows
left, q
right, q
0 20 40 60 80 100 1201
2
3Timing analysis - Knees
left, q
right, q
0 20 40 60 80 100 1200
1
2Timing analysis - Hands
left, q
right, q
0 20 40 60 80 100 1200
1
2Timing analysis - Feet
left, q
right, q
![Page 8: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/8.jpg)
0 50 100 150-0.4
-0.2
0
0.2
0.4
0 50 100 150-0.4
-0.2
0
0.2
0.4
0 50 100 150-0.5
0
0.5
1
0 50 100 1501
1.5
2
2.5
0 50 100 1501
1.5
2
2.5
3
0 50 100 1501
1.5
2
2.5
0 50 100 1501.5
2
2.5
3
0 50 100 1501
1.5
2
2.5
0 50 100 1501
1.5
2
2.5
3
0 50 100 1501
1.5
2
2.5
3
0 50 100 1501
1.5
2
2.5
3
0 50 100 1500
0.5
1
1.5
2
0 50 100 1500
1
2
3
0 50 100 1500
0.5
1
1.5
0 50 100 1500
1
2
3
0 50 100 1500
0.5
1
1.5
0 50 100 1500
1
2
3
0 50 100 1500
0.5
1
1.5
0 50 100 1501
1.5
2
2.5
3
OraclePlayer meanTo
rso
angl
esEl
bow
sW
rists
Knee
sFe
et
Left Right
Oracle vs. Player Mean
![Page 9: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/9.jpg)
![Page 10: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/10.jpg)
-1 0 10
0.01
0.02
-1 0 10
0.01
0.02
-1 0 10
0.02
0.04
-1 0 10
0.02
0.04
-1 0 10
0.02
0.04
-1 0 10
0.02
0.04
-1 0 10
0.02
0.04
-1 0 10
0.02
0.04
-1 0 10
0.02
0.04
-1 0 10
0.02
0.04
-1 0 10
0.02
0.04
-1 0 10
0.01
0.02
-1 0 10
0.01
0.02
-1 0 10
0.01
0.02
-1 0 10
0.01
0.02
-1 0 10
0.02
0.04
-1 0 10
0.01
0.02
-1 0 10
0.02
0.04
-1 0 10
0.01
0.02
True negativeTrue positiveTo
rso
angl
esEl
bow
sW
rists
Knee
sFe
etTowards MLE
![Page 11: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/11.jpg)
![Page 12: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/12.jpg)
Depth sensing
Player performance
Depth image
Skeletal tracking
WireframeFeature
representation
Normalized correlations
Pairwise matching
Matched class
Distance metric
Features
Distance report
What gesture?
How well?
K inect
Gesture model
Prototype mean
Correlation stats
Energy stats
Logistic regression
coefs
Score computationC
lass
ran
kin
g
Ora
cle
Video game asks
Gesture Classification
System
![Page 13: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/13.jpg)
![Page 14: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/14.jpg)
Confusion Matrix
• HMM-based, others 50-60%
![Page 15: Real-Time Classification of Dance Gestures from Skeleton Animation Michalis Raptis, UCLA CSD Darko Kirovski, Hugues Hoppe, Wei-ge Chen, Microsoft Research.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649ccc5503460f94995846/html5/thumbnails/15.jpg)
Summary
• Features – generic, nearly lossless• Dance aligned to beat – no major time-warp• Detection
– Normalized circular cross correlation via FFT– Maximum Likelihood estimator– Logistic regression to resolve class similarity
• Fast, scalable, extraordinary ROC