3D Human Motion Capture - GPU Technology...
Transcript of 3D Human Motion Capture - GPU Technology...
![Page 1: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/1.jpg)
Copyright © 2017 1
3D Human Motion Capture
from 2D Video using
Cloud-Based CNNs
NVIDIA GPU Technology Conference
![Page 2: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/2.jpg)
Copyright © 2017 2
Imagine if Cameras Could See People
Computer Vision
bodySLAM™ turns 2D video into 3D humans
2D video 3D human
![Page 3: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/3.jpg)
Copyright © 2017 3
Applications: AR Social Imagery & Filters
![Page 4: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/4.jpg)
Copyright © 2017 4
AR / VR Gaming
![Page 5: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/5.jpg)
Copyright © 2017 5
Health & Wellness
![Page 6: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/6.jpg)
Copyright © 2017 6
Sport Analytics
![Page 7: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/7.jpg)
Copyright © 2017 7
Eyes for Virtual Assistants
![Page 8: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/8.jpg)
Copyright © 2017 8
APIIn: 2D Video
fidelity/performance parameters
Out: Human FeaturesJoints: 21 joints including eyes and earsSkeletons: 2D, 3DSegmentation: background, individuals, limbs
PlatformsCUDAMetal Performance ShadersopenCL
BodySLAM™: Deep Learning API
![Page 9: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/9.jpg)
Copyright © 2017 9
Deep Learning / Computer Vision Secret Sauce100+ layer CNN (Caffe)Synthetic Training dataHuman tracking
Game Tech3D animationJoint FilteringInverse Kinematics
PlatformsNVIDIA GPUAWSMPS iOS
Tech Stack
![Page 10: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/10.jpg)
Copyright © 2017 10
High Level Architecture
![Page 11: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/11.jpg)
Copyright © 2017 11
Training Pipeline
![Page 12: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/12.jpg)
Copyright © 2017 12
BodyFilter™: VFX filters built on top of BodySLAM™
![Page 13: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/13.jpg)
Copyright © 2017 13
Unity Game Engine for Training Pipeline and VFX App
![Page 14: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/14.jpg)
Copyright © 2017 14
Live Demos
![Page 15: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/15.jpg)
Copyright © 2017 15
Working with CustomersOptimizationNative Mobile SupportHigher Fidelity
Future Work
![Page 16: 3D Human Motion Capture - GPU Technology Conferenceon-demand.gputechconf.com/gtc/2017/presentation/s... · Copyright © 2017 16 Questions? Let’s Start Augmenting Reality Together!](https://reader034.fdocuments.in/reader034/viewer/2022050213/5f5fc65637b64d0b575ba14d/html5/thumbnails/16.jpg)
Copyright © 2017 16
Questions? Let’s Start Augmenting Reality Together!