Emerging Embedded Applications with Technology Advancement · Emerging Embedded Applications with...
14
Intel China Academic Forum Emerging Embedded Applications with Technology Advancement Jiqiang Song Tsinghua-Intel Joint Center for Advanced Mobile Computing Technology / R&D Director, Application 2010-08-20
Transcript of Emerging Embedded Applications with Technology Advancement · Emerging Embedded Applications with...
Intel China Academic Forum
Emerging Embedded Applications with Technology Advancement
Jiqiang SongTsinghua-Intel Joint Center for Advanced Mobile Computing Technology / R&D Director, Application
2010-08-20
Intel China Academic Forum22
Agenda
• Emerging embedded applications
• Our embedded media computing research
• Summary
Intel China Academic Forum33
Embedded Technology Advancement
• SoC，SiP
• Multi-core
• Multi-IP
• Multi-sensor
• MeeGo
• Android
• iOS
• Multi-touch
• Gesture
• Face
• Voice
• 3G/4G/Wi-Fi
• BT/ZigBee/RFID
• Ad-hoc network
• GPS
Net-work
UI
Hard-ware
OS
Intel China Academic Forum44
Emerging Applications Overview
• Avatar-based video conferencing
– Social network, customer service, etc.
– Video analytics, face technologies, 3D modeling
• Context-aware advertisement
– TV ad, mobile phone ad, IVI ad, etc.
– Video/audio analytics, face technologies, sensor fusion
• Augmented-reality application
– Information, shopping, entertainment
– Video analytics, 3D scene recovery, 3D modeling
• Intelligent surveillance
– House safety, health monitoring, environment monitoring
– Sensor fusion, cross-media analytics
Intel China Academic Forum55
Embedded Media Computing Research
• Smart embedded devices may rely more on media computing
– User/context aware, personalized
– Self control instead of fully controlled by user
– Intelligent UI due to no keyboard/mouse
• Special research challenges
– Less computing power in embedded devices
May need more careful optimization, hardware accelerators and client/cloud collaboration
– Power efficiency needs to be considered
• Performance analysis
– System level analysis needed, not just for CPU
Intel China Academic Forum
Media Application to Algorithm Mapping
Avatar-based
interaction
3D content
generation
3D
navigation
CE Handheld Auto Other EmbeddedMarket
Application
Driver
monitoring
Rear-view
panorama
Personalized
TV
Informative
TV
Avatar
modeling
Avatar
control3D building
modeling
Augmented
reality
3D model
creation
View
synthesis
Module
Metadata
indexing
Home
SurveillanceProg/AD
recommd
Image-based
rendering
Video
categorizeImage anti-
distortion
Face
recognitionBody
trackingCamera
Recovery
Multi-view
stereo
Face
tracking
Cam Pose tracking
Facial
animation
Stereo
matching
Body
animation
Texture
mapping
Algorithm
Video
summary
Preference
mining
Facial
alignment
Cast
indexing
Gender/Age
Classify
Motion
analysis
F/B
Segment
Event
Detect
Region of
interest
Sensorlocation
Surface
Seg/fitting
6
Intel China Academic Forum
Face Feature Tracking at Embedded
Fatigue detection in safe driving
Avatar-based chatting at mobile
Smile shutter in digital camera
Facial recognition at set-top-box
The implementation must be efficient at embedded
Intel China Academic Forum
0
10
20
30
40
50
60
1 41 81 121 161
Tim
e(m
s)
Frames
Proposed Fused Method
0
50
100
150
200
250
1 41 81 121 161
Tim
e(m
s)
Frames
Detect+Track
Face Tracking on Embedded System
Traditional face detection/tracking algorithm has imbalanced load over time which is infeasible for embedded computing
Propose a novel fused face detection/tracking algorithm to break down a large detection task into small ones to resolve the imbalance problem
8
Jitter effect
Intel China Academic Forum
Face Tracking Performance
Tailored optimization on Intel® Atom™ processor Z5xx series platform
- Achieved 700+ fps in preview mode for VGA size video
- Comparable accuracy compared with other existing solutions
- Enable advanced features like very small face detection …
9
0
100
200
300
400
500
600
700
800
Base SIMD/HT Fast
mode
Cascade
model
Step size Task
grouping
New
strategy
Face Tracking Performance
Speed (fp
s)
Intel China Academic Forum10
3D Model Creation for Amateurs
Send images to a desktop/server
Take ~20 images of a object with a digital camera
3D model creation from images
Camera recovery results
Rough model by fast creation
Refined model by dense creation
Textured 3D model
Outputs
Intel China Academic Forum11
Seamless Computation Partitioning
Computation
– Uniform sampling
– Pre-SfM view-selection
– Pre-segmentation
– Camera recovery
– Post-SfM view selection
– Post-segmentation
– Visual hull reconstruction
– Texture mapping
– Rendering
Thin
Client
Rich
Client
High BW
Low BW
Data
• Sampled images
• Pre-selected images
• ROI for the images
• Post-selected images
• ROI for the images
Intel China Academic Forum12
Lion: 16 6M pixel photos recovered Metal, ~1.2m high
Example Results – Lion Sculpture
Smoothed VH result
MVS results from dense point cloud
Dense point cloud
Textured model
Our multi-view stereo paper got acceptedas oral paper at CVPR 2010
Intel China Academic Forum13
Summary
• Intel Labs China is researching media computing which will have great impact on the PC and embedded devices
• Intel Labs China has already delivered some leading-edge media computing technologies for the PC, and are actively working on solving more technical challenges for embedded devices
• This research will help enable a rich new world of experiences for future smart embedded devices
• Welcome more academic collaborations
Intel China Academic Forum1414
Thank You!
