Augmented Reality Head-up Displays for...
Transcript of Augmented Reality Head-up Displays for...
![Page 1: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/1.jpg)
Augmented Reality Head-up Displays for Cars
Victor Ng-Thow-Hing Principal Scientist,
Honda Research Institute USA Mountain View, CA, USA
![Page 2: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/2.jpg)
Outline
1. Why Augmented Reality for the car? 2. What are the design issues? 3. How to design, prototype and test AR
applications for cars?
![Page 3: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/3.jpg)
Explore enriching the driver’s experience while respecting the primary task of driving.
Human-Machine Interfaces (HMI) Mission Statement:
User-centered design methodologies. Start observing, than design solutions with technology components.
Methods:
![Page 4: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/4.jpg)
Challenges
Technology People
![Page 5: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/5.jpg)
Solutions
Autonomous Cars Enhanced Driver
Mercedes Google
![Page 6: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/6.jpg)
Augmented Reality for Cars From Wikipedia:
Augmented reality (AR) is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics or GPS data.
HI-CAR: Honda Interactive Contextual Augmented Reality
![Page 7: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/7.jpg)
Mobile Devices for AR
Smartphone: Portable, connected
Tablet: Portable, connected, bigger interaction surface
Car: Portable, connected, huge interaction surface + More sensor options + Larger field of view + Direct see-thru view + Transports user
![Page 8: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/8.jpg)
AR seems like a cool thing, but what is the content? And how do we display it?
![Page 9: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/9.jpg)
May 8, 2012, are2012, Santa Clara, CA, USA.
9
Cognitive Dissonance Problem
![Page 10: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/10.jpg)
Create a consistent world view
![Page 11: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/11.jpg)
Dual focus problem
http://webphysics.davidson.edu/physlet_resources/dav_optics/examples/eye_demo.html
![Page 12: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/12.jpg)
Parallax problem
![Page 13: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/13.jpg)
Perception Behavior
Situation Awareness
Visual Stimuli
Defensive Driving
Reflexes
Cognitive Issues
Driver Distraction
Inattentional Blindness
Good AR Bad AR
![Page 14: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/14.jpg)
Design-Prototype Cycle
design
research and development
user evaluation
High fidelity prototypes
Low fidelity prototypes
![Page 15: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/15.jpg)
![Page 16: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/16.jpg)
User-centered Design
User research Brainstorming observations Idea generation
![Page 17: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/17.jpg)
iteration
Life Cycle of a Driving Application
Higher fidelity
![Page 18: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/18.jpg)
Prototyping
Adjustable size powered chair
120 deg driving view
Customizable driving simulator
In-lab HUD Prototype
![Page 19: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/19.jpg)
![Page 20: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/20.jpg)
Driving aids
![Page 21: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/21.jpg)
Use Cases for UI Composer + AR
1. How to integrate with existing legacy systems (including driving simulator)
![Page 22: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/22.jpg)
Integrating UI Composer with Legacy Code
Data streams roadmodel
application HUD display GPS receiver (Bluetooth)
Location Services
Time long/lat 3-D
labels
Cameras perception
Extracted scene information
UI Composer Driving Simulator
OpenGL ES application as texture See:
Runtime/Plugins/Render/gears
![Page 23: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/23.jpg)
Communication with UIComposer
UI Composer NADS
Driving Simulator Gear changes, Speed, RPMs, State changes
Legacy Framework
udp socket connection
Shared Memory Cells
Shared memory
I/O dll
QSharedMemory
![Page 24: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/24.jpg)
No need to restart entire system
UI Composer NADS
Driving Simulator Gear changes, Speed, RPMs, State changes
Legacy Framework
udp socket connection
Shared Memory Cells
Shared memory
I/O dll
QSharedMemory
![Page 25: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/25.jpg)
![Page 26: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/26.jpg)
Feedback for driving
Maya modeling UI Composer animation + scripting
![Page 27: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/27.jpg)
Conclusions Augmented reality for cars has potential to aid and engage drivers. Must design carefully for driver. Design, prototype, test and iterate. UI Composer can help in this process and can be integrated with custom driving assistance systems.
![Page 28: Augmented Reality Head-up Displays for Carson-demand.gputechconf.com/gtc/2013/presentations/S3230-Augmented... · Augmented Reality Head-up Displays for Cars Victor Ng ... driver,](https://reader031.fdocuments.in/reader031/viewer/2022021823/5b4c413e7f8b9a934f8b895e/html5/thumbnails/28.jpg)
Thanks Team members: Karlin Bark, Lee Beckwith, Cuong Tran Contact: Honda Research Institute USA Victor Ng-Thow-Hing [email protected]