augmented reality and the human computer interface

•an overview and case study

John Reid Perkins-Buzo

Southern Illinois University

rperkinsbuzo@siu.edu

late 20th centuryUI hardware• control panels (buttons)• switches, slots and levers• lightpens• keyboard + mouse• joystick & controller

3

20th Century Model HCI

Diagram adapted from Otto Traian

strengths and limitations

• Strengths– Low bandwidth– Inexpensive– Movement constrained

• Limitations– Tech-training required motions– Movement constrained– Less integrated for human sensory

response

21st century ui basics

• touch screens• voice recognition• facial expression recognition

strengths and limitations

• Strengths– Less tech-training required– Relatively Inexpensive– Movement constrained

• Limitations– High bandwidth– Movement constrained– Less integrated for human sensory

response

augmented reality• virtual controls

– Any virtual reality control can also be incorporated into AR

– Procedural controls are also possible, and more natural

• real controls– Control surfaces in the real world can be

mapped as controllers in AR– Previous hardware controllers can also be

used

strengths and limitations

• Strengths– Few motion constraints– Integration for human sensory response– Less tech-training required

• Limitations– Relatively expensive– High bandwidth– Few motion constraints

examples• Vuphoria (https://developer.vuforia.com)

– QualComm’s AR Software Developers Kit (SDK) free in beta, now with a licensing fee (but still with a free version).

• Metaio (http://www.metaio.com/sdk/)– Leading AR developers. SDK licensing fee

(but with a free feature-limited version).

• Augment (http://augmentedev.com)– Oriented toward presentation of products.

SDK licensing fee (but with a free educational version).

my research• Moverio BT-200

11

click the image to view the video in a browser window

12

Directions Artificial Intelligence needed to provide

the necessary responses motivated by the reality of the augmented interface.

Lessons to learn from the VR track record. Always on the cusp of becoming big, but never actually making it beyond a recurring fad.

Devices need to be revisited. For instance, how should AR app interact with a smart watch?

13

Questions?

references• Gotow, J. K. Zienkiewicz, J. White and D. Schmidt.

Addressing Challenges in Delivering Augmented Reality Applications to Smartphones. Proceedings of the Third International ICST Conference on MOBILe Wireless MiddleWARE, Operating Systems, and Applications. Mobilware 2010, Chicago, IL, USA.

• Grudin, Jonathan. History in a Time of Rapid Obsolence. Human Computer Interaction Handbook ed. Julie A. Jacko. (CRC Press, May 4, 2012). xxvii-lxi.

• Papagiannakis, G, G. Singh, and N. Magnenat-Thalman. A Survey of Mobile and Wireless Technologies for Augmented Reality Systems. Computer Animation Virtual Worlds 2008. 19(1): 3-22.

• Wagner, D. and D. Schmalstieg. Making Augmented Reality Practical on Mobile Phones, part 2. IEEE Computer Graphics Applications 2009. 29(4): 6-9.