1. The State of Touch Technologies: Educational Opportunities and Challenges Touch, Motion Gesture National ConferenceOpening Address December 7, 2011 Paul Resta The University of Texas at Austin

2. Touch, motion and gesture (TMG) technologieswill engage more of the sensorium for learning Sight Hearing Touch 3. Convergence of TMG with Other Trends Impacting LearningMultimodal learning Touch, motion,Visualizationgesturetechnologies Learning Analytics Cloud computing 4. TMG and Embodied Cognition 5. Embodied CognitionEmbodied cognitiondoes not isolate brainfrom body andenvironment.Our bodies shape howwe sense and processinformation in ourinteractions with theworld. 6. Touch offers more intuitive way tointeract with digital toolsSourece: http://www.youtube.com/watch?v=pT4EbM7dCMs 7. Multitouch offers new ways tosupport collaborative learning 8. Touch feedback on screensSource: Senseg 9. Gesture-Based Computing Gestural systems offer a path for a moreholistic, human interaction of people withtechnology Gesture-based computing is potentiallytransformative technology, because itchanges not only the physical andmechanical aspects of interaction withcomputers, but also our perception ofcollaborating with computers. 10. Gestural Computing Challenges Need for standards (Different systems mayuse different conventions) gestures are ephemeral, they do not leavebehind any record of their path Gestures also vary in meaning and use byculture 11. Importance of interaction design principles in creating new gestural interfaces Visibility (also called perceived affordances or signifiers) Feedback Consistency (also known as standards) Non-destructive operations (hence the importance ofundo) Discoverability: All operations can be discovered bysystematic exploration of menus. Scalability: The operation should work on all screensizes, small and large. Reliability: Operations should work. Norman (2010) 12. Recognizing thepower of gestures for learningSource: Power Posing Carney et al, 2010 13. Mixed Reality Using TMG to fuse video gamewith real world interactionSource: Marta Ferraz, Doctoral research University of Texas at Austin 14. Moving into a multimodal worldSource: Nokia Research Center Future Technologies 15. A Day Made of GlassSource: A Day Made of Glass Corning 16. Wearable Technology ViconRevue Life Logs Sony razor thin display that bends like paperLED display you can wear as contact lens 17. TMG provides new ways to support Multimodal Learning Multiple means ofrepresentation Multiple means ofaction andexpression Multiple means ofengagement 18. Questions for Multimodal Learning Design Scaffolding needed to prepare students toeffectively use multimedia visualrepresentations and touch, motion andgesture technologies for learning How to minimize cognitive overload forstudents in interacting in a multimodalenvironment 19. Now able to capture and analyze astudents interactions on touch devices 20. Will require new ways to analyze and visualize the new massive data setsSource: Taylor Martin :Analysis of student actions with virtual manipulativeson interactive whiteboard 21. Education Visualization LabUniversity of Texas at Austin Learning Technology Center andTexas Advanced Computing Center 22. Implications of Touch and Gesture Technologies The future trajectory of touch and gesturetechnologies have profound implications for: the creation of new ways of interacting withcomputers and with each other the design of new learning environments thatengage more of the sensorium and are moreauthentic and engaging for students 23. The Future of Touch and GestureTechnologies for Learning To achieve this goal, we need to: break out of disciplinary silos Form new interdisciplinary teams of kinesiologyexperts. learning scientists, neuroscientists,software designers, engineers, computerscientists and artists working together 24. The new TMG technologies offer excitingopportunities to move toward a new vision oflearning built upon embodied cognition.