User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for...
Transcript of User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for...
![Page 1: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/1.jpg)
User-System Interaction:from gesture to action
Prof. dr. Matthias RauterbergIPO - Center for User-System Interaction
TU/e Eindhoven University of Technology
![Page 2: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/2.jpg)
© M. Rauterberg, TU/e
The optimization problem
user systemcosts
technical complexity of the user interface
optimum
![Page 3: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/3.jpg)
© M. Rauterberg, TU/e
What are the research topics of USI?
interactive systemuser
Working domain • Communication / Co-operation• Home / Office Environment• Financial / Medical Sector• Knowledge Management• Product / Process Industry• Transportation / Logistic• Teaching and Learning
Working system
Science• perception• cognition• action
Application
• user-centered design
Engineering• audio interfaces• computer vision based input• speech input / output• tactile input / output
Goal:Pa => Pd
![Page 4: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/4.jpg)
© M. Rauterberg, TU/e
Two Trends in User Interface Technology
Ubiquitous and Mobile computing Ambient rooms and Cooperative buildings
![Page 5: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/5.jpg)
© M. Rauterberg, TU/e
What are the technical challenges?
• New interaction styles– speech input/output– computer vision based input (e.g., gestures)– audio interfaces (e.g., non-speech audio)– tactile and force feedback
• New interface concepts– adaptive and intelligent software– natural user interfaces
![Page 6: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/6.jpg)
© M. Rauterberg, TU/e
Human-human communication
interpersonal relation content
Non-verbal verbal
facial expression
bodymovement
explicit emotion
implicit emotion
gesture action
![Page 7: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/7.jpg)
© M. Rauterberg, TU/e
Non-facial gestures associated with speech act
• Iconic gestures– form of this gesture supports a feature of the described action or
event (e.g., “he climbed up the pipe…”)
• Metaphoric gestures– form of this gesture is representational (e.g., “the meeting went on
and on…”)
• Deictic gestures– form of this gesture specializes or locates something in the
physical space nearby the narrator (e.g., “this document shows…”)
• Beat gestures– small baton-like gestures that serve a pragmatic function (e.g.,
“she talked first, I mean second…”)
![Page 8: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/8.jpg)
What is the research approach?
Design relevantknowledge
Empiricalevaluation
![Page 9: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/9.jpg)
© M. Rauterberg, TU/e
The most important design principle
• Perception Space– The physical space where
the user’s attention is.
• Action Space– The physical space where
the user acts in.
• Design Principle:– perception space and action
space must coincide!
![Page 10: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/10.jpg)
© M. Rauterberg, TU/e
Two design principles for natural user interfaces (NUIs):
No technical equipment inside to body space of the user!
2. design principle
Perception space and action space must coincide!
1. design principle
![Page 11: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/11.jpg)
© M. Rauterberg, TU/e
Architecture of a Natural User Interface (NUI)
Working Area
Communication & Working Area
Paper document
Electronic documents
Tognazzini B: Tog on Software Design. (1996).
Wellner P, Mackay W, Gold R: Computer-Augmented Environments: Back to the Real World. (1993)Fitzmaurice G, Ishii H, Buxton W: Bricks: Laying the Foundations for Graspable User Interfaces. (1995)
![Page 12: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/12.jpg)
© M. Rauterberg, TU/e
BUILD-IT: a tool for integrative design
• design team with experts from different disciplines
• integration of different expertise
• intuitive interaction style
• an example for a natural user interface
![Page 13: User-System Interaction: from gesture to action Prof. dr. Matthias Rauterberg IPO - Center for User-System Interaction TU/e Eindhoven University of Technology.](https://reader035.fdocuments.in/reader035/viewer/2022062721/56649f1b5503460f94c3145a/html5/thumbnails/13.jpg)
© M. Rauterberg, TU/e
How to meet the challenges?
Co-operation inside TUE Co-operation outside TUE
Research in perception, cognition, and action
Research in speech and computer graphics
Research in signal processing and pattern recognition