Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch...
Transcript of Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch...
![Page 1: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/1.jpg)
www.evl.uic.edu
Current Research in User Interaction
Arthur Nishimoto PhD Student
User Interaction, Visualization, Virtual Reality
![Page 2: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/2.jpg)
www.evl.uic.edu
Overview
• SAGE2: A Multi-User Touch Centric Interface
• Unity3D in CAVE2: First Steps in a More Interactive Virtual World
![Page 3: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/3.jpg)
www.evl.uic.edu
Scalable Amplified Group Environment
• Sharing of content across scalable resolution shared displays (SRSD)
• Multiple modes of interaction
– Laptop pointer, touch wall, mobile touch, web
![Page 4: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/4.jpg)
www.evl.uic.edu
Content Browser
• Radial Menu
• Thumbnail Browser
![Page 5: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/5.jpg)
www.evl.uic.edu
Integrating Touch in SRSDs
• Direct mouse to touch
– Limited to existing mouse interactions: click and drag
– Touch not as precise as mouse pointer
– Physical occlusion of screen by fingers, hand, etc.
![Page 6: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/6.jpg)
www.evl.uic.edu
Integrating Touch in SRSDs
• Gestures for linear menus
Top: Bailly, Gilles, Eric Lecolinet, and Yves Guiard. “Finger-Count & Radial-Stroke Shortcuts: 2 Techniques for Augmenting Linear Menus on Multi-Touch Surfaces.” 2010. Bottom: Go, Kentaro, and Hiroki Kasuga. “Multi-Tapping Shortcut: A Technique for Augmenting Linear Menus on Multi-Touch Surface.” 2012.
![Page 7: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/7.jpg)
www.evl.uic.edu
Integrating Touch in SRSDs
• SAGE
– Large icons for touch / multiple pointer
– Fixed to a side bar
– Classic single user desktop design
![Page 8: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/8.jpg)
www.evl.uic.edu
Radial Menu Design
• Buttons can be placed within user’s viewable space
• Within close distance of fingers
– Minimize arm fatigue
– Quick access
![Page 9: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/9.jpg)
www.evl.uic.edu
Thumbnail Window Design
• Handedness of the menu to keep content in user’s central field of view
• Size of the thumbnail window
– Physical movement along the wall vs. virtual (scrolling windows)
![Page 10: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/10.jpg)
www.evl.uic.edu
Unity3D in CAVE2: First Steps in a More Interactive Virtual World
![Page 11: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/11.jpg)
www.evl.uic.edu
Unity Game Engine
• Early (Pre-CAVE2) projects
![Page 12: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/12.jpg)
www.evl.uic.edu
The Battle for Hearts & Minds: Interrogation and Torture in the Age of War
• Virtual reality art performance
![Page 13: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/13.jpg)
www.evl.uic.edu
The Battle for Hearts & Minds: Interrogation and Torture in the Age of War
• Veteran testimonies of US interrogations and abuses of prisoners in Iraq during the American counter-insurgency campaign during the early 2000s
• Collaborative work between:
– Scott Rettberg - University of Bergen, Norway
– Roderick Coover - Temple University, Philadelphia
– Daria Tsoupikova, Arthur Nishimoto - University of Illinois at Chicago
• Based on interviews and dissertation by
– Political Scientist Dr. John Tsukayama
![Page 14: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/14.jpg)
www.evl.uic.edu
The Battle for Hearts & Minds: Interrogation and Torture in the Age of War
![Page 15: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/15.jpg)
www.evl.uic.edu
HASTAC 2015 • Humanities, Arts, Science, and Technology Alliance
and Collaboratory
• Michigan State University, East Lansing
• May 27-30, 2015
• Hearts & Minds Performance and Panel Discussion
• Opening Reception: Art show in planetarium
![Page 16: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/16.jpg)
www.evl.uic.edu
HASTAC 2015
![Page 17: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/17.jpg)
www.evl.uic.edu
Hearts and Minds Performance
![Page 18: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/18.jpg)
www.evl.uic.edu
Hearts and Minds Performance
![Page 19: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/19.jpg)
www.evl.uic.edu
Hearts and Minds Performance
![Page 20: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/20.jpg)
www.evl.uic.edu
Virtual Campus
• Virtual reality rendering of the University of Chicago campus
• Scott Stocking
– Spatial Data Infrastructure Manager, U of C
![Page 21: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/21.jpg)
www.evl.uic.edu
Virtual Campus
• Large amounts of architectural data
– Cubic buildings
– Google Sketchup models
– Building Information Model (BIM)
• Pushed Unity 4 to the 32-bit memory limit
• 64-bit Unity 5 installed in CAVE2 last week
![Page 22: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/22.jpg)
www.evl.uic.edu
getReal3D for Unity
• Unity plugin for managing cluster synchronization, viewer-centered perspective and user interaction for virtual reality environments
![Page 23: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/23.jpg)
www.evl.uic.edu
Omicron-Unity Module
• Collection of additional tools to aid in the development of Unity applications for deployment in CAVE2
• https://github.com/arthurnishimoto/omicron-unity/
![Page 24: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/24.jpg)
www.evl.uic.edu
Omicron-Unity Early Tests
• Spring 2012 Game Class Project in CAVE2 Simulator
![Page 25: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/25.jpg)
www.evl.uic.edu
Omicron-Unity Early Tests
![Page 26: Current Research in User Interaction · Integrating Touch in SRSDs •Direct mouse to touch –Limited to existing mouse ... Arts, Science, and Technology Alliance and Collaboratory](https://reader035.fdocuments.in/reader035/viewer/2022071104/5fdda9df9123ba454278f914/html5/thumbnails/26.jpg)
www.evl.uic.edu
Omicron-Unity Early Tests