Paul Varcholik, Joseph J. LaViola Jr., Denise Nicholson Institute for Simulation and Training...
-
Upload
dwight-gerard-douglas -
Category
Documents
-
view
216 -
download
1
Transcript of Paul Varcholik, Joseph J. LaViola Jr., Denise Nicholson Institute for Simulation and Training...
Paul Varcholik, Joseph J. LaViola Jr., Denise NicholsonInstitute for Simulation and TrainingUniversity of Central Florida
[email protected], [email protected], [email protected]
TACTUS: A Hardware and Software Framework for Research in Multi-Touch Interaction
Framework Motivation
Primary multi-touch components1. Physical interaction surface
2. Software system for collecting and interpreting points of contact
Present a barrier to entry for researchers focused on higher-level interface issues or application development
Multi-Touch Origins (pre-2000) Keyboards Touchscreens Pen-based computing 1984 – Bob Boie (Bell Labs) perhaps the first
multi-touch screen 1985 – Bill Buxton, Multi-Touch Tablet 1990 – Sensor Frame, Paul McAvinney (CMU)
Optical Sensor 3 fingers (some trouble with ambiguous finger positions)
1998 – Fingerworks
Multi-Touch Origins (2000 to present) 2001 – Paul Deitz – Mitsubishi DiamondTouch 2003 – Jazz Mutant 2004 – Andy Wilson – TouchLight Oct. 2005 – Jeff Han – FTIR, UIST Paper Feb. 2006 – Jeff Han – TED Talk 2006 – Jeff Han – Perceptive Pixel (CNN Magic Wall) Spring 2007 – Microsoft Surface Summer 2007 – Apple iPhone 2008 – Open-Source Community 2009 – Windows 7
Framework Requirements
Essential Component Description
Multi-Touch Surface Commercial-off-the-shelf hardware, requiring near zero pressure to detect an interaction point
Software Hit-Testing Ability to determine the presence and location of each point of surface contact; supporting at least four concurrent users
Software Point Tracking Identifying a continuous point of contact and reporting its velocity and duration
Framework RequirementsSecondary Component Description
Application Programming Interface (API)
A software system upon which multiple multi-touch applications can be developed
Multi-Platform Support The ability to access multi-touch interaction data from different computing environments (e.g. languages, OS’s, etc.)
Reconfiguration Modifying the software system without recompilation
Software Service Allowing multiple applications to access multi-touch interaction data simultaneously, including over a computer network
Presentation-Layer Independence
Isolating the multi-touch interaction data from the system to graphically present such data, allowing any GUI to be employed when developing multi-touch applications
Mouse Emulation Support for controlling traditional ‘Window, Icon, Menu, Pointing Device’ interaction through a multi-touch surface
Tangible Interfaces The ability to detect and interact with physical devices placed on or near the multi-touch surface
Customizable Gesture System Support for training arbitrary multi-touch gestures and mapping them to software events
Shape Detection e.g. Finger, blob, hand
Multi-Touch Hardware Frustrated Total Internal Reflection (FTIR)
1. Optical waveguide
2. Supporting structure (e.g. cabinet)
3. IR sensing camera
4. Projector and diffuser
5. IR emission source
6. Computer
Software Framework
Image processing Presentation layer Multi-platform communication Pen/writing style interaction (Ink) Gesture recognition
Image Processing Pipeline
Webcam RotateFlipBackground
Filter
Threshold
Scaling
Blob Detection
GrayScaleGrayscale
Image 8bppBinary Image
1bpp
CalibrationPoint
Tracking
ProcessedFrame
Camera Frame24bpp
Mouse EmulatorFunction Gesture DescriptionLeft Click Quick tap on the surface with one finger.
Left Click (alternate) While holding down a finger, tap another finger to the left side of the first.
Drag Perform a Left Click (alternate) but do not release the left side press. Drag both fingers to the destination and release.
Right Click While holding down a finger, tap another finger to the right side of the first.
Double Click Tap two fingers at the same time.
Mouse Wheel Scroll While holding down a finger, drag another finger vertically and to the right side of the first. Dragging up scrolls the mouse wheel up and vice versa.
Alt-Tab While not a mouse command, the Alt-Tab command is a useful Windows feature that switches between applications. To perform an Alt-Tab, hold down a finger and drag another finger horizontally above the first. Dragging to the left moves backward through the list of active applications and dragging to the right moves forward.
Multi-Touch Efforts: Additional Videos Demo Explorer Improved Calibration Multi-Touch Starcraft Mouse Emulator Ink Demonstration Particle System Demonstration SurfaceSimon SurfaceCommand
Conclusions The Framework:
In development since Fall 2007Open-source availability since Spring 2008Now in its fourth major releaseAnswers most of the specified requirementsSuccessfully employed by UCF and colleagues
from Brown UniversityDownloaded extensively by the open-source
communityProvides a robust, extensible platform for
research in multi-touch interaction
Questions?
Paul Varcholik, Joseph J. LaViola Jr., Denise NicholsonInstitute for Simulation and TrainingUniversity of Central Florida
[email protected], [email protected], [email protected]
TACTUS: A Hardware and Software Framework for Research in Multi-Touch Interaction
Resources: Commercial Platforms Microsoft Surface Mitsubishi DiamondTouch Apple iPhone & iPod Touch Perceptive Pixel N-trig MultiTouch OY (Finland) Jazz Mutant HP TouchSmart Dell Latitude XT2 TacTable
Resources: Open-Source Platforms TouchLib (NUI Group) tBeta (NUI Group) reacTIVision Bespoke Multi-Touch Framework Sparsh UI (Iowa State) BBTouch Touche`
Resources: Online
NUI Group TouchKit by NOR_/D Multi-Touch Systems that I Have Known
and Loved – Bill Buxton, Microsoft Research
Jeff Han’s TED Talk Bespoke Software