ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures
description
Transcript of ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures
![Page 1: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/1.jpg)
ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures
Dustin FreemanUniversity of Toronto Hrvoje Benko Meredith Ringel MorrisMicrosoft Research Daniel WigdorMicrosoft Surface
![Page 2: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/2.jpg)
What The Real World Is Like
![Page 3: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/3.jpg)
Multi-touch In The Real World
![Page 4: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/4.jpg)
Intuitive Gestures?
“Paste”
Wobbrock J. O., Morris M. R., and Wilson A. D. User-defined gestures for surface computing. CHI 2009.
![Page 5: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/5.jpg)
Lack of Agreement
![Page 6: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/6.jpg)
Single-Point Stroke Gestures
Kurtenbach G. and Buxton W. Issues in combining marking and direct manipulation techniques. UIST 1991.
Bau O. and Mackay W.E. OctoPocus: a dynamic guide for learning gesture-based command sets. UIST 2008.
![Page 7: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/7.jpg)
Single-point dynamic guides
![Page 8: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/8.jpg)
Single-point dynamic guides
![Page 9: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/9.jpg)
Multi-point dynamic guides
![Page 10: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/10.jpg)
Multi-point dynamic guides
![Page 11: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/11.jpg)
Idea: The User Shadow
![Page 12: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/12.jpg)
Idea: The User Shadow
![Page 13: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/13.jpg)
Gesture As State Machine
Current single-point stroke methods
![Page 14: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/14.jpg)
ShadowGuides Overview
User Shadow AnnotationsRegistration Pose Guide
![Page 15: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/15.jpg)
Registration Pose
![Page 16: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/16.jpg)
Registration Pose Visualization
![Page 17: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/17.jpg)
Expert Style
![Page 18: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/18.jpg)
Registration Pose Guide
![Page 19: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/19.jpg)
The “Cat” Gesture
User Shadow AnnotationsRegistration Pose Guide
?
![Page 20: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/20.jpg)
Cat Gesture Video
![Page 21: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/21.jpg)
Cat Gesture Video
![Page 22: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/22.jpg)
Learning The Cat Gesture
![Page 23: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/23.jpg)
Learning The Cat Gesture
![Page 24: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/24.jpg)
User Shadow Annotations
![Page 25: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/25.jpg)
User Shadow Annotations
![Page 26: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/26.jpg)
User Shadow Annotations
![Page 27: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/27.jpg)
User Shadow Annotations
![Page 28: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/28.jpg)
User Shadow Annotations
![Page 29: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/29.jpg)
User Shadow Annotations
![Page 30: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/30.jpg)
Annotations for Contact Shape
![Page 31: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/31.jpg)
Evaluation• Standardized multi-touch gesture
learning evaluation method.
![Page 32: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/32.jpg)
Evaluation• Standardized multi-touch gesture
learning evaluation method. Right?
![Page 33: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/33.jpg)
Taxonomy
![Page 34: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/34.jpg)
Representative Gesture Set
![Page 35: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/35.jpg)
Baseline: Video
![Page 36: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/36.jpg)
Baseline: Video
![Page 37: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/37.jpg)
Evaluation• Between-subjects comparison
• 11 participants Video• 11 participants ShadowGuides
• 8 blocks of 15 gestures each• Learning Phase: 7 blocks with help• Memory Phase: 1 block with no help
![Page 38: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/38.jpg)
Memory Phase Results
![Page 39: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/39.jpg)
Observations and Next Steps
• Video shows the gist of the gesture
• ShadowGuides shows details
• Hybrid approach?
![Page 40: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures](https://reader036.fdocuments.in/reader036/viewer/2022062501/56816554550346895dd7d044/html5/thumbnails/40.jpg)
Dustin [email protected]
Thanks to:Jarrod Lombardo Paul Hoover Sarah Williams Xiang Cao Tony Tang Andy Wilson Ken Hinckley Dennis Wixon August de los ReyesClifton Forlines