The End of Pointing and Clicking: Improving Computer Access with Goal Crossing
description
Transcript of The End of Pointing and Clicking: Improving Computer Access with Goal Crossing
1
The End of Pointing and Clicking: Improving Computer Access with Goal Crossing
June 30, 2009
2
People
3
Design CriteriaEase of motor-impaired performanceHow easy is it motor-wise to acquire the target? Efficiency?No tNo multiple stepsSpeed for able-bodied?Speed for motor-impaired? Safety (Unwanted targets should not be accidentally triggered)How easy/difficult is it to make a false selection?Tolerance to movement error?Is the design reversible? Is it cancelable? Can the user abort? ScalabilityDoes it handle high-density layouts?How does it scale to hundreds of menu items, links, buttons, and icons?Is it compatible with conventional point-and-click widgets? Can both coexist? Etc.Are the benefits of crossing preserved (e.g., “follow-through,” “aggregation”)?Is it easy to learn? Does it require preexisting knowledge? How hard will it be to implement?Is heavy animation required for the technique to work? Can it lend itself to a nice visual design?
4
1. Basic Crossing Ideas
1.1 Cross While Off-hand Button Held Down1.2 Cross and Click Anywhere1.3 Drag and Cross1.4 Cross and Cross-back1.5 Delta t
5
1.1 Cross While Off-hand Button Held Down
OK
6
1.2 Cross and Click Anywhere
OK
Click!
Click!
7
1.3 Drag and Cross
OK
Drag & Cross
8
OK
OK
OK
OK
OK
1.4 Cross and Cross-back
OK
The target gives visual feedback
Cross goal to activate
Cross back toconfirm selection
Cross back toconfirm selection
Cross goal to activate
Cross goal to activate
Activated Selected Double clicked (or)Drag (or)Right clicked…
9
1.4 Cross and Cross-back (cont.)
OK
OK
Then back over, turn 90-degrees and then cross the two bars
Activate an icon by crossing over icon and initial bar
Byron McMullen . Jacob Nelson . Tara Tarjoto . Tim Wibowo
10
1.5 Delta t
OK
Cross and stop for N seconds to confirm the selection
11
1.5 Delta t (cont.)
Cross and Stop
12
2. Advanced Crossing Ideas
2.1 Secondary Goal Crossing2.2 Gesture2.3 Corners and Edges2.4 Moving Targets2.5 Features of Crossing Event
13
OK OK OK OKOK
OK OKOK
OK
• BUTTON
first goal: button second goal (g2): crossing the g1 initiates g2 to appear
g2
Length-to-the-g2 variation angle variation
G2 design variation: how to make it look like a goal?
2.1 Secondary Goal Crossing
14
OKCrossing the first target activates the arc
2.1 Secondary Goal u Arc-shape
Location of the 2nd goal changes depending on the approaching angle
(a)
15
g2 fades out if they are not being selected during a certain amount of time
Right-handed, left-handed
2.1 Secondary Goal v L-Shape
16
• MULTI-LAYERED MENU: Red bar appears only when the menu has no underlying menu items
2.1 Secondary Goal w I-Shape
17
GO
2.1 Secondary Goal x ‘dontclickit’ idea
OK GOOK
ANIMATION
18
2.1 Secondary Goal y Cluster
T1 T2 T3
T4
T7 T8 T9
T6
Interactive prototype
Targets are clustered (3*3 or 2*2) and the secondary goals appear at the fixed location
1 2
3 4
19
2.2 Gesture u Zig-zag / Scribble
OK OKOK
Nope, it’s not selected
Yes
Yes
20
2.2 Gesture v Encircling Targets
ANIMATION
21
2.2 Gesture w Pigtail
ANIMATION
22
2.2 Gesture x Hover Widget
cross
cancel
Turning 90° activates the most recently crossed goal. When the mouse crosses the end of the tunnel, the crossed goal is confirmed
23
2.2 Gesture y Shivering Around a Target
Sparse layout, one small target
24
2.2 Gesture Rectangle Lasso
Philip Phung, Colin Booth, Mark Vitazko, Chris Claiborne
Public DownloadsFile Folder
To open
25
2.2 Gesture Color In
Philip Phung, Colin Booth, Mark Vitazko, Chris Claiborne
Public DownloadsFile Folder
26
2.2 Gesture Letter gesture
Chris Sinco, Rishi Talwar, Justin Wilbourne, Ken Wong
27
OK
OK
2.2 Gesture
Alexei Bespalov, Kevin Merritt, James Rodgers
28
2.3 Corners and Edges u
Constraining mouse movement with edges
29
OK
2.3 Corners and Edges v
Only Listen to Y-Movements
Now only Listen to X-Movements
• Basic Idea: Person can make the computer ignore x or y movements, creating an edge to travel along.
• The edges (or guidelines, or whatever we call them) could be activated with a keyboard/mouse button? Any other options?
• Maybe both can be activated to keep the from moving during a click?
• Could be used for crossing too.
30
Click!
Select
Skip
Select All
Corners are easy to hit.Make a context menu out of a square, where corners are options.
2.3 Corners and Edges w Box
31
2.3 Corners and Edges x
OK OK
Cancel
Open C.M
32
2.3 Corners and Edges y Diamond
Context Menu (C.M.)
Open
Cancel
Crossing one object
Previous
Open
Cancel
Crossing more than one object
Open all
Previous
Open
Cancel
Context Menu (C.M.)
Open
Cancel(Cancel)
33
2.3 Corners and Edges z Hover Widget + Diamond
Crossing one object Crossing more than one object
Context Menu
Open current
Openmultiple
XX
Open multiple
Open current
• Hitting the corner triggers the queued up targets (aggregation)
• Select or delete the targets that are in the queue
Context Menu
C.MOpen
C.M
Open
34
2.3 Corners and Edges 90-degree
OK OK
35
2.3 Corners and Edges
TARGET
36
2.3 Corners and Edges
The mouse can travel through either of the entrances/exits to begin activating the button, but the cursor must travel all the way through to the other side in order to fully trigger the button’s action.
OK
OK
Once the cursor enters the button, barriers at the top and bottom of the button prevent the passage out of the button. These barriers exist only once the mouse moves in a vertical direction.
OK
Chris Sinco, Rishi Talwar, Justin Wilbourne, Ken Wong
37
2.3 Corners and Edges Bounding Buttons
Nathan – Peter - Kaitlyn
Certain elements have un-mousable “screen edges” affixed to them to avoid pointer overshoot.
Ballistic movement can now hit and stick to targets.
38
2.4 Moving target u Coming Toward
Closest target is triggered, and then the target is coming toward the cursor
39
2.4 Moving target v Stepping Backward
Closest target is triggered, and then the target is stepping backward from the cursor
40
2.4 Moving target w Rearranging Targets
41
2.5 Features of Crossing Event u Velocity
Not selected Selected
Needs to cross above certain speed in order to select the target
Too slow
42
2.5 Features of Crossing Event v Acceleration
Not selected Selected
Stop and then accelerate the crossing speed (A>0)
a= 0
43
2.5 Features of Crossing Event Angle
Selected Not selected
Orthogonal crossing: selected
44
3. Other Crossing Ideas3.1 Kinematic Template3.2 Accessible Bubble Cursor3.3 Switch3.4 The Clock3.5 Guideline3.6 Trapezoid3.7 Extended Crossing Line3.8 Plough / Force Field Cursor3.9 Edge.Point3.10 Area Crosser3.11 Stabbing3.12 The Casual Fan3.13 Triple Cross3.14 Cross and Cross Back with Hover Widget3.15 Corner Pocket
45
3.1 Kinematic Template
Passive template: the cursor goes through the button with the help of the (passive) template
OK
OK
46
OK Cancel
EditOptions
Menu
Open
OK Cancel
EditOptions
Menu
Open
Cancel
Options Edit
Tab through targets!
OK MenuOK
OpenEdit
Cancel Menu
3.2 Accessible Bubble Cursor
47
3.3 Switch
Gravity effect + visual feedback
OK
OK
OK
activated
Switching on
48
3.4 The Clock
In this concept, a lever appears inside a circular overlay on the target when the cursor gets close to a goal. To trigger the selection of a target, the user “pushes” the lever (like a clock hand) in a circle until a full rotation is made. The cursor “sticks” to lever, facilitating rotation like the hand on a clock. Once the rotation movement has begun, the user can escape the “stickiness” by sliding the curser off the outer edge of the lever towards the outside of the circle.
Clarke Freeman, Alex Jansen, Kristofer Martin, Josh Rakita
49
3.5 Guideline
If the cursor follows the guideline for a certain amount of time and then cross, the target is selected
Selected
Not Selected
Stop for a while -> guideline appears
Selected
50
3.6 Trapezoid
OK OK OK
51
3.7 Extended Crossing Line u
Extended Crossing Goal
Voronoi tessellation
52
3.7 Extended Crossing Line v
Extended Crossing Goal
Voronoi tessellation Clustering targets around the cursor
53
3.8 Plough / Force Field Cursor
Clarke Freeman, Alex Jansen, Kristofer Martin, Josh Rakita
Cross with acceleration
Inspired by specifically considering the difficult circumstance of a densely arranged set of targets, the force field cursor effectively pushes targets around a circle, or force field, which remains centered on the cursor during tracking movement. Multiple targets in contact with the force field are distributed around the entire circumference. Selection is made by accelerating towards one of the acquired potential targets, achieving an “escape velocity” that allows the cursor leave the center of the field and cross over the desired target.
54
3.9 Edge.Point
Nathan – Peter - Kaitlyn
As you move your mouse around the screen all Icons in the range shown by the area cursor will visually “tunnel” to your cursor. To toggle between freezing and releasing the area cursor press space bar. Freezing the cursor will also freeze all tunnels that are displayed. While tunnels are frozen they will “catch” your mouse cursor and bound it within the tunnel. This allows for easy and quick navigation to all available targets. As a reminder the tunnels and their bounding properties can be escaped from at any time by hitting space bar.
55
3.10 Area Crosser
Nathan – Peter - Kaitlyn
An area cursor that traces a path from the mouse cursor to each target. Instead of traveling down the entire path, a goal is presented at the very beginning to cross and thus activate the button.
Pros: Pure crossing goal, reduces cursor travel distance.
Cons: Hard to implement actions other than "cross."
CLICK!
56
3.11 Stabbing
Philip Phung, Colin Booth, Mark Vitazko, Chris Claiborne
A user moves to a group of items; as the cursor approaches them, the items move around the cursor in a fan motion allowing for the user to select one by moving their mouse backwards over the target. The user is then presented with a menu of possible selection, which they can thereby make their selection by passing back over the icon.
Select Delete
57
3.12 The Casual Fan
Chris Sinco, Rishi Talwar, Justin Wilbourne, Ken Wong
45° arch to commit to selection
45° arch to commit to selection
90° arch for escape selection
Cancel
Activate the target
Activate the target
58
3.13 Triple Cross
Alexei Bespalov, Kevin Verritt, James Rodgers
The technique works by having the user cross over a goal three times, before he crosses over any other goals three times within additional crosses of the original goal. This technique is similar to the cross and cross-back technique, however it adds both safety and flexibility to the technique by adding a third cross of the goal.
59
Public DownloadsFile Folder
3.14 Cross & Cross-back with Hover Widget
Alexei Bespalov, Kevin Verritt, James Rodgers
The technique works by having the user cross over a goal three times, before he crosses over any other goals three times within additional crosses of the original goal. This technique is similar to the cross and cross-back technique, however it adds both safety and flexibility to the technique by adding a third cross of the goal.
Copy Properties
Rename
Open
Delete
Cut
Send
60
3.15 Corner Pocket
Greg Bickford, Don Bushell, Gavin Elster, Andy Boer
61
4. Designing Actions and Context Menu
4.1 Accessing context menu with Hover Widget (list)4.2 Accessing context menu with Hover Widget (circular)4.3 Accessing context menu with Hover Widget + Diamond (list)4.4 Accessing context menu with Hover Widget + Diamond (circular)4.5 Accessing context menu with Secondary goal crossing
62
4.1 Accessing Context Menu with Hover Widget (list)• Right turn: Access context menu (= right click)
MoveCopyDeleteOpen with…CutSelectGroupProperties
MoveCopyDeleteOpen with…CutSelectGroupProperties
• List context menu can handle sub-context menu well
OpenOffice.orgMicrosoft PowerpointKeynote
• ‘Select’ corresponds to the single left click in Windows
Confirm
MoveCopyDeleteOpen with…CutSelectGroupProperties
• Relative turn• Absolute 90-degree turn
63
4.2 Accessing Context Menu with Hover Widget (circular)
Cancel
Move Copy
Select
Open with…
Cancel
Copy
Select
Open with…
Cancel
Cut
Move
Group
Properties
Move Copy
Select
Open with…
Cancel
Move Copy
Select
Open with…
Cancel
• Right turn: Access context menu
64
4.3 Accessing Context Menu with Hover Widget + Diamond (list)
UpDown
OpenCutCopyMoveDeleteSelectRenameProperties
Open
MoveCopySelect allOpen with…CutDeleteGroupPropertiesBack
‘Select all’ becomes one of the context menu items
• Turn right • Left turn, and at the right corner of the diamond
65
4.4 Accessing Context Menu with Hover Widget + Diamond (circular)
C.MPrevious
C.M
Delete
Copy Select
Open with…
Cancel
Group
C.M
Open
Previous
Cancel
Open
66
4.5 Accessing Context Menu with Secondary Goal Crossing
Move
Copy
Select
Open with…
CancelOpen
Move
Copy
Select
Open with…
Cancel
Open
• Sequencecross -> secondary goal appears -> turn 90 degree
• Cancel1) timeout2) menu item3) cross back4) loophole
merge left-click, right-click, single-click and double-click
Move
Copy
Select
Open with…
Open
67
The End