03 Input Device - Department of Computer Scienceirani/cs792/03_Input_Device.pdf · 1 Input...
Transcript of 03 Input Device - Department of Computer Scienceirani/cs792/03_Input_Device.pdf · 1 Input...
1
Input Technologies and Techniques
What’s an input deviceEvaluation and analysis of input devices
Input Device StatesInteraction Modalities
Hinckley, K., Input Technologies and Techniques, Chapter 7. Handbook of Human-Computer Interaction, ed. by Andrew Sears and Julie A. Jacko. Lawrence Erlbaum & Associates. 10/26/2006 74.792 - Graduate Course in HCI 2
What’s an input device
• Input devices sense physical properties of people, places or things
• However, they do not operate in isolation, i.e. need visual feedback– otherwise similar to a pen without paper
• Must include:– the physical sensor (positioning wheels)– the feedback presented to the user (cursor)– the ergonomic of the device (fits in hand)– interplay between all the interaction techniques supported by a system
(clicking, moving, selecting, etc.)
• Need an understanding of input technologies to design interaction techniques that match a user’s natural workflow
10/26/2006 74.792 - Graduate Course in HCI 3
Input Device Properties
• Several properties characterize most devices:– Property sensed– Number of dimensions– Indirect vs. direct– Device acquisition time– Gain
10/26/2006 74.792 - Graduate Course in HCI 4
Property Sensed
• Most devices sense – Linear position (tablets sense position of pen)– Motion (mice sense change in position)– Force (Isometric joysticks, IBM Trackpoint)– Angle or change in angle (rotary input)
• Absolute input device– position sensing
• Relative input device– motion sensing
• Relative device requires visual feedback but also can be inefficient due to clutching
2
10/26/2006 74.792 - Graduate Course in HCI 5
Number of Dimensions
• Devices sense one or more input dimensions– Two linear dimensions (mouse, x/y)– Angular dimension (knob)– 6 degree-of-freedom (magnetic tracker, senses 3-
position dimensions and 3 orientation dimensions)
• A pair of knobs is a 1D+1D device, mouse with scroll wheel is a 2D+1D multi-channel device
• Multiple degree-of-freedom devices sense three or more simultaneous dimensions of spatial position or orientation
10/26/2006 74.792 - Graduate Course in HCI 6
Degrees of Freedom vs. Dimensions
the bat
Ware, C. and Jessome, D. (1988), Using the Bat: A Six Dimensional Mouse for Object Placement. IEEE Computer Graphics and Applications. November 8-6, 65-70.
Usually confuse the idea of dimensions and degrees-of-freedom
10/26/2006 74.792 - Graduate Course in HCI 7
Degrees of Freedom vs. Dimensions
A Two-Ball Mouse Affords Three Degrees of FreedomI. Scott MacKenzie, R. William Soukoreff, & Chris PalCHI 97 Electronic Publications: Late-Breaking/Short Talks
3DOF Mouse
10/26/2006 74.792 - Graduate Course in HCI 8
Degrees of Freedom vs. Dimensions
Ravin Balakrishnan, Thomas Baudel, Gordon Kurtenbach, George W. Fitzmaurice. (1997). The Rockin'Mouse: Integral 3D manipulation on a plane. Proceedings of ACM CHI 1997 Conference on Human Factors in Computing Systems, p. 311-318.
4DOF Mouse
3
10/26/2006 74.792 - Graduate Course in HCI 9
Degrees of Freedom vs. Dimensions
Hinckley, K., Sinclair, M., Hanson, E., Szeliski, R., Conway, M., The VideoMouse: A Camera-Based Multi-Degree-of-Freedom Input Device, ACM UIST'99 Symposium on User Interface Software & Technology, pp. 103-112.
5DOF Mouse
10/26/2006 74.792 - Graduate Course in HCI 10
Indirect vs. Direct
• Mouse is indirect, i.e. must move mouse to move pointer on screen
• Touch-screens or tablets are direct input devices, i.e. unified input and display surface– Occlusion is typically a problem
10/26/2006 74.792 - Graduate Course in HCI 11
Device Acquisition Time
• Acquisition time: average time to move hand to device
• Homing time: average time to return to a ‘home’ position, i.e. mouse to keyboard
• For common desktop workflows, pointing and selecting dominate acquisition/homing time– integration of pointing with keyboard may not improve
overall performance
10/26/2006 74.792 - Graduate Course in HCI 12
Gain
• Control-to-display gain or C:D ratio– distance moved by an input device/distance moved on the
display
• Composite measurement taking into account device size and display size
• Gain has very little effect on the time to perform pointing movements– not a commonly used metric
4
10/26/2006 74.792 - Graduate Course in HCI 13
Pointing Devices
• Mouse:– Invented in ’67– Used for pointing– Picks up changes in x,y– As good as pointing with finger– Integrated with buttons/wheels etc
• Trackball:– Senses relative motion of partially exposed ball in
2DOF– Engage different muscle groups than the mouse, but
an alternative for those who experience discomfort
10/26/2006 74.792 - Graduate Course in HCI 14
Pointing Devices
• Isometric joystick:– Force sensing– Rate of cursor is proportional to the force exerted– Returns to center when released– Good when space is at a premium
• Isotonic joystick:– Sense angle of deflection– Different than isometric joystick
J. Lipscomb and M. Pique (1993). Analog Input Device Physical Characteristics. SIGCHI Bulletin 25 (3): 40-45.
10/26/2006 74.792 - Graduate Course in HCI 15
Pointing Devices
• Touchpads:– Small touch-sensitive devices found on laptops– Use relative mode for cursor control– Can operate in absolute mode by dragging finger and
leaving it on edge of the pad– Necessitates multiple clutchings
• Touchscreens/pen-operated devices:– Fingers, or electromagnetic digitizers– Parallax error, mismatch between sensed input
position and apparent position
10/26/2006 74.792 - Graduate Course in HCI 16
Input Device States
• Disaccord between states of a GUI and states and events sensed by devices
• Difficult to support interface primitives such as click, drag, double-click, and right-click
• Useful to diagram device states– Identifies relationship between events sensed by
input device and interaction technique demands
5
10/26/2006 74.792 - Graduate Course in HCI 17
Three-State Model of Graphical Input
• Buxton's 3-state model for graphical input devices
• Expression of the operation of computer pointing devices in terms of state transitions
• Expressive vocabulary for exploring the relationship between pointing devices and the interaction techniques they afford
10/26/2006 74.792 - Graduate Course in HCI 18
Input Device States
10/26/2006 74.792 - Graduate Course in HCI 19
Input Device States
• Three-states:– The states are Out of range (State 0), for clutching or repositioning a mouse on a mouse pad; – Tracking (State 1) for moving a tracking symbol such as a cursor about a display– Dragging (State 2) for moving an icon on the desktop or for grouping a set of objects or a range of text
• Seems simple and obvious but can add insight to the existing body of pointing device research
– can be extended to multi-button interaction, stylus input, and direct vs. indirect input
10/26/2006 74.792 - Graduate Course in HCI 20
Input Device States
• Based on Buxton’s model, the mouse & touch-sensitive devices are a two-state device
6
10/26/2006 74.792 - Graduate Course in HCI 21
Activity: Input Device States
• Based on the previous state diagram, can you describe a limitation of touch-sensing input (PDA’s)
• Do you know of a device that supports all 3 states
10/26/2006 74.792 - Graduate Course in HCI 22
Input Device States
• Can fully capture core interaction by extending the 3-state model
10/26/2006 74.792 - Graduate Course in HCI 23
Evaluation and Analysis of Input Devices
• GOMS, Keystroke-Level Model• Fitts’ Law • Hick-Hyman Law• Power Law of Practice• Accot’s Steering Law
10/26/2006 74.792 - Graduate Course in HCI 24
GOMS Models
• Goals, Operators, Methods, Selection rules
• Card, Moran, Newell, 1980/83– a priori assumptions and predictions– wide coverage of various subtasks in HCI– empirically proven learnable and usable in practice– good approximation to actual experimental data
7
10/26/2006 74.792 - Graduate Course in HCI 25
What is a GOMS model?
• Goals - the state the user wants to achieve – i.e. find a website
• Operators - the cognitive processes & physical actions performed to attain those goals– i.e. decide which search engine to use
• Methods - the procedures for accomplishing the goals– i.e. drag mouse over field, type in keywords, press the go button
• Selection rules - determine which method to select when there is more than one available
• Assist with:– Predicting usage patterns in editors– Power key assignments to reduce typing keystrokes– Predicting performance with hierarchical menus
10/26/2006 74.792 - Graduate Course in HCI 26
Goals
• Something user tries to accomplish– i.e. copy a file, create a directory
• Often hierarchical– May require sub-tasks to be accomplished
• Typically an action-object pair– i.e. copy file
• Can be thought of in lay terms
10/26/2006 74.792 - Graduate Course in HCI 27
Operators
• Actions user performs, or actions system allows user to take
• Operator is something that is simply executed– e.g., press a key on keyboard
• Also action object pair– e.g. press button, drag cursor
10/26/2006 74.792 - Graduate Course in HCI 28
Types of operators
• External operators– Observable “physical” actions between user & system
• Mental operators– Internal actions performed by users– Non observable – Contentious, hypothetical, guess?
8
10/26/2006 74.792 - Graduate Course in HCI 29
Methods
• Well learned sequence of operators needed to achieve a goal– External + mental operators
• If user can perform a task, it means they have a method for the task– Method is routine cognitive/perceptual/motor skill– No planning required to determine actions
10/26/2006 74.792 - Graduate Course in HCI 30
Selection Rules
• If more than one method, selection rule routes control to most appropriate method
• Often internal rules users follow to pick best method– i.e., to delete a paragraph of text, there could be different
methods depending on length of paragraph, with appropriate selection rules
– i.e. how many ways to delete a paragraph? Which is the most effective?
10/26/2006 74.792 - Graduate Course in HCI 31
Four GOMS models
• Keystroke level model– Some simplifying assumptions– Pre-established operators for prediction
• CMN GOMS (Card Moran Newell)– Psuedo code notation, strict structure
• NGOMSL (Natural GOMS Language)– Program form, very general– Breadth first expansion of top level goals into methods– Only one to predict both performance and learning times
• CPM GOMS (Cognitive Perceptual Motor GOMS)– Based on model-human processor– Allows parallel performance of operators
10/26/2006 74.792 - Graduate Course in HCI 32
Keystroke Level Model
• Only address one aspect of task performance: time
• Predicts expert error-free task-completion time with the following inputs:– a task or series of subtasks– method used– command language of the system– motor-skill parameters of the user– response-time parameters of the system
• Predict time an expert would take to execute the tasks– Assuming no errors
9
10/26/2006 74.792 - Graduate Course in HCI 33
KLM operators
• Six operators– Keystroke
• Avg time determined by std typing tests– Pointing
• Pointing with a mouse or other device on a display to select an object.• Varies from 0.8 – 1.5 seconds
– Homing• Bring ‘home’ hands on the keyboard or other device• 0.4 seconds based on various studies
– Mental• 1.35 seconds, experimentally determined
– Response
• Texecute = TK + TP + TH + TM + TR
10/26/2006 74.792 - Graduate Course in HCI 34
Encoding Methods
• E.g., replace 5 letter word with another in a text editor• Reach for mouse Hmouse
• Point to word Pword
• Select word K• Home on keyboard Hkeyboard
• Call replace cmd M,Kreplace
• Type new 5 letter word 5Kword
• Texecute = TM + TP + 2TH + 7Tk
10/26/2006 74.792 - Graduate Course in HCI 35
Where does GOMS apply?
• Where users perform tasks at expert level– users have mastered a skill– users are not problem solving– users know what to do, just act on the steps
10/26/2006 74.792 - Graduate Course in HCI 36
Using GOMS
• Qualitatively– Used for designing training programs, help systems– Focus design on problem areas
• Quantitatively– Good predictions of performance time– Maybe some predictions on learning
10
10/26/2006 74.792 - Graduate Course in HCI 37
GOMS weaknesses
• The model applied to skilled users, not to beginners or intermediate– model doesn't account for either learning of the system or its recall after
a period of disuse– skilled users occasionally make errors; however, the model doesn't
account for errors.
• Model explicit about elementary perceptual and motor components.– cognitive processes in skilled behaviour are treated in a less
distinguished fashion
• Model does not address:– mental workload– functionality: which tasks should be performed by the system– user fatigue
10/26/2006 74.792 - Graduate Course in HCI 38
Hick-Hyman Law
• Law for choice reaction-time, given in the form of a prediction equation
• Given a set of n stimuli (flashing objects), associated with n responses (selecting object), reaction time (RT) can be given as follows:
RT = a + b log 2(n) – a, b empirically determined constants
• Examples include:– selection time by phone operator when light behind button
appears (Card et al ., 1983)– measuring and predicting time to select items in a hierarchical
menu (Landauer & Nachbar, 1985)– predicting text-entry rates on soft keyboards with non-qwerty
layouts, since users have to visually scan the layout (MacKenzieet al., 1995, 1999)
10/26/2006 74.792 - Graduate Course in HCI 39
A Quiz Designed to Give You Fitts
•http://www.asktog.com/columns/022DesignedToGiveFitts.html
• Microsoft Toolbars offer the user the option of displaying a label below each tool. Name at least one reason why labeled tools can be accessed faster. (Assume, for this, that the user knows the tool and does not need the label just simply to identify the tool.)
10/26/2006 74.792 - Graduate Course in HCI 40
A Quiz Designed to Give You Fitts
1. The label becomes part of the target. The target is therefore bigger. Bigger targets, all else being equal, can always be acccessedfaster. Fitt's Law.
2. When labels are not used, the tool icons crowd together.
11
10/26/2006 74.792 - Graduate Course in HCI 41
A Quiz Designed to Give You Fitts
• You have a palette of tools in a graphics application that consists of a matrix of 16x16-pixel icons laid out as a 2x8 array that lies along the left-hand edge of the screen. Without moving the array from the left-hand side of the screen or changing the size of the icons, what steps can you take to decrease the time necessary to access the average tool?
10/26/2006 74.792 - Graduate Course in HCI 42
A Quiz Designed to Give You Fitts
1. Change the array to 1X16, so all the tools lie along the edge of the screen.
2. Ensure that the user can click on the very first row of pixels along the edge of the screen to select a tool. There should be no buffer zone.
10/26/2006 74.792 - Graduate Course in HCI 43
A Quiz Designed to Give You Fitts
• Microsoft offers a Taskbar which can be oriented along the top, side or bottom of the screen, enabling users to get to hidden windows and applications. This Taskbar may either be hidden or constantly displayed. Describe at least two reasons why the method of triggering an auto-hidden Microsoft Taskbar is grossly inefficient.
10/26/2006 74.792 - Graduate Course in HCI 44
A Quiz Designed to Give You Fitts
• Screen edges are prime real estate. You don't waste an entire edge that could be housing a couple of dozen different fast-access icons just for one object, the Taskbar
• The auto-hidden Taskbar is entirely too easy to display by accident. Users are constantly triggering it when trying to access something that is close to, but not at, the edge
• The Taskbar would not have any of these problems, yet be even quicker to get to if it were located at any one of four corners of the display. Throw the mouse up and to the left, for example, and you'll have a taskbar displayed. Fast access without the false triggering
12
10/26/2006 74.792 - Graduate Course in HCI 45
Fitts’ Law
• Robust and highly adopted model of human movement
• Originated as interest of applying information theory to the analysis and understanding of difficulty of movement tasks & human rate of information processing
• Used Shannon’s law for information capacityC = B log2(S / N + 1)
» S is the signal power and N is the noise power
• Based on the following analogies:– Amplitude of aimed movement == electronic signal– Spatial accuracy of movement == electronic noise
10/26/2006 74.792 - Graduate Course in HCI 46
Fitts’ Law
• Described the analogy in two papers:
– a serial, or reciprocal, target acquisition task wherein subjects alternately tapped on targets of width W separated by amplitude A
– experiment using a discrete task, wherein subjects selected one of two targets in response to a stimulus light
Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47, 381-391.
Fitts, P. M., & Peterson, J. R. (1964). Information capacity of discrete motor responses. Journal of Experimental Psychology, 67, 103-112.
10/26/2006 74.792 - Graduate Course in HCI 47
Fitts’ Law
• Quantify a movement task's difficulty — ID, the index of difficulty
ID = log2(A / W + 1) (bits)A = amplitude, W = width of object
• Movement time to complete a task is predicted using a linear equation
MT = a + b * ID (secs)a & b are empirically determined using linear regression
• Throughput (TP) or Index of Performance (IP) isTP = ID / MT (bits/sec)
10/26/2006 74.792 - Graduate Course in HCI 48
Fitts’ Law
• To determine a and b design a set of tasks with varying values for A and W (conditions)
• For each task condition – multiple trials conducted and the time to execute each is
recorded and stored electronically for statistical analysis
• Accuracy is also recorded– either through the x-y coordinates of selection or – through the error rate — the percentage of trials selected with
the cursor outside the target
13
10/26/2006 74.792 - Graduate Course in HCI 49
Fitts’ Law
Same ID → Same Difficulty
Target 1 Target 2
10/26/2006 74.792 - Graduate Course in HCI 50
Fitts’ Law
Smaller ID → Easier
Target 2Target 1
10/26/2006 74.792 - Graduate Course in HCI 51
Fitts’ Law
Larger ID → Harder
Target 2Target 1
10/26/2006 74.792 - Graduate Course in HCI 52
Fitt’s Law
1555 1.48 644 2.92 2.40 Mean:
1480 0.83 615 4.17 2.32 40 160
1788 1.67 823 5.42 3.17 20 160
2353 2.08 979 3.75 4.09 10 160
1175 0.83 481 1.67 1.58 40 80
1442 2.08 604 1.67 2.32 20 80
1874 2.08 762 2.92 3.17 10 80
1001 0.42 361 1.25 1.00 40 40
1293 2.08 501 3.33 1.58 20 40
1587 1.25 665 2.08 2.32 10 40
MT (ms) ER (%) MT (ms) ER (%)
Device 'B' Device 'A' ID (bits) W (pixels) A (pixels)
Example data sets for two devices from a Fitts' law experiment
14
10/26/2006 74.792 - Graduate Course in HCI 53
Fitt’s Law
10/26/2006 74.792 - Graduate Course in HCI 54
Fitt’s Law• If primary goal in Fitts’ law experiment is to determine performance
between devices/interaction techniques, then throughput (TP) is best criterion
– TP = ID/MT
• If for a given device ID = 4.09 bits and task is executed in MT = 979 ms– human rate of information processing for that task is 4.09 / 0.979 = 4.18
bits/s or TP = 4.18 bits/s
• Mean throughput across all the A-W conditions for Device 'A' is TP = 2.40 / 0.644 = 3.73 bits/s
• For Device 'B', TP = 2.40 / 1.555 = 1.57 bits/s
• Using throughput we conclude users' performance with Device 'A' was about 3.73 / 1.57 = 2.4 times better than performance with Device 'B'
10/26/2006 74.792 - Graduate Course in HCI 55
Setting it up– MacKenzie, I. S. (1995). Movement time prediction in human-computer interfaces. In R. M.
Baecker, W. A. S. Buxton, J. Grudin, & S. Greenberg (Eds.), Readings in human-computer interaction (2nd ed.) (pp. 483-493). Los Altos, CA: Kaufmann. [reprint of MacKenzie, 1992]
– http://www.yorku.ca/mack/GI92.html
• Vary A,W values for at least 4 ID conditions– Small A, small W– Small A, large W– Large A, small W– Large A, large W– 2-4 variations in between
• Clicking start position presents object to click on– Record whether user missed– Record time to click on stimulus
• Design with several repetitions and several blocks10/26/2006 74.792 - Graduate Course in HCI 56
Case Study #1: Text Entry Rates on Mobile Phones
• Can we predict text entry rate on mobiles using Fitts’ Law?
15
10/26/2006 74.792 - Graduate Course in HCI 57
Case Study #1: Text Entry Rates on Mobile Phones
• Two main approaches:– Multi-tap:
• presses each key one or more times to specify the input character • large overhead: 33 key presses 15 characters of text• "on average" multi-tap method requires 2.034 keystrokes per
character
77 88 444 222 55 0 22 777 666 9 66 0 333 666 99 q u i c k _ b r o w n _ f o x
– One-key disambiguation:• Add linguistic knowledge to make best guess• Can be ambiguous in some cases, have to correct
7 8 4 2 5 0 2 7 6 9 6 0 3 6 9 q u i c k _ b r o w n _ f o x
10/26/2006 74.792 - Graduate Course in HCI 58
Case Study #1: Text Entry Rates on Mobile Phones
• Text entry on a mobile phone, for example, consists of aiming for and acquiring (pressing) a series of keys "as quickly and as accurately as possible“
• Time to press any key, given any previous key, can be readily predicted using Fitts' law
• For index finger input = MT = 165 + 52 ID
• and for thumb input = MT = 176 + 64 ID
10/26/2006 74.792 - Graduate Course in HCI 59
Case Study #1: Text Entry Rates on Mobile Phones
• Elements to build a text-entry prediction model are:
– Information on position and size of keys (ruler)
– Letter assignment to keys (any phone)
– Relative probabilities of digrams (probabilities of letter pairs) in target language (sources)
0.000000.000020.004800.008100.009600.01800Space
0.000020.000080.000000.000000.000000.00003Z
0.025000.000000.000350.000000.000010.00099D
0.000440.000000.000000.000120.000000.00340C
0.000340.000000.000000.000000.000130.00130B
0.000470.000110.003600.002900.001300.00002A
SpaceZDCBA
t-h or e-space have high Pg-k or f-v have low P
10/26/2006 74.792 - Graduate Course in HCI 60
Case Study #1: Text Entry Rates on Mobile Phones
40.6 45.7 One-key with disambiguation
20.824.5
22.527.2
Multi-tap- wait for timeout- timeout kill
Thumb Index Finger
Predicted Expert Entry Rate (wpm) Method
• Time to enter each i-j sequence is predicted using Fitts’ law giving MTij, weighted by the probability of the digram in the target language Pij
MTL = ∑∑ (Pij × MTij )WPM = MTL × (60 / 5) (avg 5 chars/word)
2 assumptions: - all words are in dictionary- when ambiguity arises the intended word is the most probable
16
10/26/2006 74.792 - Graduate Course in HCI 61
Case Study: Using Fitts’ to redesign text entry
http://www.exideas.com/ME/http://www.exideas.com/ME/Pressfolder/PressReleaseDec2-2003.html
Nesbat, S. “A System for Fast, Full-Text Entry for Small Electronic Devices“, Proceedings of the Fifth International Conference on Multimodal Interfaces, ICMI 2003 (ACM-sponsored), Vancouver, November 5-7, 2003.
MessagEase Onscreen Keyboard
Example of an interface designwhich can be adapted to multiple devices
10/26/2006 74.792 - Graduate Course in HCI 62
Using Letter Frequency
10/26/2006 74.792 - Graduate Course in HCI 63
Nine Most Frequent Letters: Double Click
ETNROIASH
10/26/2006 74.792 - Graduate Course in HCI 64
Eight Less Frequent Letters: Two Taps
DCUPGBQJ
17
10/26/2006 74.792 - Graduate Course in HCI 65
Remaining Nine Letters: Two Taps
FMYWVXKZ
10/26/2006 74.792 - Graduate Course in HCI 66
Adding Space, Shift, and Mode
10/26/2006 74.792 - Graduate Course in HCI 67
Special Characters
38 special characters entered by two taps;6000+ characters can be entered with combine.
10/26/2006 74.792 - Graduate Course in HCI 68
Soft Keyboard Design
Soft KeyHard Key
Single DragTwo ClicksLess Frequent Letters
Single TapDouble ClickMost Frequent Letters
The same mapping used for letters
18
10/26/2006 74.792 - Graduate Course in HCI 69
Special Characters
Entered with a single drag
10/26/2006 74.792 - Graduate Course in HCI 70
Optimization and Evaluation
• Exhaustively simulated all permutations of letters within each group
• The configuration with the max speed was selected
10/26/2006 74.792 - Graduate Course in HCI 71
Fitts’ Law
Movement Time from one key to another:
MT = a + b*log2(A/W+1)
WA
10/26/2006 74.792 - Graduate Course in HCI 72
Digraph Probability
The probability Pij that letter j will follow letter i in a body of text:
ΣΣPij = 1
0.000000.000020.004800.008100.009600.01800Space
0.000020.000080.000000.000000.000000.00003Z
0.025000.000000.000350.000000.000010.00099D
0.000440.000000.000000.000120.000000.00340C
0.000340.000000.000000.000000.000130.00130B
0.000470.000110.003600.002900.001300.00002A
SpaceZDCBA
19
10/26/2006 74.792 - Graduate Course in HCI 73
Performance Measure – Hard Key
Calculation of max theoretical entry speed:• Movement Time
– MT = a + b log2(A/W+1)
• Total time (2 Clicks)– CT = MT1 + MT2
• Total time (Dble Click – no movement)– CTDC = 2a + b log2(A/W+1)
• Average Time
– CTav = ΣΣ(Pij × CTij)
• Speed – WPM = (1/ CTav) ×(60/5)
10/26/2006 74.792 - Graduate Course in HCI 74
Performance Measurement – Soft Key
– Most frequent characters – Single tap: • TLi = (1/4.9) log2 [(D0-i/W) + 1]; if D0-i> 0,• TLi = a; if D0-i = 0
– Less frequent characters – Drag:• TLjk = t0-j + tdown + tj-k + tup• TLjk = (t0-j + tdown+ tup) +
(tj-k + tup+ tdown) –(tdown+ tup)
• TLjk = TLj + TLk– a t0-j: time to move to key jtj-k: time to move from key j to key ktdown: time to move stylus downtup: time to move stylus up
10/26/2006 74.792 - Graduate Course in HCI 75
Hard Key (Cell Phone) Comparison
User Study
0
2
4
6
8
10
12
Multi-tap MessagEase
WPM
Theoretical
0
10
20
30
Multi-tap MessagEase
WPM
130%
209%
10/26/2006 74.792 - Graduate Course in HCI 76
Example: Expanding targets
TO READ:• McGuffin, M. & Balakrishnan, R., Acquisition of
Expanding Targets. Proceedings of ACM Conference on Human Factors in Computing Systems (CHI) 2002, pages 57-64
20
10/26/2006 74.792 - Graduate Course in HCI 77
Example: Expanding targets
FurnasGeneralized fisheye viewsCHI 1986 Bederson
Fisheye MenusUIST 2000
Mackinlay, Robertson, CardThe Perspective WallCHI 1991
10/26/2006 74.792 - Graduate Course in HCI 78
Mac OS X “dock”
Does this make acquisition easier ?
• Size of the interface widget (viewing region) changes dynamically
– Provide the user with a magnified target area at their focus of attention (area around the cursor)
– Expanding toolbar implemented in latest Apple OS X operating system
10/26/2006 74.792 - Graduate Course in HCI 79
Advantages and Disadvantages
• Advantages– Icons are displayed in reduced size to solve the
increasing number of commands and icons – Larger amount of screen real estate devoted to the
display of the underlying data
• Disadvantages– Can reduce the user’s ability to select the desired
icon efficiently
10/26/2006 74.792 - Graduate Course in HCI 80
Fitts’ Law
• 3 different scenarios describing what Fitts’ Law is modeling
– Iterative corrections model
– Impulse variability model
– Optimized initial impulse model
21
10/26/2006 74.792 - Graduate Course in HCI 81
Iterative Corrections Model
• States that the movement consists of many discrete sub-movements
• Each sub-movement takes the user closer to the target
• Sub-movements are triggered by feedback indicating the target has not been reached yet
10/26/2006 74.792 - Graduate Course in HCI 82
Impulse Variability Model
• Movement consists of initial impulse delivered by the muscles towards the target, flinging the limb towards the target
• Last part of movement time consists of limb coasting towards target
• Either type of explanation cannot explain the Fitt’s Law completely given a range of tasks
10/26/2006 74.792 - Graduate Course in HCI 83
Optimized Initial Impulse Model (1)
• Most complete and successful explanation to the Fitts’ Law
• Combination of the iterative corrections and the impulse variability models– movement is initiated towards the target– task is completed if the movement lands at the target– another movement is required if it lands outside the
target– same processes will be carried out until the target is
reached
10/26/2006 74.792 - Graduate Course in HCI 84
Distance
Speed
What does Fitts’ Law really model?
W
UndershootOvershoot
Open-loop
Closed-loop
22
10/26/2006 74.792 - Graduate Course in HCI 85
Expanding Targets
• Basic Idea:– Big targets can be acquired faster, but take up more
screen space– So: keep targets small until user heads toward them– Can this be used for devices with small viewing
space?
Cancel
Okay
Click Me !
10/26/2006 74.792 - Graduate Course in HCI 86
Experiment Goals
The experiment was designed to answer the followingquestions for a typical expanding target selection task:
1. Can such a task be modeled by Fitts’ law?2. If it can be modeled by Fitts’ law, is it possible to predict
performance in such tasks from a base set of data where no expansion takes place?
3. Is movement time dependent on the final target width and not the initial one at onset of movement?
4. At what point should the target begin expanding?5. Do different target expansion strategies affect performance?
10/26/2006 74.792 - Graduate Course in HCI 87
Experimental Setup
Target
Start Position
W
A
10/26/2006 74.792 - Graduate Course in HCI 88
Experimental Setup
Expansion:How ?
Animated
Expansion
23
10/26/2006 74.792 - Graduate Course in HCI 89
Experimental Setup
Expansion:How ?
Fade-in
Expansion
10/26/2006 74.792 - Graduate Course in HCI 90
Experimental Setup
Expansion:How ?When ? P = 0.25
10/26/2006 74.792 - Graduate Course in HCI 91
Experimental Setup
Expansion:How ?When ? P = 0.5
10/26/2006 74.792 - Graduate Course in HCI 92
Experimental Setup
Expansion:How ?When ? P = 0.75
24
10/26/2006 74.792 - Graduate Course in HCI 93
Pilot Study
• 7 conditions:– No expansion (to establish a, b values)– Expanding targets
• Either animated growth or fade-in• P is one of 0.25, 0.5, 0.75
– (Expansion was always by a factor of 2)
10/26/2006 74.792 - Graduate Course in HCI 94
Pilot Study
7 conditionsx 16 (A,W) valuesx 5 repetitionsx 2 blocksx 3 participants= 3360 trials
10/26/2006 74.792 - Graduate Course in HCI 95
Pilot Study: Results
Time (seconds)
ID (index of difficulty)1 2 3 4 5 6 7 8
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
10/26/2006 74.792 - Graduate Course in HCI 96
Pilot Study: Results
Time (seconds)
ID (index of difficulty)1 2 3 4 5 6 7 8
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
)1(log 2 ++WAba
25
10/26/2006 74.792 - Graduate Course in HCI 97
Pilot Study: Results
Time (seconds)
ID (index of difficulty)1 2 3 4 5 6 7 8
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
)1(log 2 ++WAba
)121(log 2 ++
WAba
10/26/2006 74.792 - Graduate Course in HCI 98
Pilot Study: Results
Time (seconds)
ID (index of difficulty)1 2 3 4 5 6 7 8
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
static targets − measured expanding targets − lower boundexpanding targets − measured
P = 0.25
10/26/2006 74.792 - Graduate Course in HCI 99
Pilot Study: Results
Time (seconds)
ID (index of difficulty)1 2 3 4 5 6 7 8
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
static targets − measured expanding targets − lower boundexpanding targets − measured
P = 0.5
10/26/2006 74.792 - Graduate Course in HCI 100
Pilot Study: Results
Time (seconds)
ID (index of difficulty)1 2 3 4 5 6 7 8
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
static targets − measured expanding targets − lower boundexpanding targets − measured
P = 0.75
26
10/26/2006 74.792 - Graduate Course in HCI 101
• Pilot Study suggests the advantage of expansion doesn’t depend on P
• So, set P = 0.9 and perform a more rigorous study
Implications
10/26/2006 74.792 - Graduate Course in HCI 102
Full Study
• 2 conditions:– No expansion (to establish a, b values)– Expanding targets, with
• Animated growth• P = 0.9• Expansion factor of 2
10/26/2006 74.792 - Graduate Course in HCI 103
Full Study
2 conditionsx 13 (A,W) valuesx 5 repetitionsx 5 blocksx 12 participants= 7800 trials
10/26/2006 74.792 - Graduate Course in HCI 104
Results
Time (seconds)
A, W values
27
10/26/2006 74.792 - Graduate Course in HCI 105
Results
Time (seconds)
ID (index of difficulty)3 4 5 6 7
0.8
1
1.2
1.4
1.6
1.8
10/26/2006 74.792 - Graduate Course in HCI 106
Results
Time (seconds)
ID (index of difficulty)3 4 5 6 7
0.8
1
1.2
1.4
1.6
1.8
10/26/2006 74.792 - Graduate Course in HCI 107
Results
Time (seconds)
ID (index of difficulty)3 4 5 6 7
0.8
1
1.2
1.4
1.6
1.8
10/26/2006 74.792 - Graduate Course in HCI 108
Results
Time (seconds)
ID (index of difficulty)3 4 5 6 7
0.8
1
1.2
1.4
1.6
1.8
static targets − measured expanding targets − lower boundexpanding targets − measured
P = 0.9
28
10/26/2006 74.792 - Graduate Course in HCI 109
Implications
• For single-target selection task,– Expansion yields a significant advantage, even when
P=0.9
• What about multiple targets ?
10/26/2006 74.792 - Graduate Course in HCI 110
Implications for Design (1)
• Experimental results can influence the design of buttons, menus, or other selectable widgets
• Interface with multiple expanding targets does not need to predict cursor's trajectory to anticipate which widgets to expand– Instead, just expand widgets as the cursor approaches them
10/26/2006 74.792 - Graduate Course in HCI 111
Implications for Design (2)
• Expansion Strategies for adjacent widgets (e.g. toolbars)
• Expanding a widget around its center will cause overlap & occlusion with nearby widgets
– Expanding a group of widgets around a group’s center
– Expand nearest widgets and move adjacent widgets away
– Expand nearest widgets, but allow some overlap as well as expand adjacent widgets so they are easier to see
10/26/2006 74.792 - Graduate Course in HCI 112
Summary
• Expanding targets acquisition of can be accurately modeled by Fitts’ Law
• User performance is aided by target expansion
• Targets that are always expanded can be acquired just as fast as targets that expand just as the user reaches them
• Implications of results can be applied towards the design of UI widgets for devices with limited viewing space
29
10/26/2006 74.792 - Graduate Course in HCI 113
Improvement to Fitts’: Bubble Cursor
TO READ:• Tovi Grossman, Ravin Balakrishnan. The Bubble
Cursor: Enhancing target acquisition by dynamic resizing of the cursor’s activation area, ACM CHI, 2005, p. 281-290.
10/26/2006 74.792 - Graduate Course in HCI 114
Bubble Cursor
10/26/2006 74.792 - Graduate Course in HCI 115
Bubble Cursor
• Improvements by– Decreasing A
• Drag-and-pop [Baudisch et al.]• Object pointing [Guiard et al.]
– Increasing W• Area cursor [Kabbash & Buxton]• Enhanced area cursor [Worden at al]• Expanding targets [McGuffin & Balakrishnan]
– Decreasing A and Increasing W• Semantic pointing [Blanch et al]
10/26/2006 74.792 - Graduate Course in HCI 116
Design of Bubble Cursor
Modification to area cursor
Problem! Circular cursor resolves problem
30
10/26/2006 74.792 - Graduate Course in HCI 117
Design of Bubble Cursor
Size problem
10/26/2006 74.792 - Graduate Course in HCI 118
Design of Bubble Cursor
10/26/2006 74.792 - Graduate Course in HCI 119
Design of Bubble Cursor
10/26/2006 74.792 - Graduate Course in HCI 120
Design of Bubble Cursor
31
10/26/2006 74.792 - Graduate Course in HCI 121
Design of Bubble Cursor
10/26/2006 74.792 - Graduate Course in HCI 122
Results
10/26/2006 74.792 - Graduate Course in HCI 123
Results
10/26/2006 74.792 - Graduate Course in HCI 124
Experiment 2
32
10/26/2006 74.792 - Graduate Course in HCI 125
Results
Video
10/26/2006 74.792 - Graduate Course in HCI 126
Models for Trajectory-Based HCI Tasks
• Trajectory tasks are becoming more common– Navigating through nested menus– Drawing curves– Moving in 3D worlds
• Cannot be successfully modeled using Fitts’ law
• “Steering through tunnel” as paradigm to represent trajectory-based tasks
“Beyond Fitts’ Law: Models for trajectory based HCI tasks.”Proceedings of ACM CHI 1997 Conference
10/26/2006 74.792 - Graduate Course in HCI 127
Beyond pointing: Trajectory based tasks
10/26/2006 74.792 - Graduate Course in HCI 128
Beyond pointing: Trajectory based tasks
• Experimental paradigm focused on is steering between boundaries (constrained motion)
• It appears that the time to produce trajectories sets the relative speed-accuracy ratio: the larger the amplitude, the less precise the result is.
• Want to derive and validate quantitative relationships between completion time and movement constraints in trajectory-based tasks
33
10/26/2006 74.792 - Graduate Course in HCI 129
Beyond pointing: Trajectory based tasks
• EXPERIMENT 1: GOAL PASSING
• A steering task with constraints only at the ends of the movement
• Result: goal passing task follows same law as in Fitts’ tapping task
10/26/2006 74.792 - Graduate Course in HCI 130
Beyond pointing: Trajectory based tasks
• EXPERIMENT 2: INCREASING CONSTRAINTS
• What happens if you place more goals along the trajectory?– Allows to formulate a hypothetical relationship of the steering
task
• Result: model successful in describing the difficulty of the task
10/26/2006 74.792 - Graduate Course in HCI 131
Beyond pointing: Trajectory based tasks2 goals passing
)1(log2 +=WDID
D
W
3 goals passing)1
2(log2 2 +=
WDID
D/2 D/2
N+1 goals passing)1(log2 +=
NWDNID
D/N D/N D/N
∞ goals passing?
WDID =∞
D
W
10/26/2006 74.792 - Graduate Course in HCI 132
Steering Law
Fixed width tunnel
WDbaMT
WDID +== ,
D
W
Narrowing tunnel
dx
W2W1 W(x)∫=
D
xWdxID
0 )(
General Steering Law
∫= c sWdsID
)(
W(s)ds
c
ID = D/(W2-W1)*ln(W2/W1)
34
10/26/2006 74.792 - Graduate Course in HCI 133
Design Implications
• Interacting with current GUIs, one often implicitly performs various path steering tasks
• i.e. menu selection
• Each step in menu selection is a linear path steering task, similar to the one in Experiment 2
10/26/2006 74.792 - Graduate Course in HCI 134
• Pointing is most universal and best adapted interaction paradigm in human computer interfaces
• However some disadvantages exist:– time-consuming if object pointed to is small– pointing-driven widgets consume screen real estate– double-clicking is not trivial for novice users, due to rapid succession of clicks (temporal dependence)
Crossing Based Interfaces: Motivation
“More than dotting the i's --- foundations for crossing-based interfaces.” Johnny Accot , Shumin Zhai. Proceedings of the SIGCHI conference on Human factors in computing systems. April 2002
10/26/2006 74.792 - Graduate Course in HCI 135
• Macintosh menu bar affords orthogonal selection since height is infinite
• Windows provides collinear selection
Crossing based interfaces
10/26/2006 74.792 - Graduate Course in HCI 136
Crossing interfaces: example designs
35
10/26/2006 74.792 - Graduate Course in HCI 137
Crossing interfaces: example designs
10/26/2006 74.792 - Graduate Course in HCI 138
Crossing interfaces: example designs
10/26/2006 74.792 - Graduate Course in HCI 139
Crossing interfaces: example designs
10/26/2006 74.792 - Graduate Course in HCI 140
Guiard’s Model
• Originated from the area of motor behavior referred to as bimanual control or laterality
• Both hands perform a different set of tasks
• Given this knowledge and handedness of people, interesting to evaluate how interaction accommodates best the division
• Results in descriptive model of bimanual skill, given by Guiard in 1987 paper
Guiard, Y. (1987). Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of Motor Behavior, 19, 486-517.
36
10/26/2006 74.792 - Graduate Course in HCI 141
Guiard’s Model
• follows the non-preferred hand • works within established frame of reference set by the non-preferred hand • performs fine movements
Preferred
• leads the preferred hand • sets the spatial frame of reference for the preferred hand • performs coarse movements
Non-preferred
Role and Action Hand
10/26/2006 74.792 - Graduate Course in HCI 142
Guiard’s Model
• Example:– a right-handed artist sketches a design of car
– acquires a template with left hand (non-preferred hand leads)
– template is manipulated over the workspace (coarse movement, sets the frame of reference)
– Right hand picks stylus (preferred hand follows) and placed close to the template (works within frame of reference set by the non-preferred hand)
– Artist sketches (preferred hand makes precise movements)
10/26/2006 74.792 - Graduate Course in HCI 143
Guiard’s Model
• People naturally gravitate to using two hands
• Performance times are reduced
• Can be used for interfaces that employ:– Drawing designs– Fabricating virtual objects– Positioning– Reshaping
10/26/2006 74.792 - Graduate Course in HCI 144
Bimanual Control and Desktop Computer Affordances
• How does the distribution of keys on a keyboard facilitate task division between right/left hands?
– where does interaction with the mouse fit into the model?
• Right side bias toward power keys (executive keys + modifier keys, marked in red dots)
37
10/26/2006 74.792 - Graduate Course in HCI 145
Bimanual Control and Desktop Computer Affordances
• Dominance on right hand side, good for the 80s but how does this work with GUIs and pointing devices that are now commonplace?
• Right handed have to reach over with left or leave the mouse
• Is there an advantage for left-handed users?
10/26/2006 74.792 - Graduate Course in HCI 146
Bimanual Control and Desktop Computer Affordances
Right hand — press ENTER (Note: avoids error prone double-click operation)
Left hand — manipulate pointer with mouse and single click on icon
Open a file, open a folder, or launch a program
Left hand — manipulate pointer with mouse and select link by clicking on it
Right hand — navigate to link via PAGE UP and/or PAGE DOWN keys
Click on a link in a browser
Right hand — press ENTER (Note: OK button is the default)
Left hand — manipulate pointer with mouse and click on an option
Select an option in a window
Right hand — press DELETE (probably with little finger)
Left hand — manipulate pointer with mouse and select text/object by double clicking or dragging
Delete
Trailing/OverlappingMovement Leading Movement Task
Common tasks performed by a left-handed user manipulating mouse in the left hand
10/26/2006 74.792 - Graduate Course in HCI 147
Bimanual Control and Desktop Computer Affordances
• Tasks described previously are faster for left-handed users than right-handed users
• When pointing is juxtaposed with power key activation (excluding SHIFT, ALT, & CONTROL), the desktop interface presents a left-hand bias
10/26/2006 74.792 - Graduate Course in HCI 148
Bimanual Control and Desktop Computer Affordances
• Scrolling typically accomplished by dragging “elevator” of scrollbar along the right-hand side of an application’s window
•Takes up to 2 secs per trial and is obtrusive and non-transparent
• In perspective of Guiard’s model of bimanual control, we can delegate scrolling to non dominant hand
• follows/overlaps scrolling • works within frame of reference set by scrolling • demands precision (fine)
Selecting, editing, reading,drawing, etc.
• precedes/overlaps other tasks • sets the frame of reference • minimal precision needed (coarse)
Scrolling
Characteristics Task
38
10/26/2006 74.792 - Graduate Course in HCI 149
Redesigning the Scrolling Interface
10/26/2006 74.792 - Graduate Course in HCI 150
Papers to Read
• Please read the list of papers given in the following slides.
• The videos are available online
10/26/2006 74.792 - Graduate Course in HCI 151
Interactive TechniquesLarge Displays
1) Shahzad Malik, Abhishek Ranjan, Ravin Balakrishnan. (2005). Interacting with large displays from a distance with vision-tracked multi-finger gestural input. Proceedings of UIST 2005 - the ACM Symposium on User Interface Software and Technology. p. 43-52. (uist2005_videohand.mov)
2) Clifton Forlines, Ravin Balakrishnan, Paul Beardsley, Jeroen van Baar, Ramesh Raskar. (2005). Zoom-and-Pick: Facilitating visual zooming and precision pointing with interactive handheld projectors. Proceedings of UIST 2005 - the ACM Symposium on User Interface Software and Technology. p. 73-82. (uist2005_zoomandpick.mov)
3) Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P. Bederson, B., and Zierlinger, A. Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-operated Systems. In Proceedings of Interact 2003, Zurich Switzerland, August 2003, pp. 57-64. (2004-Baudisch-CHI04-DragAndPop.mpeg)
4) Anastasia Bezerianos, Ravin Balakrishnan. (2005). The Vacuum: Facilitating the manipulation of distant objects. Proceedings of CHI 2005 – the ACM Conference on Human Factors in Computing Systems. p. 361-370. (chi2005_vacuum.mov)
5) Xiang Cao, Ravin Balakrishnan. (in press, 2006). Interacting with dynamically defined information spaces using a handheld projector and a pen. To appear in Proceedings of UIST 2006 – the ACM Symposium on User Interface Software and Technology. (uist2006_handheldprojector.mov)
6) Baudisch, P., Cutrell, E., and Robertson, G. High-Density Cursor: A Visualization Technique that Helps Users Keep Track of Fast-Moving Mouse Cursors. In Proceedings of Interact 2003, Zurich Switzerland, August 2003, pp. 236-243. (2003-Baudisch-Interact03-HighDensityCursor.wmv)
10/26/2006 74.792 - Graduate Course in HCI 152
Interactive TechniquesSmall Displays
1) Baudisch, P. and Rosenholtz, R. Halo: A Technique for Visualizing Off-Screen Locations. In Proceedings of CHI 2003, Fort Lauderdale, FL, April 2003,pp. 481-488. (2003-Baudisch-CHI03-Halo.wmv)
2) Mackinlay, J., Good, L., Zellweger, P., Stefik, M., and Baudisch, P. City Lights: Contextual Views in Minimal Space. In Proceedings of CHI 2003 (Short paper), Fort Lauderdale, FL, April 2003,pp 838-839. (2003-Good-Citylights.mpeg)
3) Lam, H. and Baudisch, P. Summary Thumbnails: Readable Overviews for Small Screen Web Browsers. In Proceedings of CHI 2005, Portland, OR, Apr 2005, pp. 681-690. (2005-Baudisch-CHI05-SummaryThumbnails.mov)
4) Irani, P, Gutwin, C. Yang, X-D, Improving selection of off-screen targets with hopping. CHI 2006: 299-308