New Interaction Techniques Department of Computer Sciences University of Tampere, Finland September...
-
Upload
hillary-sybil-west -
Category
Documents
-
view
215 -
download
1
Transcript of New Interaction Techniques Department of Computer Sciences University of Tampere, Finland September...
New Interaction Techniques
Department of Computer SciencesUniversity of Tampere, Finland
Department of Computer SciencesUniversity of Tampere, Finland
September – December, 2003
Grigori Evreinov
www.cs.uta.fi/~grse/
Dwell Time Selection & more…
TAUCHI MMIG G. Evreinov 01_26 10.10.2003
Cursor Object Inter-actions
Dwell Time
Time
X
Y
Time
demo1
[1]
TAUCHI MMIG G. Evreinov 02_26 10.10.2003
MagicCursor 2000: Dwell Selection Clicking Solution, http://www.gstsdesigns.com/madentec/MagicCursor.htm
Gus! Dwell Cursor, http://www.gusinc.com/dwell.html
The Smart-Nav AT package (Dwell clicking 0.2-2.0 s), http://www.naturalpoint.com/prod/d_dwell.htm
Dwell Time
TAUCHI MMIG G. Evreinov 03_26 10.10.2003
Gaze Control
[1] http://www.metrovision.fr/mv-vi-notice-us.html[2] http://www.delta7.asso.fr/Deltavision%202001/ecrire3.html
Dwell Time
TAUCHI MMIG G. Evreinov 04_26 10.10.2003
[31] http://www.metrovision.fr/mv-vi-notice-us.html
LC Technologies, Inc.
The Eye-Gaze System Makers
Dwell Time
TAUCHI MMIG G. Evreinov 05_26 10.10.2003
Dwell Time
[9] Myers, B.A., et al. Interacting at a Distance: Measuring the Performance of Laser Pointers and Other Devices. CHI 2002.
[12] Olsen Jr., D.R. and Nielsen, T. Laser Pointer Interaction. CHI 2001.
Laser Control
TAUCHI MMIG G. Evreinov 06_26 10.10.2003
Dwell Time
Natural Point trackIRTM
http://www.naturalpoint.com ,
http://www.headtracker.com/
Marsden, R. 100% Hands-free Computer Access- Madentec’s 2000 Serieshttp://www.madentec.com/products/comaccess/2000/whitepaper.html
http://www.csun.edu/cod/conf/2000/proceedings/0185Marsden.htm
demo2
demo3
Head Control
TAUCHI MMIG G. Evreinov 07_26 10.10.2003
Usability-testing software for… txtEntry Eye-
Gaze
Dwell Time
Clear Data
TAUCHI MMIG G. Evreinov 08_26 10.10.2003
TWords.txt /phrases
GridData1:test words/chars
entered text /charstime per char, ms
GridData2:
char per word /phrasenum. of entered words
num. of strokes /clicks per word /phrase
time per word/phrase, s
lblTestSymbol
txtText1
On-screen Keyboard
Break test
Test initialization
Ctrl+K=>move keys
SetSigns
SetCharacters
BackSp
Statistics()
Rtime, mss (st.dev), ms
Errorswpm
TestTime, s
lblSave_Click()
GridData2_Click()to save column
GridData1_Click() to save column
txtPersonData comments…
fraDataWordsLoading
SetTestlblOpen_Click()
SetData
Trial start
Timer1
Timer2
Lesson6 txtEntry_EyeGaze3
SpotActivate
lblDwDelay
lblDwell
Ctrl+L=>EditCapts.
Up/Down
Timer3 Timer4
Ctrl+W=>Dwell
MouseInput SymbDec
HideBorders
Dwell Time
Clear Data
TAUCHI MMIG G. Evreinov 09_26 10.10.2003
lblTestSymbol
txtText1
On-screen Keyboard
Break test
Test initialization
Ctrl+K=>move keys
SetSigns
SetCharacters
fraData
SetData
Trial start
Lesson6 txtEntry_EyeGaze3
SpotActivate
lblDwDelay
lblDwell
Ctrl+L=>EditCapts.
Up/Down
tmrDwDelay tmrDwell
Ctrl+W=>Dwell
MouseInput SymbDec
HideBorders
Dwell Time
TAUCHI MMIG G. Evreinov 10_26 10.10.2003
Dwell TimeAdaptive
Dwellingaccording to Windows Interface Design Guide*, the action associated with
the control is initiated when the mouse button is released
if the pointer is dragged to another location, the control will return to its
original state and undesirable click will be stopped
for instance, the user can put cursor over icon or command button and
choose it by clicking the mouse
sometimes, clicks are undesirable or impossible (laser pointer, gaze / head
/ ‘brain’ control…)
to provide productivity and easy access during communication with
computer it is necessary a balance between flexibility of the dialogue
structure, adaptability on each level of the interface, cognitive abilities of
the person and limitations used interaction techniques
semantic, syntactic, lexical* The Windows Interface. An Application Design Guide. Microsoft Press, Redmond, Washington, 1992.
TAUCHI MMIG G. Evreinov 11_26 10.10.2003
Dwell Timethe time is one of the most important critical parameters of the system
feedback loop
a normal time of simple visual-motor control task in accomplishing with the
help of intact muscles of the finger is about 150-250 ms
the time is progressively increased up to 1000 ms and more, if an
additional semantic analysis or physical limitations take place
to prevent wrong selection external (dwell) timer, mental and motor
activities of the user should strictly be coordinated
the feedback cues (highlighted areas) could play a role of strobe-signals to
stimulate user behavior and to support, to stabilize temporal framework for
rhythmically-alternating cognitive and motor activities
if the system could individually and dynamically turn, dwell mode could be
more flexible
TAUCHI MMIG G. Evreinov 12_26 10.10.2003
Dwell Timehow we could know what temporal window satisfies of user requirements
and when it should be changed?
menu pointing could be considered as temporal process or stimulus-
dependent goal-directed behavior of the user; herewith, a behavior model
includes a sequence of actions both on the user side – cognitive
processing and motor actions, and procedures are within interface –
highlighting or other verification about the state of a particular alternative
the physical parameters of feedback signals may either facilitate
synchronization of the interactive process or hinder performance
in both cases, physical stimulus starts the motor reaction that could be
measured
based on real-time analysis the visual-motor reaction time, we could
predict or optimize dwell interval*.
* Bourhis, G., Pino, P. Temporal analysis of the use of an augmentative communication device. AAATE’03. IOS Press, Netherlands (2003), 294-298.
Tthreshold
selectionhighlighting
the menu item
TAUCHI MMIG G. Evreinov 13_26 10.10.2003
Dwell Time
the temporal diagram of the algorithm for measuring the user performance
through visual-motor reaction time and correction of the dwell interval
T0 – the first variable interval, T1 – the second variable interval and T2 = T1
Tthresh. - dynamical threshold Tthresh. = T0 + T1
Time
Tdwell
RTi
T0 T2T1
selectionselection
TAUCHI MMIG G. Evreinov 14_26 10.10.2003
Dwell Time
after each highlighting the menu item, we can record a time of user reply in
a kind of selection time
besides that, we can count an average magnitude for AveRT on each 5
realizations
now, if (AveRT < Tthresh. – dT) then we may decrease T0 on dT too
that is, a new scan interval will equal
Tdwell = (T0 + 2 T1) – dT or Tdwell = (T0 – dT) + 2 T1
demo4
TAUCHI MMIG G. Evreinov 15_26 10.10.2003
Dwell Timechanging scan interval can occurred with equal probability in plus and
minus, if user reactions are symmetrical regarding dynamical threshold
Tthresh.
if user changed typing speed, each user replies
AveRT < Tthresh. and Tdwell
are permanently decreased due to decreasing T0, therefore the time to
change Tthresh.
in a kind of criterion could be index of some number of dT
for instance, if T0 have changed on ((-3) dT), the magnitude of Tthresh.
may also be decreased, at least, on one half of this magnitude
then, a new scan interval will equal
Tdwell = (T0 – 3 dT) + 2 (T1 – 3 dT/2)
after changing dynamical threshold, probability of user replies in a field T2
will be higher and AveRT > Tthresh.
this situation will lead, or not, to increasing T0
TAUCHI MMIG G. Evreinov 16_26 10.10.2003
Dwell Time
Usability-testing software for… Adaptive
dwelling
TAUCHI MMIG G. Evreinov 17_26 10.10.2003
Dwell Time
Shape1 Shape2
Shape9 Shape10
MenuSave
Open
ShapePos
Data
Timer1
lblGrid1
fraData
Timer2
lblDwell
lblThresh
lblDelta
lblWait
txtTest comments…
chkColor
chkBorder
chkSound
chkTactile
lblNAve
chkAve
chkSAve
lblIndexC
chkCThresh
T0 changing T1 changing
lblGrid2, 3 Clear
lblDrawGraphlblError chkCursor chkSpots
picGraph1
picGraph2
TAUCHI MMIG G. Evreinov 18_26 10.10.2003
Dwell TimebPos = False
kThresh = CInt(t2 - t1)
t1 = GetTickCountTimer1.Enabled = True
Timer1.Interval = tDwell
Yes
No
Yes
Exit
Private Sub Form_MouseMove(…)
Xtmp1 = CInt(X) Ytmp1 = CInt(Y)
cursor is inside Shape1
bShape1 = True
NoYes
bThresh = True
Yest2 = GetTickCount
t1 > 0
Yes
reply's time or (T0 + T1)
Shape1.BorderColor = &H80&
Change Border Color =1
Yes
bShape1 = True
bShape2 = True
bShape2 = False reset BorderColor
reset FillColor
Yes
Timer1.Enabled = 0, Int. = 0Timer2.Enabled = 0, Int. = 0ImmWebControl1.StopEffect
bShape1 = Falsereset BorderColor, reset FillColor
If bThresh = True ThenChangeDwell
If bThresh = True ThenChangeDwell
chkSAve.Value = 0 And chkAve.Value = 0
TAUCHI MMIG G. Evreinov 19_26 10.10.2003
Dwell Time
Private Sub ChangeDwell()
kThresh > 100 or tDwell/2
Yes
No
ChangeDwell
ArrThresh(nClick) = tThresh ArrCThresh(nClick) = tThresh ArrScan(nClick) = tDwell t1 = 0: t2 = 0
ArrThresh(nClick) = kThresh
nClick < (nAve + 1)
Yes
No
ArrDTmp(nClick) = kThresh
aveThresh = tThresh
Shift of array and count of the sliding aveThresh
aveThresh < min lower limit = tDelta + 20
Yes
NoaveThresh = min
aveThresh < tThresh - tDelta
aveThresh > tThresh + tDelta
chkSAve.Value = 1
chkAve.Value = 1
aveThresh = tThresh
Line1.BorderColor = &HFF00FF
Line1.Y2 = picGraph1.ScaleHeight - tThresh Line1.Y1 = Line1.Y2
TAUCHI MMIG G. Evreinov 20_26 10.10.2003
Dwell Time
aveThresh < tThresh - tDelta aveThresh > tThresh + tDelta
tDwell = tDwell - tDelta
sDelta = sDelta - tDelta
ArrScan(nClick) = tScan
tDwell = tDwell + tDelta
sDelta = sDelta + tDelta
sDelta > tIndex tDelta sDelta < (-1) tIndex tDelta
new current value
tThresh = tThresh + CInt(sDelta / 2)
sDelta = 0
tThresh = tThresh + CInt(sDelta / 2)
sDelta = 0
tThresh < 50 tThresh = 50
lower limit
ArrCThresh(nClick) = tThresh new current value
Private Sub ChangeDwell()
tDwell < 50
tDwell = 50lower limit
TAUCHI MMIG G. Evreinov 21_26 10.10.2003
Dwell Time
after some kind of calibration the proposed algorithm will automatically
keep dwell interval near convenient magnitude with given speed or
increment dT
adaptive temporal interval could be useful for many applications, which
require of the periodic time correction in dependence on user performance,
for instance, in the systems (head / eye / finger tracking) using dwell time
to simulate mouse actions etc.
TAUCHI MMIG G. Evreinov 22_26 10.10.2003
Dwell Time
T-Adaptive Unit
form transfers buttons' events to the TextBox therefore you can use KeyPreview or
directly txtControl TextBox
TAUCHI MMIG G. Evreinov 23_26 10.10.2003
Dwell Time
Private Sub ChangeScan()
rTime > 100 or tScan/2
Yes
No
ChangeScan
ArrThresh(nClick) = tThresh ArrCThresh(nClick) = tThresh ArrScan(nClick) = tScan t1 = 0: t2 = 0
ArrThresh(nClick) = rTime - tScan
nClick < (nAve + 1)
Yes
No
ArrDTmp(nClick) = rTime - tScan
aveThresh = tThresh
Shift of array and count of the sliding aveThresh
aveThresh < min lower limit
Yes
NoaveThresh = min
aveThresh < tThresh - tDelta
aveThresh > tThresh + tDelta
T0
TAUCHI MMIG G. Evreinov p 24_26 10.10.2003
Dwell Time
aveThresh < tThresh - tDelta aveThresh > tThresh + tDelta
tScan = tScan - tDelta
sDelta = sDelta - tDelta
ArrScan(nClick) = tScan
tScan = tScan + tDelta
sDelta = sDelta + tDelta
sDelta > tIndex tDelta sDelta < (-1) tIndex tDelta
new current value
tThresh = tThresh + CInt(sDelta / 2)
sDelta = 0
tThresh = tThresh + CInt(sDelta / 2)
sDelta = 0
tThresh < 50 tThresh = 50
lower limit
ArrCThresh(nClick) = tThresh new current value
Private Sub ChangeScan()
tScan < 100
tScan = 100lower limit
TAUCHI MMIG G. Evreinov p 25_26 10.10.2003
References[1] Accot, J., Zhai, Sh. More than dotting the i’s — Foundations for crossing-based interfaces, CHI 2002,
April 20-25, 2002, Minneapolis, Minnesota, USA. / AccotZhai2002.pdf
[2] Zhai, Sh., Morimoto, C., Ihde, S. Manual And Gaze Input Cascaded (MAGIC) Pointing. In Proc. CHI’99:
ACM Conference on Human Factors in Computing Systems. 246-253, Pittsburgh, 15-20 May1999. /
magic.pdf
[3] Jacob, R.J.K. Eye Movement-Based Interaction Techniques and the Elements of Next-Generation,
Non-WIMP User Interfaces, http://www.roetting.de/eyes-tea/history/020919/jacob.html
[4] Jacob, R.J.K. What You Look At Is What You Get: Eye Movement-based Interaction Techniques.
CHI’90. / EyeMovBasedInteraction.pdf
[5] Jacob, R.J.K. Eye Movement-based Human-Computer Interaction Techniques: Toward Non-Command
Interfaces. / EyeMovementBased.pdf
[6] Shell, J.S., Vertegaal, R., Skaburskis, A.W. EyePliances: Attention-Seeking Devices that Respond to
Visual Attention. CHI2003 / EyePliances.pdf
[7] Hyrskykari, A. Gaze Control as an Input Device. / gazecontrol.pdf
[8] Aoki, H., Itoh, K., Sumitomo, N. and Hansen, J.P. Usability of Gaze Interaction Compared to Mouse
and Head-Tracking in Typing Japanese Texts on a Restricted On-Screen Keyboard for Disabled
People. / GazeInteraction_iea2003-aoki.pdf
[9] Myers, B.A., Bhatnagar, R., Nichols, J., Choon Hong Peck, Kong, D., Miller, R. and Long, A.Chr.
Interacting at a Distance: Measuring the Performance of Laser Pointers and Other Devices. CHI 2002,
April 20-25, 2002, Minneapolis, Minnesota, USA. / InteractingAtDistance.pdf
[10] Cheng, K., Pulo, K. Direct Interaction with Large-Scale Display Systems using Infrared Laser Tracking
Devices. Australasian Symposium on Information Visualisation, Adelaide, 2003. /
DirectInteractionLargeScaleDisplays.pdf
Dwell Time
TAUCHI MMIG G. Evreinov p 26_26 10.10.2003
[11] Hwang, F. Partitioning Cursor Movements in “Point and Click” Tasks. ACM 1-58113-630-7/03/0004.
CHI 2003, April 5-10, 2003, Ft. Lauderdale, Florida, USA. / PartitioningCursor Movements.pdf
[12] Olsen Jr., D.R. and Nielsen, T. Laser Pointer Interaction. CHI2001, SIGCHI’01, March 31-April 4,
2001, Seattle, WA, USA. / LaserPointerInteraction.pdf
[13] Steriadis, C.E. and Constantinou, Ph. Designing Human-Computer Interfaces for Quadriplegic People.
ACM Transactions on Computer-Human Interaction, Vol. 10, No. 2, June 2003. /
Interafce_QuadriplegicPeople.pdf
[14] Lankford, C. Effective Eye-gaze Input Into Windows tm. Eye Tracking Research & Applications
Symposium 2000 Palm Beach Gardens, FL, USA. / EffectiveEyeGazeInput.pdf
[15] Bates, R. Have Patience with Your Eye Mouse! Eye-Gaze Interaction with Computers Can Work. pp.
33-37. / EyeMouseGazeInteraction.pdf
[16] Ware, C., Mikaelian, H.H. An Evaluation of an Eye Tracker as a Device for Computer Input. CHI + GI
1987 / EvaluationEyeTracker_Input.pdf
[17] Hinckley, K., Pausch, R., Goblel, J.C. and Kassell, N.F. A Survey of Design Issues in Spatial Input. /
Spatial_InputSurvey.pdf
[18] Edwards, G. A Tool for Creating Eye-aware Applications that Adapt to Changes in User Behavior. /
EyeAwareApplications_Adapt.txt
[19] Youngblut, C., Johnson, R.E. et al. Review of Virtual Environment Interface Technology, Institute for
Defence Analyses, available at http://www.hitl.washington.edu/scivw/scivw-ftp/publications/IDA-pdf/
Dwell Time