Tuomas Husu
description
Transcript of Tuomas Husu
Tuomas Husu / Department of Computer Science 1www.helsinki.fi/yliopisto
The Usability of KONE DCS: How (and why) the characteristics of the group affect
the use of the system
Tuomas Husu
29.6.2010
www.helsinki.fi/university 2Tuomas Husu / Department of Computer Science 29.6.2010
Research problem
www.helsinki.fi/university 3Tuomas Husu / Department of Computer Science 29.6.2010
What we did?
www.helsinki.fi/university 4Tuomas Husu / Department of Computer Science
A. Group size• 2 persons• 4 persons
B. Degree of acquaintance• “Friends”• “Strangers”
C. Destination• Same• Different
29.6.2010
2 x 2 x 2 Factorial design
www.helsinki.fi/university 5Tuomas Husu / Department of Computer Science
No. Combination Size Familiarity Destination1 A1 B1 C1 2 friends same
2 A1 B1 C2 2 friends different
3 A1 B2 C1 2 strangers same
4 A1 B2 C2 2 strangers different
5 A2 B1 C1 4 friends same
6 A2 B1 C2 4 friends different
7 A2 B2 C1 4 strangers same
8 A2 B2 C2 4 strangers different
29.6.2010
2 x 2 x 2 Factorial design:Variable combinations
www.helsinki.fi/university 6Tuomas Husu / Department of Computer Science 29.6.2010
Video 1
www.helsinki.fi/university 7Tuomas Husu / Department of Computer Science 29.6.2010
Measures and covariates
• Used left DOP• Problems: ID twice
• T to DOP 3 s• T at DOP 4 s
• Total time 20 s• Elevator B
www.helsinki.fi/university 8Tuomas Husu / Department of Computer Science 29.6.2010
Measures and covariates
• Used left DOP• Problems: ID twice
• T to DOP 3 s• T at DOP 4 s
• Total time 20 s• Elevator B
• Used right DOP• Problems: no• T to DOP 2 s• T at DOP 7 s
• Total time 22 s• Elevator B
www.helsinki.fi/university 9Tuomas Husu / Department of Computer Science 29.6.2010
Measures and covariates
• Used left DOP• Problems: ID twice
• T to DOP 3 s• T at DOP 4 s
• Total time 20 s• Elevator B
• Used right DOP• Problems: no• T to DOP 2 s• T at DOP 7 s
• Total time 22 s• Elevator B
• Did not use DOP• Total time 16 s
• Elevator B
www.helsinki.fi/university 10Tuomas Husu / Department of Computer Science 29.6.2010
Measures and covariates
• Used left DOP• Problems: ID twice
• T to DOP 3 s• T at DOP 4 s
• Total time 20 s• Elevator B
• Used right DOP• Problems: no• T to DOP 2 s• T at DOP 7 s
• Total time 22 s• Elevator B
• Did not use DOP• Total time 16 s
• Elevator B
• Used left DOP• Problems: no• T to DOP 5 s• T at DOP 3 s• Total time 15 s
• Elevator B
www.helsinki.fi/university 11Tuomas Husu / Department of Computer Science 29.6.2010
Measures and covariates
Participant
Destination
Used DOP
Which DOP
Succeeded
Optimal
Elevator car
Missed the car
Was in time
Traffic jam
T to DOP
T at DOP
Elevator wait
Total time
12 8 T R T T B F T F 2 7 1 22
5 8 T L F F B F T T 3 4 0 20
4 8 F B F T F 16
13 8 T L T T B F T F 5 3 0 15
www.helsinki.fi/university 12Tuomas Husu / Department of Computer Science
• Before and after
29.6.2010
System usability scale (SUS)
… …
www.helsinki.fi/university 13Tuomas Husu / Department of Computer Science
• Focus group interview afterwards• Impressions about the system• Objective of the system• DOP: functions and ease of use• Problems• Et cetera
29.6.2010
Interview
www.helsinki.fi/university 14Tuomas Husu / Department of Computer Science 29.6.2010
Elevator Quiz!
www.helsinki.fi/university 15Tuomas Husu / Department of Computer Science
• On average: 22,4 seconds
29.6.2010
Results: Total time
www.helsinki.fi/university 16Tuomas Husu / Department of Computer Science 29.6.2010
Results: Total time
2 persons 4 persons18
19
20
21
22
23
24
20.344
23.391
Group size
Seco
nds
www.helsinki.fi/university 17Tuomas Husu / Department of Computer Science
Friends Strangers21.4
21.6
21.8
22
22.2
22.4
22.6
22.8
23
21.896
22.854
Degree of acquaintance
Seco
nds
29.6.2010
Results: Total time
www.helsinki.fi/university 18Tuomas Husu / Department of Computer Science
Same dest. Diff. dest20
20.5
21
21.5
22
22.5
23
23.5
24
21.198
23.552
Destination
Seco
nds
29.6.2010
Results: Total time
www.helsinki.fi/university 19Tuomas Husu / Department of Computer Science
Interaction effect (total time)
29.6.2010
Results: Total time
2 persons 4 persons0
5
10
15
20
25
30
Same destination
Tim
e (s
econ
ds)
2 persons 4 persons0
5
10
15
20
25
30
Different destinationTi
me
(sec
onds
)
FriendsStrangers
www.helsinki.fi/university 20Tuomas Husu / Department of Computer Science
• On average: 84% used panel
29.6.2010
Results: Did they use the panel?
www.helsinki.fi/university 21Tuomas Husu / Department of Computer Science
2 persons 4 persons40%
50%
60%
70%
80%
90%
100% 89.06%
81.25%
Group size
Shar
e (p
erce
nts)
29.6.2010
Results: Did they use the panel?
www.helsinki.fi/university 22Tuomas Husu / Department of Computer Science
Friends Strangers40%
50%
60%
70%
80%
90%
100%
76.04%
91.67%
Degree of acquaintance
Shar
e (p
erce
nts)
29.6.2010
Results: Did they use the panel?
www.helsinki.fi/university 23Tuomas Husu / Department of Computer Science
Same dest. Diff. dest.40%
50%
60%
70%
80%
90%
100%
71.88%
95.83%
Destination
Shar
e (p
erce
nts)
29.6.2010
Results: Did they use the panel?
www.helsinki.fi/university 24Tuomas Husu / Department of Computer Science
Interaction effect (share of people who used DOP)
29.6.2010
Results: Did they use the panel?
2 persons 4 persons40%
50%
60%
70%
80%
90%
100%
Same destination
Shar
e (p
erce
nt)
2 persons 4 persons40%
50%
60%
70%
80%
90%
100%
Different destinationSh
are
(per
cent
)
FriendsStrangers
www.helsinki.fi/university 25Tuomas Husu / Department of Computer Science
• On average: 2 persons per elevator
29.6.2010
Results: Persons per elevator
www.helsinki.fi/university 26Tuomas Husu / Department of Computer Science
2 persons 4 persons0
1
2
3
4
0,002
0,003
Group size
Pers
ons
per e
leva
tor
29.6.2010
Results: Persons per elevator
www.helsinki.fi/university 27Tuomas Husu / Department of Computer Science
Friends Strangers0
1
2
3
4
0,002
0,002
Degree of acquaintance
Pers
ons
per e
leva
tor
29.6.2010
Results: Persons per elevator
www.helsinki.fi/university 28Tuomas Husu / Department of Computer Science
Same dest. Diff. dest.0
1
2
3
4
0,003
0,001
Destination
Pers
ons
per e
leva
tor
29.6.2010
Results: Persons per elevator
www.helsinki.fi/university 29Tuomas Husu / Department of Computer Science
Interaction effect (persons per elevator car)
29.6.2010
Results: Persons per elevator
2 persons 4 persons0
1
2
3
4
Same destination
Pers
ons
per e
leva
tor c
ar
2 persons 4 persons0
1
2
3
4
Different destinationPe
rson
s pe
r ele
vato
r car
FriendsStrangers
www.helsinki.fi/university 30Tuomas Husu / Department of Computer Science
On average: 74% succeeded by first attempt
29.6.2010
Results: DOP success rate
www.helsinki.fi/university 31Tuomas Husu / Department of Computer Science
2 persons 4 persons40%
50%
60%
70%
80%
90%
100%
68.42%
78.85%
Group size
Succ
ess
rate
(per
cent
s)
29.6.2010
Results: DOP success rate
www.helsinki.fi/university 32Tuomas Husu / Department of Computer Science
Friends Strangers40%
50%
60%
70%
80%
90%
100%
80.82%
72.73%
Degree of acquaintance
Succ
ess
rate
(per
cent
s)
29.6.2010
Results: DOP success rate
www.helsinki.fi/university 33Tuomas Husu / Department of Computer Science
Same dest. Diff. dest.40%
50%
60%
70%
80%
90%
100%81.16%
72.83%
Destination
Succ
ess
rate
(per
cent
s)
29.6.2010
Results: DOP success rate
www.helsinki.fi/university 34Tuomas Husu / Department of Computer Science
Interaction effect (DOP call success rate)
29.6.2010
Results: DOP success rate
2 persons 4 persons40%
50%
60%
70%
80%
90%
100%
Same destination
Succ
ess
rate
2 persons 4 persons40%
50%
60%
70%
80%
90%
100%
Different destinationSu
cces
s ra
te
FriendsStrangers
www.helsinki.fi/university 35Tuomas Husu / Department of Computer Science
Changes in success rate during the test
29.6.2010
Results: DOP learnability
1 2 3 4 5 6 7 8 9 10111213141516171819202122232425262728293031320%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Time (sessions)
www.helsinki.fi/university 36Tuomas Husu / Department of Computer Science 29.6.2010
Results
A B C D0
5
10
15
20
25
30
35
40
Using left DOP
Elevator
Shar
e (%
)
A B C D0
5
10
15
20
25
30
35
40
Using right DOP
ElevatorSh
are
(%)
www.helsinki.fi/university 37Tuomas Husu / Department of Computer Science
No. Question Avg Before After Change1 I think that I would like to use this system frequently 4,4 4,4 4,4 ± 0%
2 I found the system unnecessarily complex 1,5 1,5 1,5 ± 0%
3 I thought the system was easy to use 4,2 4,3 4,2 − 2%
4 I think that I would need the support of a technical person to be able to use this system
1,5 1,6 1,4 − 10%
5 I found the various functions in this system were well integrated 3,9 4,1 3,6 − 12%
6 I thought there was too much inconsistency in this system 1,9 1,9 1,9 + 2%
7 I would imagine that most people would learn to use this system very quickly
4,2 4,1 4,3 + 4%
8 I found the system very cumbersome to use 1,7 1,6 1,8 + 10%
9 I felt very confident using the system 4,1 4,1 4,1 ± 0%
10 I needed to learn a lot of things before I could get going with this system
1,3 1,3 1,4 + 7%
29.6.2010
Results: SUS
www.helsinki.fi/university 38Tuomas Husu / Department of Computer Science
No. Question Avg Before After Change1 I think that I would like to use this system frequently 4,4 4,4 4,4 ± 0%
2 I found the system unnecessarily complex 1,5 1,5 1,5 ± 0%
3 I thought the system was easy to use 4,2 4,3 4,2 − 2%
4 I think that I would need the support of a technical person to be able to use this system
1,5 1,6 1,4 − 10%
5 I found the various functions in this system were well integrated 3,9 4,1 3,6 − 12%
6 I thought there was too much inconsistency in this system 1,9 1,9 1,9 + 2%
7 I would imagine that most people would learn to use this system very quickly
4,2 4,1 4,3 + 4%
8 I found the system very cumbersome to use 1,7 1,6 1,8 + 10%
9 I felt very confident using the system 4,1 4,1 4,1 ± 0%
10 I needed to learn a lot of things before I could get going with this system
1,3 1,3 1,4 + 7%
29.6.2010
Results: SUS
www.helsinki.fi/university 39Tuomas Husu / Department of Computer Science
• Premises, views and elevator interiors were awesome• Elevators moved fast and smoothly• The more people in the hall, the more difficult it was• Elevator hall was too narrow• DOP was difficult to use (buttons etc)• It was difficult to distinguish elevators• No clue what ★ and − buttons were (DOP)
29.6.2010
Results: Interviews
www.helsinki.fi/university 40Tuomas Husu / Department of Computer Science
• Things become complicated if something goes wrong with DOP call• DOP time is ~4 seconds if everything is ok, but easily 10-30 seconds when problems occur• (Quite often something goes wrong)• Other people gets away with elevators while others are struggling with DOP’s
29.6.2010
Conclusion
www.helsinki.fi/university 41Tuomas Husu / Department of Computer Science
• They used DOP more loyally than we expected (over 80%)…
29.6.2010
Conclusions: What else?
www.helsinki.fi/university 42Tuomas Husu / Department of Computer Science 29.6.2010
Video 2
www.helsinki.fi/university 43Tuomas Husu / Department of Computer Science
• They used DOP more loyally than we expected (over 80%)• DOPs were surprisingly difficult to use…
• 77% succeeded at once• 62% succeeded at once without errors
29.6.2010
Conclusions: What else?
www.helsinki.fi/university 44Tuomas Husu / Department of Computer Science 29.6.2010
Video 3
www.helsinki.fi/university 45Tuomas Husu / Department of Computer Science
• They used DOP more loyally than we expected (over 80%)• DOPs were surprisingly difficult to use
• 77% succeeded at once• 62% succeeded at once without errors
• Elevators were not easily distinguishable…
29.6.2010
Conclusions: What else?
www.helsinki.fi/university 46Tuomas Husu / Department of Computer Science 29.6.2010
Video 4
www.helsinki.fi/university 47Tuomas Husu / Department of Computer Science
• They used DOP more loyally than we expected (over 80%)• DOPs were surprisingly difficult to use
• 77% succeeded at once• 62% succeeded at once without errors
• Elevators were not easily distinguishable• Because of narrow corridor and placement of DOP’s the movement was not smooth…
29.6.2010
Conclusions: What else?
www.helsinki.fi/university 48Tuomas Husu / Department of Computer Science 29.6.2010
Video 5
www.helsinki.fi/university 49Tuomas Husu / Department of Computer Science
• They used DOP more loyally than we expected (over 80%)• DOPs were surprisingly difficult to use
• 77% succeeded at once• 62% succeeded at once without errors
• Elevators were not easily distinguishable• Because of narrow corridor and placement of DOP’s the movement was not smooth
29.6.2010
Conclusions: What else?
www.helsinki.fi/university 50Tuomas Husu / Department of Computer Science 29.6.2010
Design implications
www.helsinki.fi/university 51Tuomas Husu / Department of Computer Science 29.6.2010
Design implications:“Premises and people flow”
www.helsinki.fi/university 52Tuomas Husu / Department of Computer Science 29.6.2010
Design implications:“Premises and people flow”
My destination is 15th floor!
D
Ok! Where is elevator D? Oh,
there!
www.helsinki.fi/university 53Tuomas Husu / Department of Computer Science 29.6.2010
Design implications:“Premises and people flow”
www.helsinki.fi/university 54Tuomas Husu / Department of Computer Science 29.6.2010
Design implications:“DOP’s elevator indication”
www.helsinki.fi/university 55Tuomas Husu / Department of Computer Science 29.6.2010
Design implications:“DOP’s special functions”
www.helsinki.fi/university 56Tuomas Husu / Department of Computer Science 29.6.2010
Design implications
www.helsinki.fi/university 57Tuomas Husu / Department of Computer Science
• Delay (authentication)• Button sensitivity• Visual feedback (buttons)• Response time• Timeout delay
29.6.2010
Design implications:“DOP’s technical polishing”
www.helsinki.fi/university 58Tuomas Husu / Department of Computer Science 29.6.2010
Discussion