Usability modeling and measurement
-
Upload
xbosoft -
Category
Technology
-
view
3.128 -
download
0
description
Transcript of Usability modeling and measurement
![Page 1: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/1.jpg)
Usability Modeling and Measurement
Philip Lew
www.xbosoft.com
![Page 2: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/2.jpg)
Agenda
• Introduction– The importance of usability – Specifics for the web / mobile
• What is Usability?• Usability Modeling and Measurements• Case study• Conclusion• Next Steps
![Page 3: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/3.jpg)
Importance of Usability
![Page 4: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/4.jpg)
• Usability is important especially for web applications (SaaS, and websites) – And now especially mobile applications
• Most prevalent development delivery model today
• Without good usability:– Users will leave the applications– For mobile, if they can’t learn in 30 seconds,
they won’t come back
Why is Usability Important?
![Page 5: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/5.jpg)
Web and Mobile have Changed the Terrain
• Business models have changed– Instead of paying
upfront and ‘owning’ the software
– Pay as you go, pay by subscription
• Behavior and expectations have changed
![Page 6: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/6.jpg)
Many Alternatives• Depending on the
type of application– Users could leave
and go elsewhere– Simply not use your
mobile app (if only providing alternative access to a main application)
![Page 7: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/7.jpg)
Thin Client DeploymentMost Prevalent Development Delivery Model Today
• Mobile and Cloud are singing the same song• Service delivery versus product delivery
![Page 8: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/8.jpg)
What is Usability
In relation to qualityIn relation to user experience
![Page 9: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/9.jpg)
General Thoughts on Usability
• Understandability• Learnability• Operability• Attractiveness• Navigation• Responsiveness-performance• Efficiency
![Page 10: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/10.jpg)
Usability in Quality Standards• How is usability
defined?– Standard
definitions– Many others
who say similar things
• Usability– Component
of quality as listed in many standards
Standard DefinitionISO 9126-1(2000)
The capability of the software product to be understood, learned, used, and attractive to the user, when used under specified conditions.
ISO 9241-11(1998)
The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.
IEEE 600.12(1990)
The ease with which a user can learn to operate, prepare inputs for, and interpret outputs of a system or component.
![Page 11: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/11.jpg)
Seffah [33]
Usability Model Comparisons
![Page 12: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/12.jpg)
Current Research / Usability Standards
Kappel et al., The Discipline of Systematic Development of Web Applications, 2003, John Wiley
and Sons.
![Page 13: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/13.jpg)
Current Research – Usability Standards
Quality
Usability
Satisfaction
ISO9126-1
ISO 9241-11
IEEE 610
IEEE XXXYISO 25010
![Page 14: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/14.jpg)
Bigger Picture Quality From ISO point of view
ISO 25010CMMI
Usa
bilit
y
Usa
bilit
y
![Page 15: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/15.jpg)
Usability as a Key Characteristic of Product Quality
Source: ISO 25010
![Page 16: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/16.jpg)
Source: ISO 25010
What is Usability-Effect of the Software Product
Degree to which specified users can achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.
![Page 17: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/17.jpg)
Source: ISO 25010
What is Usability-EffectIn Actual Usage
• Effectiveness– The degree to which specified users can achieve specified goals with
accuracy and completeness in a specified context of use.• Efficiency
– The degree to which specified users expend appropriate amounts of resources in relation to the effectiveness achieved in a specified context of use.
– NOTE Relevant resources can include time to complete the task, materials, or the financial cost of usage.
• Satisfaction– The degree to which users are satisfied in a specified context of use.
Satisfaction is further subdivided into sub-characteristics:• Likability (cognitive satisfaction)• Pleasure (emotional satisfaction)• Comfort (physical satisfaction)• Trust
![Page 18: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/18.jpg)
Usability in Actual Usage
• User role• Objective• Task• Environment• Domain• …
specified users
specified goals
specified context of use
What else can you think of?
![Page 19: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/19.jpg)
Defining Usability For Your Organization
• Attributes expressed hierarchically• Any number of sub-levels is OK• Achieving the sub-attributes=achieving the high level
attribute -measurable
Quality
Attribute 1
Characteristic 1 Characteristic 2 Characteristic n
Subcharacteristic 1 Subcharacteristic 2 Subcharacteristic n
Attribute 2 Attribute 3 Attribute 1
Usability
![Page 20: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/20.jpg)
Let’s Define UsabilityFrom the Product Viewpoint
Usability
Save Location
Characteristic 1 Characteristic 2 Characteristic n
Subcharacteristic 1 Subcharacteristic 2 Subcharacteristic n
Attribute 1 Attribute 2 Attribute n
UnderstandableNavigation
Control Stability Previous-Next
Home Location Ease of finding
![Page 21: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/21.jpg)
Bigger Picture on Usability
• Can be measured from the design point of view or of the product
• Can be measured ‘in-use’ with real users
Usa
bilit
y
Usa
bilit
y
![Page 22: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/22.jpg)
Defining Usability from an Effect-Real usage Point of View
Quality
Errors
Effectiveness Efficiency Characteristic n
Accuracy Completeness Subcharacteristic n
Attribute 2 Attribute 3 Attribute 1
Usability
Satisfaction
![Page 23: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/23.jpg)
Other Potential Attributes Measurements for ‘in use’ Usability
• Effectiveness– Completion rates– Error rate– Help usage
• Efficiency– Task time– Backtracking
• Learnability– Learning rate– Task time deviation
![Page 24: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/24.jpg)
Developing a CombinedUsability Model
Application Design and Environment
User Behavior and Activity
Measurable and quantifiable factors
Measured ‘in-use’ Usability
![Page 25: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/25.jpg)
External Quality Requirements (for Shopping Cart Entity) 1 Usability 1.1 Understandability 1.1.1 Icon/label ease to be recognized 1.1.2 Information grouping cohesiveness 1.2 Learnability 1.2.1 ……………………………………………………….. 1.3 Operability 1.3.1 Control permanence 1.3.2 Expected behaviour of Controls 2 Content Quality 2.1 Content Suitability 2.1.1 Basic Information Coverage 2.1.1.1 Line item information completeness 2.1.1.2 Product description appropriateness 2.1.2 Coverage of other Contextual Information 2.1.2.1 …………………………………………………………
Example of Product Quality Model with Attributes
(Operability in ISO 25010)
![Page 26: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/26.jpg)
Measurable Attributes
• Attribute name• Description and purpose• How to measure• What is measured• Measurement/Calculation• Range (min, max)• Objective• Current
Once you have a model (what you are going to measure), then you start doing IT!
![Page 27: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/27.jpg)
Usability Measurement
27
Attribute Scale How Measure orCalculation
Objective Current
Help completeness
Percent ofMenu items with help
Comparemenus andhelp items
% 90% 40%
Ease ofaccess
Keystrokes tofind/use a feature/function/information
Sample 50items
% 3 12
Consistency Number locations for same button
Examinemenus anddoc.
integer 1 5
Accuracy Number reportederrors
Collect from log files
Integer or % <5 10
![Page 28: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/28.jpg)
Usability Measurement Methods
Let’s get started
Logging
Heuristic Evaluation
Labs
Focus groups
WalkThroughs
Satisfaction Surveys
![Page 29: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/29.jpg)
Current Research-Summary of Usability Evaluation Methods
![Page 30: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/30.jpg)
Usability LoggingMeasurement and Data Collection
• Identify users by using session ID to identify a unique user.
• Iteratively insert code into the application
• Collect data• Analyze the data for each
attribute in different dimensions and aggregations
• Determine the need for further calculations and what attributes to measure further
• Revise the data we are collecting, adding or decreasing granularity
![Page 31: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/31.jpg)
Satisfaction Surveys
The process:1. Calculate the usability score (satisfaction) of version X2. Do the survey3. Change to version X.1 – make changes to the
software directly correlated to the usability factors to either increase or decrease the usability score
4. Do the survey again 5. See if differences made change impact the survey
results
![Page 32: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/32.jpg)
Notes on Satisfaction and Usability
• Satisfaction is a subjective feeling dependent on many things other than usability:– A user can be highly
satisfied but the application with low usability.
– An application can be highly usable (high usability) but the user is not satisfied!
Highly usablesoftware
My passworddoesn’t work
don’t havewhat I want I’m unsatisfied
I’m satisfied!
!low usability
software
Finished my work today
Nice weather today
![Page 33: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/33.jpg)
External Quality Requirements Measure EI value P/GI value Global Quality Indicator 61.97%
1 Usability 60.88% 1.1 Understandability 83% 1.1.1 Icon/label ease to be recognized 100% 1.1.2 Information grouping cohesiveness 66% 1.2 Learnability 51.97% 1.2.1 ……………………………………………… … 1.3 Operability 49.50% 1.3.1 Control permanence 100% 1.3.2 Expected behaviour 50% 2 Content Quality 63.05% 2.1 Content Suitability 63.05% 2.1.1 Basic Information Coverage 50% 2.1.1.1 Line item information completeness 2 50% 2.1.1.2 Product description appropriateness 50% 2.1.2 Coverage of other Contextual Information 76.89% 2.1.2.1 ……………………………………………….. … 2.1.2.2 Return policy information completeness 33%
Sample Evaluation
![Page 34: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/34.jpg)
Healthcare SoftwareHeuristic Evaluation
![Page 35: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/35.jpg)
Conclusion
• Usability is an abstract concept• Defining usability is different for each
organization• Need a model for your organization• The model is the foundation of what to
measure• Once you can measure, then you can evaluate
and improve
![Page 36: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/36.jpg)
Next Steps
• Produce an action plan– What usability attributes are important to your
organization?• Develop a model
– What data can you collect/Which technique can you use
• Maybe some elements of the model drop out-can’t be measured that easily
– Start collecting and developing benchmarks• Discuss with your manager/team
![Page 37: Usability modeling and measurement](https://reader034.fdocuments.in/reader034/viewer/2022052410/555c03a1d8b42a56448b53e8/html5/thumbnails/37.jpg)
Thanks
Questions and Answers