Integrating Quality, Quality in Use, Actual Usability and User

Experience

• Introduction / Related Work• 2Q2U Framework

• Extending ISO EQ Model• Extending ISO QinU Model• Model Instantiation

• SIQinU Improvement Strategy• Summary / Current Work

AgendaAgenda

Russian FederationCEE SEC’10CEE SEC’10 Moscow

Luis Olsina

GIDIS_Web Engineering School

Universidad Nacional de La Pampa

General Pico, Argentina olsinal@ing.unlpam.edu.ar

Philip Lew/Li Zhang

School of Computer Engineering

Beijing University of Aeronautics and Astronautics

Beijing, China

philiplew@gmail.com/lily@buaa.com.cn

Introduction: What is Quality?

Quality is not an absolute concept but rather a relative, multi-dimensional and contextual one

Quality of a Sw Product (as per ISO/IEC)

– The totality of characteristics of an entity that bears on its ability to satisfy stated or implied needs

– The quality of a system is the degree to which the system satisfies the stated and implied needs of its various stakeholders.

Introduction: What is a Quality Model?

Quality Model– Defined set of characteristics and the relationships between

them which provide the basis for specifying quality requirements and evaluating quality

Introduction: Views of Quality

Three Views of Sw. Quality (ISO 9126-1: 2001/25010:2009)

– Internal Quality● It can be measured and evaluated by static attributes of

documents such as specification of requirements, architecture, or design; pieces of source code, and so forth.

– External Quality● It can be measured and evaluated by dynamic properties of the

running code in a computer system, i.e. when the module or full application is executed in a computer or network simulating as close as possible the actual environment

– Quality in Use ● QinU is the degree to which a sw/Web application used by

specific users meets their needs to achieve specific task goals with effectiveness, efficiency, flexibility, safety and satisfaction in specific contexts

● It evaluates the degree of excellence,

5

Contributions

Contributions of this research are:● A proposed extension to ISO 25010 internal/external

quality and quality in use models● Proposed integrated and flexible framework 2Q2U

(internal / external Quality, Quality in use, actual Usability and User experience) for modeling software/WebApp quality requirements and particularly quality in use to relate the concepts of actual usability and user experience

● An instantiation, for illustration purposes, of the ISO 25010 quality in use model.

● The SIQinU strategy and process (Strategy for understanding and Improving Quality in Use in a non-intrusive way)

6

Related Work

● We have examined the related work with an eye for improvement opportunities

● ISO 9126-1 and 25010/25012

● Information Quality related work (Strong et al., Olsina et al.)

● User Experience related work (Bevan , Hassenzahl, etc.)

● Learnability and usability related work (Nielsen, etc)

Data vs. InformationSome slight difference in meaning between data and

information concepts. ● A piece of data is raw material; even though it can

bear some degree of information. ● Data come from attribute measurements, facts,

formula calculations, etc. – basically they have categorical or numerical values, a

scale type, and may also have an explicit procedure to produce or collect them.

● Structured data sets are often represented in databases.

● On the other hand, information has an added value over data. – That is, information is the meaningful interpretation of

data for a given context, purpose, and user viewpoint.

Data vs. Information

● Usually, a traditional software program is a mixture of functions and data.

● Many web pages are content oriented, i.e. are intended to deliver information (usually unstructured semantically). For example: – A scientific article can be hyperlinked and posted as

content Web pages.

– A webpage component, e.g. a shopping cart, can edit an item quantity and recalculate prices (applying a function over data). However, also may show contextual information …

9

Examining ISO 25010/25012

● ISO 25010 standard considers three views of quality– Internal/External quality, and Quality in Use

– Learnability categorized as an internal/external quality sub-characteristic under the operability characteristic.

● ISO 25012 considers data as an entity with a separate model and intended to be used in conjunction with ISO 25010. – There is a need to integrate information quality as part of the overall

quality of an application rather than as a separate entity

– Particularly for WebApps fully dependent on presenting information ...

● We propose to augment the ISO 25010 standard to include information quality as a characteristic of the internal/external quality models

Extending the ISO 25010 I/EQ Models

Content quality (Infoquality): the degree to which the software/WebApp provides accurate, suitable, accessible and legally compliant information when used under specified conditions

a) ISO 25010 internal/external quality modelb) adding information quality

Content Quality Sub-characteristics

Sub-characteristic Definition

Content Accuracy

The degree to which the software/WebApp provides information that is correct, credible and current.

Content Suitability

The degree to which the software/WebApp provides information with the right coverage, added value, and consistency, considering the specified user tasks and goals.

Content Accessibility

The degree to which the software/WebApp provides information that is accessible for all users (with or without disabilities) taking into account both technical and representational aspects.

Content Legal Compliance

The degree to which the software/WebApp adheres to standards, conventions, and legal norms related to content as well as to intellectual property rights.

12

Examining ISO 25010 QinU Model

● ISO 25010 quality in use model includes previous ISO 9126-1 quality in use characteristics while adding others.

● Learnability, an internal/external sub-characteristic of operability in ISO 25010 has not been moved to usability in use

– For fully functional complex WebApps, users have a relatively long learning curve when compared to a simple website

● Learnability in ISO 25010 as a product quality does not account for:

– Process of learning in real context of use– Some researchers (including Bevan) have determined learnability

to be linked directly with usability in use

13

Extending the ISO 25010 QinU Model

a) ISO 25010 quality in use modelb) adding learnability in use

14

Examining User Experience (UX)

● ISO standard definition for User eXperience is still not available.

● Hassenzahl characterized user’s goals as:

– Pragmatic (do-goals) : refers to the product's perceived ability to support the task achievement and focuses on the product’s utility and usability in completing tasks, the ‘do-goals’ of the user.

– Hedonic (be-goals) : product's ability to support the user’s achievement of ‘be-goals’, such as being happy, stimulated and satisfied.

● Real goal of the end user is fulfilling be-goals

● Be-goals are the driver of user experience

– Do-goals are not the end goal of the user – Pragmatic quality has no value by itself, other than

enabling accomplishment of be-goals.

15

Proposed 2Q2U framework and models

● Framework for modeling NFR for internal/external Quality, Quality in use, actual Usability and User experience (2Q2U)

● Extends the ISO 25010 standard – add 2 characteristics

– Information quality– Learnability in use

● Add two new concepts, actual usability and UX, to which characteristics and sub-characteristics can be related in a flexible way :

– Actual Usability: degree to which specified users can achieve specified goals with effectiveness in use, efficiency in use, learnability in use, and accessibility in use in a specified context of use.

– Actual User Experience: degree to which specified users can achieve actual usability, safety, and satisfaction in use in a specified context of use.

16

Proposed 2Q2U modeling framework …

17

Instantiating 2Q2U

• Actual usability is related to satisfying the do goals of the end user in completing real tasks • Not a prerequisite for actual UX, but has influence as one of its characteristics.

• Satisfaction in use is part of actual UX rather than in actual usability. • Satisfaction in use and safety are in italic to denote their hedonic nature.

• Safety is depicted as a hedonic characteristic of actual UX • Contributes to the user’s emotional needs for security and trust rather than just

satisfying a do-goal.

QinU Model Instance: Requirement Tree

1. Quality in Use 1.1. Actual User Experience

1.1.1. Actual Usability 1.1.1.1. Effectiveness in use

1.1.1.1.1. Sub-Task Correctness (synonym: Accuracy) 1.1.1.1.2. Sub-Task Completeness 1.1.1.1.3. Task Successfulness (synonym: Effectiveness)

1.1.1.2. Efficiency in use 1.1.1.2.1. Sub-Task Correctness Efficiency 1.1.1.2.2. Sub-Task Completeness Efficiency 1.1.1.2.3. Task Successfulness Efficiency

1.1.1.3. Learnability in use 1.1.1.3.1. Sub-Task Correctness Learnability 1.1.1.3.2. Sub-Task Completeness Learnability 1.1.1.3.3. Task Successfulness Learnability

Overview of SIQinU

Strategy for Understanding and Improving Quality in Use in a non-intrusive way

20

Summary and Current Work

● Provided reasoning and recommendations for adding information quality and learnability in use to extend the ISO 25010 standard.

● Characterized and described two new concepts, actual usability and actual user experience– Relating them in the 2Q2U modeling framework

● The SIQinU strategy and process (Strategy for understanding and Improving Quality in Use in a non-intrusive way) – A repeatable strategy and process for understanding and

improving QinU in a non-intrusive way● by inspecting also the external quality and improving the WebApp

– Using SIQinU we are currently performing a full case study for a Defect Tracking WebApp (JIRA)

BUA, Beijing, China

MSc. Philip Lew

E-mail philiplew@gmail.com

Dr. Li Zhang

E-mail lily@buaa.com.cn

Questions ?Questions ?

UNLPam, La Pampa, Argentina

Dr. Luis Olsina

E-mail: olsinal@ing.unlpam.edu.

ar

Authors References

Sassano, R.; Olsina L.; Mich, L. 2009, Modeling Content Quality for the Web 2.0 and follow-on Applications. Chapter XXI In: “Handbook of Research on Web 2.0, 3.0 and X.0: Technologies, Business, and Social Applications”, San Murugesan (Ed). IGI Global, Information Science Reference, pp. 371-386. ISBN: 978-1-60566-384-5

Olsina, L; Papa, F.; Molina, H; 2008, How to Measure and Evaluate Web Applications in a Consistent Way, Chapter in Springer Book titled Web Engineering: Modelling and Implementing Web Applications, Rossi, Pastor, Schwabe, & Olsina Eds.

Lew P., Olsina L., Li Zhang; (2010) Quality, Quality in Use, Actual Usability and User Experience as Key Drivers for Web Application Evaluation, In: LNCS 6189, Springer, ICWE 2010, Vienne, Austria, pp. 218-232