Everything, but … testingA company and process focused approach of usability

Rita almendra Henri Christiaans

Innovation, but on what level?

Key themes and references

• In search of excellence (Peters & Waterman, 1982)• Total Quality Management (TQM)• Taguchi methods

• Just in time manufacturing (JIT)• Downsizing• Restructuring• Globalised manufacturing economies of scale

• Time to Market (TIM) processes• Extended enterprise / virtual organisations

(Davidow&Marlone,)• Concurrent engineering• Strategic information management

• Corporate Re-engineering (Hammer, 1993)• Employee empowerment• Quality Function Deployment (QFD)• Mass customisation (Davidow& Malone,1992;Zell,97)

• User focus• Cultural and social context• Value added• Differentiation

Source: Les Wynn, DMI 2000

1980 1985 1990 1995 2000

Cost

Quality

Delivery

Design

Customer first

Customerfocus

Productionfocus

Time frame

Changing Role of Design

Emotion product

Innovative Innovative productproduct

Styled Styled productproduct

The Doing and The Doing and Thinking designerThinking designer

The Doing The Doing designerdesigner

Specialization

Sustainability

User Centered

Strategic Role

Compelling Experience

Mass Customization

Standardization

Artificial Obsolence

Profit Centered

Passive Role

Enriching Function

Mass Production

Emerging Tendencies

What Is Usability?

• Usefulness  Degree to which users can successfully achieve goals/complete tasks  

• Effectiveness  

Ability of users to accomplish goals with speed & ease  

• Learnability  Ability to operate the system to some defined level of competence after some predetermined amount of training  

• Satisfaction  

Attitude of users, including perceptions, feelings and opinions of the product  

*Booth, Paul. An Introduction to Human-Computer Interaction.

London: Lawrence Erlbaum Associates, 1989  

USABILITYTHEORY &METHODS

USABILITYPRACTICE

ISO

-9241-1

1

Jord

an (

1998)

Nie

lsen (

1999)

Bevan (1991)Keinonen (1998)

Shackel (1984)

Norman (1988)

GAP

Project Overview

Overall research design of PhD project

User-centered design

User-centered design

User-centered design: not just testing

Testing: just do it

User-centered design

product development = multidisciplinary

Product Development Actors

Interaction Designer

• designs user interface

Development Engineer

• technology and production

Market Intelligence

• collects market information• defines marketing strategy

Product Manager

• Coordinates development• Sets priorities for product

Usability Specialist

• evaluates and improves usability

Industrial Designer

• designs physical appearance

10 (hands on) tips for user-centered product development

What meansusability to us ?

1

Whydo we actually want this?(define the value of usability)

2

Don’t letdesigners just ride their hobbyhorse

3

Safe what has been proved to begood(UI ‘paradigm’, product generations, knowledge management)

4

04/11/23

Innovatewhen it makes sense

5

Implementunderstanding(from user research and user testing)

6

Letdesignerslearn(user tests and after-sales feedback)

7

Give designersrich information

8

Information

Think concept ANDproduct

9

Making the right product,

making the product right.

(Bill Buxton)

Design forConsumerAND User

10

(bron: creating passionate users)

(bron: creating passionate users)

Before and after…

Consumer- functionality- results- aesthetics- brand- price

User- results- reliability- user friendliness

Market research or usability ?

Consumer demands

Activity Consumer Demands

Information / questions

research

buying What, where, how, price etc.

target groups, law and standards, distribution

Market research

transporting

When, how, who usergroups Market- & usability research

using Preparing, using, storing, cleaning, …

usergroups Usability research

pleasure Physio, Socio, Psycho, Ideo

cultural differences Market- & usability research

safety standards Usability- & technical research

maintenance

Usability- & technical research

….

Facilitating human-centred design:the ‘Ping-pong’ modelBrief: from management questions to research questions

Communication results: conclusions and recommendations

Monitoring

Monitoring of products available in the market to get a clear picture of the context of use and performance of a product or product group.

.

Inspiration is capturing the context of productuse; by using context mapping tools deeper insight in peoples aspirations and needs is gained.

Inspiration .

(Tuuli Matelmaki, 2005)

Exploration is searching for what interaction and/or relation can or should take place in using the product and what aspects are relevant for this interaction and/or relation.

Exploration

.

Evaluation of the concept(s) of the product. This is testing the way the different interactions/relations work out in a qualitative way.

Evaluation

.

Verification

Verification will take place in a quantitative research to verify if the, expected, interactions work in the right way.

.

Communication

Designers’ timescales are often pressured, due to commercial pressures, so if research is not presented in an usable format, it will be discarded or ignored.

Why early evaluation and testing?

Source: Hawksmere - ISO seminar material

$ 1,000 $ 6,000 $ 60,000

Analysis & Design Implementation Maintenance

phases

costs

• The costs to ‘repair’ mistakes:

When evaluating?

Analysis Construction Transition

User Involvement

Expert Involvement

Maintenance

Expert Review

Surveys

Focus Group Sessions

Active Usability Testing

Intermediate Usability Testing

Remote Usability Testing

Discovery

Elaboration

Concept Testing

Continuous Usability Evaluation

Target Group Analysis

Sample Usability Methods  

• User and Task Analyses    • “Contextual Inquiry”   • Observations of users in their natural setting  • Focus Groups  • Interviews  • Expert Reviews  • Usability Tests  

Research-Led

Design-Led

ParticipatoryDesign

generative toolsDesign

and Emotion

Critical Design

User-centered Design

contextual enquiry

Lead-user inovation

appliedethnography

Usability testing

Human factors and ergonomics

Dutch/Scandinavian design

Research-Led

Part

icip

ato

ry m

ind

set

Exp

ert

min

dset

Probes

Sanders, 2002

Design-Led

Research-Led

Design-Led

ParticipatoryDesign

generative toolsDesign

and Emotion

Critical Design

User-centered Design

contextual enquiry

Lead-user inovation

appliedethnography

Usability testing

Human factors and ergonomics

Dutch/Scandinavian design

Research-Led

Part

icip

ato

ry m

ind

set

Exp

ert

min

dset

Probes

Sanders, 2002

Design-Led

Script method (based on Alan Cooper’s Personnas, 1998) is a clear and detailed visualisation of the (future) usage in a way that the designer can imagine what the impact of the design will be. Based on objective research material and observations a cast of ‘archetypes of users’ and a context of use (physical and social environment) is described. All interactions, events and dialogues are described and visualised in scenes and story boards.

Scripts

Scripts in practice

Development of a new combination of products that should lead to a new ‘coffee experience’.

Client is convinced that the ‘coffee quality’ in combination with the ‘machine convenience’ will lead to customers satisfaction.

Client is focussed on building a strong positive corporate and brand image.

After product release the call rates and product failures must be almost zero.

Fictive case

Coffee Script

Research: Usability and market research.

Synopsis: Visualisation of interactions and relations.

Target group: Interdisciplinary design team.

Script and Storyboard:Visualisation of the impact of the design.

The script the client had in mind

Cast

Coffee Lover: the perfect consumer

Coffee Drinker: the inconstant consumer

Mr. Murphy: what can go wrong

Coffee Lover

Personal goal: enjoy family life

• Practical goal: make a high quality coffee at home

• Characteristics: perfectionist, always reads instructions, careful and patient

Coffee Drinker

• Personal goal: impress

• Practical goal: have a coffee at the office

• Characteristics: sceptic, always in a hurry, never reads instructions, rude

Mr. Murphy

• Personal goal: enjoy without effort

• Practical goal: efficiency when working at home

• Characteristics: clumsy, impatient, never reads instructions

ContextBased on consumer demands:

Coffee Lover Coffee Drinker

Mr. Murphy

Environment Home with family

Office Home, living alone

Buying

Transporting

Placing Scene 1

Storage Scene 2

Dispensing Scene 3

Coffee Quality

Cleaning

Standby

Maintenance

Scene 1: Coffee Lover gets a Senseo Crema•The family surprised Coffee Lover with the Senseo Crema. •After opening the box she carefully reads the users manual. She checks if the system is complete and not damaged.•She installs the Senseo Crema in their large kitchen, cleans it and puts it on to see if it works.•Within some minutes she manages (with the help of the quick reference card) to have her first Mild Roast.

Scene 2: Coffee Drinker's assorti

•Of course Coffee Drinker has the whole assorti of tastes at the office.•Right after lunch, just before he has to leave for a meeting, he needs a quick strong coffee. Unfortunately he mixed up the tops of the boxes.•Taking his first sip he is annoyed to taste the very mild one … he immediately wants to have another but when he notices there is too little water in the machine he leaves the office in a hurry … unsatisfied.

Scene 3: Mr. Murphy gets a coffee

• He likes ‘large’ coffees, longer satisfaction with less effort.

• When he makes his regular large he can’t find his mock … well the cup seems big enough.

• He notices too late that the cup is not big enough, not knowing what to do he just takes the cup away … just lucky he didn’t burn his hands!

Scripts: a powerful tool

• Strong visualisation of (real) interactions and consequences on relations.

• Tool for inspiration and decision making.

• Opens discussion on diversity of user- and target group.

• Pitfall: objective information becomes subjective.

Assignment

Brief• Definir as especificações de um novo produto –

barbecue tendo em consideração os perfis dos utilizadores (à frente apresentados)

• Ter particular atenção relativamente aos problemas “leves”

• Considerar aspectos de usabilidade que vão além das questões técnicas de uso e abarquem questões cognitivas

• Definir os descritores da análise a ser feita• Fazer análise SWOT• Antecipar problemas/soluções no uso dos produtos pensados• Mapear produto na sua complexidade de relações internas e

com o exterior• Identificar factores chave que podem potenciar/reduzir

usabilidade e subsequente satisfação dos utilizadores

Procedimentos• 3 grupos (pelo menos 2 pessoas cada)• De preferência 1 designer e 1 ergonomista em cada• Cada grupo trabalha um dos perfis dos utilizadores• Depois de feita a análise esta é apresentada à audiência (5

minutos)• Discussão dos trabalhos

Grupo A Grupo B Grupo C. Família classe média

. Idades à volta 35 anos

. Educação universitária

. Filhos com idades inferiores a 12 anos

. Residência em apartamento com áreas de exterior – varandas, páteos

. Rede de amigos com perfil idêntico

.Aptência para relação com a natureza mediada pela tecnologia

. Hábitos rituais de convívio com família alargada

. Família nuclear – 2 membros

.Idade superior a 60 anos

. Nível instrução médio

. Com filhos e netos

. Funcionamento de rede de suporte à família – rectaguarda no apoio às crianças e outras actividades

. Prática de reuniões regulares com grupos de amigos – jogos, almoços, viagens etc.

. Habitação unifamilar – vivenda com espaço exterior diferenciado

. Estudantes universitários

. Residência comunitária

. Rendimento baixo/médio

. Idas muito espaçadas a casa

. Espaço comum para realização de festas/convívio

Perfis dos utilizadores

Perfi

l A

Perfi

l B

Perfi

l C

DESIGN

Perfi

l A

Perfi

l B

Perfi

l C

ERGONOMIA

CENÁRIOS (VISÃO INTEGRADA) ELABORAÇÃO DE GUIÕES

constrangimentos

Ideias/potencial

...

constrangimentos

Ideias/potencial

...

(SWOT)

C

B

A

h.h.c.m.christiaans@tudelft.nl

almendra@fa.utl.pt

www.uselog.comproduct usability weblog

Muito Obrigado!