Case Study: Evaluating candidate features through usability testing

and the KANO modelJoseph Dickerson

Goals of the test

• Objective was to identify what (if any) proposed mobile banking features should be prioritized for the next release of the product

• This was done by testing the features with participants and using the KANO Model to the identify what features are positively (or negatively) received by participants

The KANO Model

• The KANO Model is used in product development and looks at customer preferences in five key categories

– Delighters (or “Attractive”)

– Satisfiers (or “Value-Adds”)

– Expected (or “Must-Have”)

– Unimportant (or “Indifferent”)

– Detractors

Delighters

Baseline

Satisfiers

Detractors tend to occur when a basic need is

implemented badly or a unneeded feature “overwhelms” an

experience

Process

• 12 Participants were recruited based on legacy personas– All had to be current mobile banking users

• The participants were stepped through a series of screens, one set for each of the six proposed features– All features were described by the facilitator using

the same objective verbiage – The order in which features were presented were

different for each participant

Proposed features evaluated

• Click to Dispute (a transaction)• Scan a bill (to pay it)• Open a new account (through app)• Setup a budget• Search Transactions• Deposit a Check (aka Remote Deposit Capture)– This was a “control” – we already know this feature

scored highly with customers

Process

• After being walked through each feature the participant was asked to rate the feature and discuss their reactions if it was offered by their bank– I like it– I expect it– I’m neutral– I can tolerate it– I dislike it

• Then ask their reaction if it was NOT offered by their bank• The responses were then analyzed and classified using

the KANO Model

8

Analysis MethodNegative Question

Like Expect Neutral Tolerate Dislike

Positive Question

Like - Attractive Attractive AttractiveOne-

Dimensional

Expect Undesired Unimportant Unimportant Unimportant Must-Have

Neutral Undesired Unimportant Unimportant Unimportant Must-Have

Tolerate Undesired Unimportant Unimportant Unimportant Must-Have

Dislike Undesired Undesired Undesired Undesired -

Responses were averaged and weighed for both questions and also analyzed based on persona mapping

Results

• Had one “No-show”, so only 11 participant’s feedback was captured

• Of the six features reviewed, only two features were indicated as “Must Haves” – Search Transactions– Deposit a Check

• Two features were indicated as “Attractive”– Scan a bill (to pay it)– Setup a budget

• Two features were indicated as “Unimportant”– Open a new account– Click to dispute (a transaction)

10

Observations• Search Transactions was a potential “Delighter”

– Many positive comments about the perceived usefulness of this feature

• Participants liked the idea of setting up a new account on the phone, but doubted they would use it

• The majority of participants said they would never use the Click to Dispute (a Transaction) feature – they would call the bank’s support number to do this task

• Scan a Bill (to pay it) was well received but many participants doubted they would use it

• None of the features were perceived to be potential Detractors

Recommended features and focus

• Based on these results, we recommended a focus on adding Search Transactions for the next release of the mobile banking product

• Additional evaluation of Scan a Bill (to pay it) and Setup a Budget was needed

• Open a New Account should be considered as a feature to be added to the tablet banking application

12

Participant Responses

How would you feel if this feature existing in your mobile banking app?

Participant Reaction

Feature Like Expect Neutral Tolerate Dislike

Click to Dispute (a transaction) 2 2 6 1 0Scan a bill 6 1 3 0 1

Open a new account (through app) 2 0 6 0 3

Setup a budget 4 1 5 1 0Search Transactions 8 2 1 0 0

Deposit a Check 4 6 1 0 0

13

Participant Responses

How would you feel if this feature didn't exist in your mobile banking app?

Participant Reaction

Feature Like Expect Neutral Tolerate DislikeClick to Dispute (a transaction) 0 1 8 0 2

Scan a bill 1 4 3 2 1

Open a new account (through app) 1 3 7 0 0

Setup a budget 2 4 3 1 1

Search Transactions 0 2 3 2 4

Deposit a Check 0 1 2 2 6