MeasuringU

10 Things to Know about SUS

Jeff Sauro

@MeasuringUmeasuringU.com

MeasuringU

SYSTEM USABILITY SCALE (SUS): 1986

MeasuringU

SYSTEM USABILITY SCALE (SUS)

1. I think that I would like to use this system frequently.

2. I found the system unnecessarily complex.

3. I thought the system was easy to use.

4. I think that I would need the support of a technical person to be able to use this system.

5. I found the various functions in this system were well integrated.

6. I thought there was too much inconsistency in this system.

7. I would imagine that most people would learn to use this system very quickly.

8. I found the system very cumbersome to use.

9. I felt very confident using the system.

10. I needed to learn a lot of things before I could get going with this system.

MeasuringU

SCORING SUS

For odd items: subtract one from the user response.For even-numbered items: subtract the user responses from 5

•This scales all values from 0 to 4 (with four being the most positive response).

•Add up the converted responses for each user and multiply that total by 2.5.

•This converts the range of possible values from 0 to 100 instead of from 0 to 40.

MeasuringU

1. SUS SCORES ARE NOT PERCENTAGES

2. Average SUS Score is a 68 (500 Products)

Included in TryMyUI

MeasuringU

3. SUS MEASURES USABILITY & LEARNABILITY

4. I think that I would need the support of a technical person to be able to use this system.

10. I needed to learn a lot of things before I could get going with this system.

Learnability Factor

MeasuringU

ACQUIESCENT & EXTREME RESPONSE BIASES

MeasuringU

THE DARK SIDE OF ALTERNATING:

Mistakes, Misinterpreting & Miscoding

• 18 of 107 (17%) Standard SUS questionnaires contained internal inconsistencies

– Users Agreeing to 3+ positive & negative items

• 3 out of 27 datasets (11%) were miscoded by researchers

MeasuringU

4. REVERSING THE ITEMS CAUSES MORE HARM THAN GOOD.

2. I found the website unnecessarily complex.

8. I found the website very cumbersome to use. -

-

2. I found the website to be simple. +

8 I found the website very intuitive. +Double Check Item & Response Coding, Use Software Like TryMyUI or use the All Positive SUS

All Positive Version of the SUS

MeasuringU

5. FAMILIARITY BREEDS CONTENT

Websites Consumer Software

Experience with software and websites tends to increase SUS scores by 10% on average

10%

10%

MeasuringU

6. USABILITY PREDICTS CUSTOMER LOYALTY

SUS Scores explain between 30% - 50% of responses to the Net Promoter Question

MeasuringU

7. RAW SUS SCORES AREN’T NORMALLY DISTRIBUTED BUT THE SAMPLE MEAN IS

311 Raw SUS scores from Budget.com

1000 SUS Means

N = 8 N = 20 N = 30

Parametric Statistical Calculations & the Mean work fine with SUS

MeasuringU

8. You Can use SUS on Small Sample Sizes

You can use SUS on any sized sample. Use confidence intervals to quantify the sampling error.

1000 differences between actual SUS scores and observed SUS scores when n=5

MeasuringU

9. SUS SCORES WERE NOT MEANT TO BE DIAGNOSTIC

They aren’t meant and can’t tell you what causes scores to be high or low.

SUS Scores explain on average only about 10% of the variation in Task Completion Rates

MeasuringU

10. SUS IS TECHNOLOGY AGNOSTIC & RELIABLE

SUS & SUMI have a strong correlationCorrelation between SUS & SUMI

MeasuringU

10 THINGS TO KNOW ABOUT SUS

1. SUS Scores are Not Percentages

2. Average SUS Score is a 68 (500 Products)

3. SUS Measures Usability & Learnability

4. Reversing the items causes more harm than good.

5. Familiarity Breeds Content

6. Usability Predicts Customer Loyalty

7. Raw SUS Scores aren’t Normally Distributed but the sample mean is

8. You Can use SUS on Small Sample Sizes

9. SUS scores were not meant to be diagnostic

10. SUS is technology agnostic & reliable

16

MeasuringU

@MeasuringU

Jeff Sauro is a Six-Sigma trained statistical analyst and pioneer in quantifying the user experience. He specializes in making statistical concepts understandable and actionable.

Jeff has published over fifteen peer-reviewed research articles on statistics and usability and presents tutorials and papers regularly at the leading Human Computer Interaction conferences: CHI, UPA, HCII and HFES.

Jeff received his Masters in Learning, Design and Technology from Stanford University and has worked for Oracle, PeopleSoft, Intuit and General Electric.

www.MeasuringU.com

MeasuringU is a usability and quantitative research firm based in Denver, Colorado USA focusing on the statistical analysis of human behavior and quantifying the user experience for Fortune 1000 companies.

17

ABOUT MEASURING U