Webinar:

How to Measure the ROI of User Experience

Susan Weinschenk, Weinschenk Institute, LLCAlfonso de la Nuez, UserZoom

#uzwebinar

Susan WeinschenkFounder and Principal Weinschenk Institute, LLC@thebrainlady

Alfonso de la Nuezco-Founder and co-CEOUserZoom@delanuez23@userzoom

Quick Housekeeping

• Chat box is available if you have any questions

• There will be time for Q&A at the end

• We will be recording the webinar for future viewing

• All attendees will receive a copy of the slides/recording

• Twitter hashtag: #uzwebinar

Sample ROI Calculation100 visitors a day come put an item in the shopping cart of your e-commerce

site

But 30% don’t complete because of a user experience problem with the cart

You are losing 30 sales a day due to a UX issue

@ $10 a purchase = $300 a day lost

300 x 365 = $109,500 a year lost

It will take $20,000 and 2 weeks to fix the UX issue

In 10 weeks you will have made up the $20k you spent

In 2.5 months you will have paid for the fix with the increased revenue: ROI in 2.5 months

What We’ll Cover

different ways to calculate ROI

how to deal with estimates and assumptions for key indicators

what key indicators you should use

how to actually measure to compare with estimate

how to estimate cost of the UX work itself

which should you do: fix later or get it right from the start?

the role of tools (card sorting, user testing, etc) in your ROI calculations

fixes rather than UX from beginning

My First ROI Calculation (circa 1994)

Software for bank tellers

5000 users

Interface was hard to learn and hard to use -- poor user experience

Assumptions: waste 5 seconds per transaction

100 transactions per person per day

5000 people x 100 transactions x 5 seconds = 2.5 M seconds = 41,666 minutes = 694 hours x $7/hour = $4,858 a day x 252 days = $1,224,216

That’s Not All!Take training from 2 weeks to 2 days, saves $1000 in training costs per new

person

10% turnover = 500 people a year x $1000 = $500,000 per year in training

PLUS help desk -- reduce calls by half, save 300 calls per month x $10 per call = $3,000 a month = $36,000 per year

$1,224,216 + $500,000 + $36,000 = $1,760,216 in first year

Cost to fix: $500,000

Recoup investment in 3.5 months

ROI = 3.5 months

To Calculate Your ROI1. Decide on your key indicators

2. Estimate lost opportunity or cost of having a poor user experience

3. Estimate the cost to research and fix a poor experience or cost to create a new one to start with -- cost of doing it right

4. Do your calculations

5. Turn your money into length of time to recoup the investment

There is no magic ROI formula

What should we measure?time on task?task success?satisfaction?efficiency?

errors?

There is no one “key performance indicator”

ROI Calculation Examples

•Errors:•(# of Errors) X (Avg. Repair Time) x (Employee Cost) X (# of employees) = Cost Savings

•(2 errors/week) X (60 minutes) X ($30/hour) X (100 employees) = $6,000 per week = $300,000 per year

•Cost of Development and Maintenance:•(# of changes) X (Avg. hours per change) X (cost of developer) X (4, if change is late!) = Cost Savings

•(20 changes) X (8 hours each) X ($40/hour) = $6,400 if fixed early, or $25,600 if changed late

ROI Calculation Examples

•Productivity:•(Time Saved) X( Employee Cost) X (# of employees) = Cost Savings

•(1 hour a week) X ($30/hour) X (1000 employees) = $30,000 per week = $1,500,000 a year

ROI Calculation Examples

http://spectrum.ieee.org/computing/software/why-software-fails

The problem only gets worse as IT grows ubiquitous. This year, organizations and governments will spend an estimated $1 trillion on IT hardware, software, and services worldwide. Of the IT projects that are initiated, from 5 to 15 percent will be abandoned before or shortly after delivery as hopelessly inadequate. Many others will arrive late and over budget or require massive reworking. Few IT projects, in other words, truly succeed.

In fact, studies have shown that software specialists spend about 40 to 50 percent of their time on avoidable rework rather than on what they call value-added

work, which is basically work that's done right the first time.

•Unrealistic or unarticulated project goals•Inaccurate estimates of needed resources•Badly defined requirements•Poor reporting of the project's status•Unmanaged risks•Poor communication among customers, developers, and users•Use of immature technology•Inability to handle the project's complexity•Sloppy development practices•Poor project management•Stakeholder politics•Commercial pressures

Why Projects Fail

Designing the User Experience

• User Research• Storyboarding, Designing, Prototyping, Visual Design• User Testing & Iteration

Is It Worth It?

• Cost to design it right the first time• Added cost than if you just designed it the way you normally do• Cost to fix the user experience problems after the fact• Opportunities lost if the user experience isn’t right

Watch Out

• Definition of “done”• Definition of “success”

When You Design The User Experience

–Saves user time–Saves developer time–Increases customer satisfaction–Increases use of the technology–Reduces training costs–Reduces calls to the help desk–Increases self-service

Project Fix or From Scratch?

$ amount to be gained or saved

Cost of UX to fix or design

ROI

A Fix

B From Scratch

C From Scratch

D Fix

E Fix

Using ROI To Plan Your UX Work

Medicine Net Website Re-Design

The Role of Testing & Research Tools in ROI

• Use to benchmark for “before” and “after” comparisons• Data from which to base your assumptions & estimates

Advantages of Remote Tools• Less expensive than in-person,

moderated data collection• Collect data from more people

for less cost -- enough to run statistic significance

• Collect data from diverse geographies for less cost

• Reduce the cost per project of doing user research and user testing during design

• Get results fast

intellectuals solve problems; geniuses prevent them

~albert einstein

Susan Weinschenk, Ph.D.

Weinschenk Institute, LLC

Email: susan@theteamw.com Twitter: @thebrainladyPhone: 847-909-5946Website: www.theteamw.comFacebook: Weinschenk Institute