8 Usability Lessons for Freddie Macfrom the 2004 UPA Conference

Prepared for the Usability Community of PracticeBy Mark D. AlvesAugust 11, 2004

2

Overview of UPA Conference

The 13th annual Usability Professionals’ Association conference, Connecting Communities, attracted 470 attendees to Minneapolis, Minnesota from June 7-11, 2004.

Some events were major presentations for all attendees while at other times smaller, concurrent sessions were offered. Usability vendors were onsite as well.

This year’s materials are at: http://upassoc.org/conferences_and_events/upa_conference/2004/

Next year's conference is in Montreal, Canada from June 27 - July 2. Some of the most interesting information comes out of informal

discussions in between sessions or after hours. Freddie Mac’s usability experience and challenges are not unique.

Some companies are ahead of us, some behind.

3

Examples of Common Items That Could Use Improved Usability

Minneapolis Sculpture Garden

4

Examples of Common Items That Could Use Improved Usability...

Mini-golf hole designed by an artist.

5

Examples of Common Items That Could Use Improved Usability...

Mini-golf hole designed by an artist.

6

8 Usability Lessons for Freddie Mac

1. If your users are split 50-50 on a design choice, let them decide.2. Your lab doesn’t have to be sterile.3. Join UTEST now and participate later in the community.4. Speed up design and testing, but recognize there’ll be false positives and missed errors.5. Prepare for the difference between an intranet and an enterprise portal.6. Use TechSmith’s Morae for reviewing test participants’ actions and results, http://techsmith.com/products/morae/.

7. Get good card sorting results with helpful tools and a handful of testers.8. Create a successful online “live” event, even when most of it is pre-produced.

Microsoft games and joysticks: down & out or up & away? If your users are split 50-50, then make them pick. For Freddie Mac’s website, offer information both ways. Perhaps the forms are listed numerically and by category; or the products are listed alphabetically and by need.

Bose speakers: their usability lab is set up like a living room. Make your test environment similar to what your users’ areas are like if you can’t actually test them in their natural settings. Perform some Homefront tests at users’ desks.

UTEST is a private, non-archived listserv for discussing budding usability ideas and solving usability problems. To request membership, contact Dr. Tharon Howard at tharon@clemson.edu after reading about the special policies and culture of the community: http://people.clemson.edu/~tharon/utest/policies.html

Speed up design and testing. Add an enterprise portal with care. TechSmith’s Morae software captures test participants’ screen behaviors without the obtrusiveness and intimidation factor of a video camera.

http://techsmith.com/products/morae/ Get good card sorting results with helpful tools and a handful of testers. Create a successful online “live” event, even when most of it is pre-produced.

7

Lessons From Other ParticipantsLesson 1: If your users are split 50-50, let them decide.

When it comes to video games, Microsoft employs usability principles while striving for “appropriate challenges” instead of minimal challenges. Their testing showed that gamers were split 50-50 on certain games as to whether pushing the joystick up should make a character’s gun go down and away (like a pilot) or move up (like a map). Their solution was to force the user to pick a preference before the game can begin.

For Freddie Mac websites: offer information both ways. Perhaps your forms are listed numerically and by category; or your products are listed alphabetically and by customer need.

Lesson 2: Your lab doesn’t have to be sterile. Bose makes high-end audio speakers so their usability lab is set up like a living

room. Make your test environment similar to what your users’ areas are like if you can’t actually test them in their natural settings.

For Homefront: perform some tests at users’ desks.

Lesson 3: Join UTEST and later participate in the community. UTEST is a private, non-archived listserv for discussing budding usability ideas and

solving usability problems. To request membership, contact Dr. Tharon Howard at tharon@clemson.edu after reading about the special policies and culture of the community: http://people.clemson.edu/~tharon/utest/policies.html

8

Speed Up Design and TestingLesson 4: Speed up design and testing, but recognize there’ll be false positives and missed errors.

Inspection evaluations are efficient, but not effective– Miss 25% of problems– Find more non-problems than problems (for every 2 problems found you’ll

have 1 that you missed) User testing is effective, but not efficient Mind Design Systems, http://mindd.com, is working on software to improve the

efficiency of human testing, but this won’t be commercially available this year. One way the software improves efficiency is by stopping a task after a certain number of clicks; you don’t waste time if a user gets really lost.

Ask true/false questions if testing comprehension. This cuts down on the user agonizing over the answer.

For designing intranets, relying on content analysis can speed up the process.– Use paired-comparison tests to see preference (if any) between content– User selects preference or indicates none– User reviews the hierarchy of preferences that results– Don’t focus on employee task analysis, but on what content the staff

needs.

9

Add an Enterprise Portal to Your Intranetwith Care

Lesson 5: Prepare for the difference between an intranet and an enterprise portal.

Boeing is working on adding customizable enterprise portals to its 8 million page intranet.

– Keep user feedback forms simple by skipping the user-selected categories. (25% picked the wrong category, 25% picked “other” so 50% of the data was useless.

– New users typically “test” a portal by searching for applications that they know already exist on the old site: make sure the search engine picks them up.

– Users are receptive to “my pages” and related customization.– Users are resistant to “subscribing” to a community rather than just

looking at a website: “I don’t want to join a club.”

10

Use TechSmith’s Morae for Reviewing Test Particpants’ Actions and Results

Lesson 6: Use TechSmith’s Morae for reviewing test participants’ actions and results.

TechSmith’s Morae software, http://techsmith.com/products/morae/, captures test participants’ desktop behaviors and can integrate a log of activity with any sound and video you record. Captured behaviors include:

– Mouse clicks (left and right)– New windows opened– Entered text– Accessed menus– Any markers you insert

You can capture all activity or specify when the recording starts, such as upon clicking on a certain link or opening a particular application.

Remote viewer option lets you see the subject’s desktop over a LAN connection so you can be in another room. No one-way mirrors required.

11

Card Sorting: Use It Quickly and Easily

Lesson 7: Get good card sorting results with helpful tools and a handful of testers.

How Many Users Are Enough for a Card-Sorting Exercise?Poster presented by Larry Wood of BYU, who developed Web Sorthttp://www.websort.net

Correlations:– 5 users=0.75– 15 = 0.90– 20 = 0.93– 30 = 0.95 – 60 = 0.98

12

Card Sorting: Use It Quickly and Easily (cont.) Card Sorting: How Many Users to Test?

Jakob Nielsen co-opts a UPA poster http://www.useit.com/alertbox/20040719.html

A 13-step Approach to Card Sortingby Gerry McGovernhttp://www.gerrymcgovern.com/nt/2002/nt_2002_09_23_card_sorting.htm

Card sorting tools and technology– Analyzing Card Sort Results with a Spreadsheet Template

by Joe Lamantia http://www.boxesandarrows.com/archives/analyzing_card_sort_results_with_a_spreadsheet_template.php or http://tinyurl.com/lcuj

– uzCardSort for Uzilla http://uzilla.mozdev.org/cardsort.html– IBM’s EZSort http://www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/410– CardZort by Jorge A. Toro

http://condor.depaul.edu/%7Ejtoro/cardzort/cardzort.htm– NIST WebCAT web category analysis tool

http://zing.ncsl.nist.gov/WebTools/WebCAT/overview.html– “Classified” from Information & Design (Australia)

http://www.infodesign.com.au/usabilityresources/classified/

13

Sponsor a Live (Enough) Event OnlineLesson 8: Is it Live or is it Freddie Mac? “We now bring in enough people to fill a stadium for the cost of a barn

dance.” Scott Anderson, director of e-marketing, Sun Microsystem. Sun’s online “live” events replace the hodgepodge of tactics they formerly employed for their quarterly R&D announcements.

Take the best of online news sites, websites for temporary museum exhibits, ESPN-type sports updates, live chats with celebrities/newsmakers, and online learning sites.

Combine them to create the urgency and excitement of a “live” event, with the certainty of pre-recorded commercials and the shelf-life of important product support materials.

It’s cheaper than doing the event in person; it’s more interesting than a regular website; and the results are measurable.

14

Sponsor a Live (Enough) Event Online cont.

Use the Tivo philosophy experience: what’s happening now is central, the past is accessible and the future is only glimpsed at.

The design implications?– The past: Content that has already been released is available for

viewing at any time.– The present: Content is being released linearly.– The future: Promote future content and encourage users to come

back; whether the content is prepared or not, it is not released until the pre-determined time.

Organize prepared content around “live” events. Break a usability code in exchange for business results: everything is

not available all of the time.

15

Sponsor a Live (Enough) Event Online cont.

Implications of using video:– Promoting “live” aspect draws crowds– Users will view content for a longer period of time when they are

watching a live event.– Focus on quality, not quantity.– People want accompanying text to be scannable, not full

transcripts. Have a backup plan

– Users who opt not to install plugins or players– Prepare your help desk for extra volume and more technical

questions– Allow extra time for QA to test multiple browsers and versions of

the same browser

16

Sponsor a Live (Enough) Event Online cont.

Examples of tightly integrated video and website.

17

Sponsor a Live (Enough) Event Online cont.

Post-event, you default to playing the first video (not “now playing”).

18

Sponsor a Live (Enough) Event Online cont.

Options for applying this at Freddie Mac externally.– In conjunction with our own customer conferences or analyst

meetings – Product announcements– As part of regularly scheduled mission updates– In support of a job fair– Integrated with a national event, like homeownership month.

Internal options– Division events where staff is distributed across multiple locations– Diversity events to maximize employee participation– “All”-employee events where not everyone can attend

19

Websites Are Never Finished, But Keep Evolving

Good design always happens.

20

Websites Are Never Finished, But Keep Evolving

Good design always happens...just like the patient always stops bleeding.

21

Available Documents from UPA ConferenceWednesday, June 9, 2004

Making an Enterprise Portal Useful for EmployeesKen Becker, The Boeing Company

Voice Recording Tips for the Non-AudiophileTimothy Keirnan, Tec-Ed, Inc.

The Blind Leading the Blind: Theorizing a Web for the Visually ImpairedJessica D. Moore, Joseph Matthews, AARP Services

Panel: HCI and Human Factors Master's Degree ProgramsKirsten Robinson, Bentley College; Jessica Webster, Tufts University; Amanda Nance, Georgia Tech; Dana Gelman, Carnegie Mellon University; Rebecca Tremaglio, University of Michigan

Expediting the Usability Testing ProcessRobert Bailey, Computer Psychology; Kent Bailey, Mind Design Systems

How to Predict the Future: Becoming an Agent for ChangeSusan Fowler, FAST Consulting

Architecting Time: Designing Online EventsKatrina Friedman, Hot Studio

Site-Level Webapp Navigation Using Breadcrumb and Sibling MenusJames Snell, Concur Technologies

Making a Better Web FormCaroline Jarrett, Effortmark, LTD; Christopher Minott, LoanBright.com

When Your Group Can't Do It All: Investing UCD Resources WiselyLisa Battle, Lockheed Martin

User Experience Issues of IP Telephony: Not Your Plain Old Telephone AnymoreYihsiu Chen, PhD, AT&T Labs

An Alternative Method for Intranet Design Based on Content AnalysisRegis Magyar, Brian M. Anderson, Nancy Babiarz, James Marion, Panasonic Wireless

Accessibility at an Enterprise Software Company: From Guidelines to Accessible InterfacesGeorge Hackman, Oracle

How to Ensure Usability Offshore When Doing UI DevelopmentLiam Friedland, Westbridge Technology

22

Available Documents from UPA ConferenceThursday, June 10, 2004

The Value of Student Internships in Usability: Connecting Students with Professional Usability CommunitiesLee-Ann Kastman Breuch, Alice de la Cova, University of MinnesotaTask Analysis Tune-Up Paul McInerny, IBM; Interaction Guidelines for Japanese Web Forms Manabu Ueno, SociomediaMeeting the Challenges Posed by International Usability Testing: Two Case StudiesJenny Blackburn, Julianne Bryant, Getty ImagesDemographic Differences in Preferred Web Site Content Ann Chadwick-Dias, Donna Tedesco, Thomas Tullis, Fidelity Inv.A Structured Template for Writing Scenarios Paul McInerny, IBMUsability Guidelines for 3G Mobile Internet: A Korean Case Se-Hoon Kim, Young-Hee Kim, Mi-Jin Kim, Hey-Jin Chung...Too Much Usability? Steps and Missteps Ron Vutpakdi, Landmark Graphics CorporationAchieving Experience Equity and Universal Usable Access: Research Results and an Innovative FutureMary Theofanos, National Cancer Institute; Ginny Redish, Redish and AssociatesBalancing Rigor, Adaptation and Mentoring: Field Study with Customers to Initiate a Usability ProgramLaurie Kantner, Tec-Ed, Inc.; Rebecca Shaufl, Manatron, Inc.Going in Blind Doesn't Help: Cues for Navigation Sarah J. Swierenga, PhD, Laura G. Militello, Megan E. Gorman...Advanced Topic Seminar: Progress Indication and Responsiveness: User Needs, Design, and ImplementationJeff Johnson, UI Wizards; Paul McInerney, IBMUsability of Health Websites: What Have We Learned? American Institutes for ResearchWebsite Usability Questionnaires Thomas Tullis, Fidelity Investments; Jacqueline N. Stetson, Fidelity InvestmentsHow Standards Could Benefit Your Business Nigel Bevan, Serco Usability ServicesAdvanced Topic Seminar: Using Sounds & Graphics in Application Interfaces Alice Preston; Susan Fowler, FAST ConsultingPanel: Building an Online "Community“ Tharon Howard, Clemson University; Caroline Jarrett, Effortmark... The Usability Imperative Inherent in the Semantic Web Duane Degler, IP Gems; Renee Lewis, Pensare GroupEvaluating the Usability of Mobile Handheld Devices Regis Magyar, Brian M. Anderson, Nancy A. BabiarzAchieving Global Competitive Advantage: The Business Benefits of Designing for UsabilitySteve Jacobs, Information Technology Technical Assistance & Training CenterPeer Reviewed Paper: Integrating Accessibility into User-Centered Design Process: A Case StudyKay Lewis, Accessiblity Institute, University of Texas at AustinPeer Reviewed Paper: Freehand Interactive Design Offline (FIDO): A New Method for Participatory DesignDonna Tedesco, Ann Chadwick-Dias, Thomas Tullis, Fidelity Investments

23

Available Documents from UPA ConferenceFriday, June 11, 2004

Invited Speaker: Landscapes of Memory: History for Communities to Live ByKurt F. Anschuetz, Rio Grande Foundation for Communities and Cultural LandscapesDesign Patterns and Guidelines for Usable and Accessible Web ApplicationsDavid Hoffman, Lisa Battle, Lockheed MartinJustifying Funding and Space for a Usability LabAlice de la Cova, University of MinnesotaPeer Reviewed Paper: Participatory Techniques for the Design of a New Input DeviceEvelyne Millien, Camille Roux, Centre MultiCom Laboratoire CLIPS-IMAGWe Bombed in New Haven: Lessons from the TheaterWhitney Quesenbery, Whitney Interactive Design, LLCPeer Reviewed Paper: Participatory Design in Cognitive WalkthroughsSusan Mings, PhD, Scott Ottaway, PhD, Microsoft Corp.

PostersViSA: Video Segmentation and Annotation Afzal Godil, National Inst. of Standards and TechnologyNew Techniques for Understanding Communities Erika Darling, Emily Leventhal, The MITRE CorportationAdapting User-Centered Design to the Consumer Electronics Research and Development ProcessJoonhwan Kim, SungWoo Kim, Younghwan Pan, Samsung ElectronicsEye-tracking in Usability Testing: What Questions Can It Help Us Answer?Judith Ramey, Mary Deaton, Rich DeSantis, Elisabeth Cuddihy, Cory King, Quan Zhou, Jana Jones, University of WashingtonBecause Sometimes More is Less Julie Bzostek, Brett Walters, Micro Analysis and DesignRules of the Road: A Practical Guide to International Research Kathi Kaiser, Lyman Casey, Centralis Partners, Inc.Taskflow Charting: An Interactive Validation Method for Task Analysis Kraig Finstad, Intel CorporationHow Many Users Are Enough for a Card-Sorting Study? Larry Wood, Brigham Young University; Thomas Tullis, Fidelity InvestmentsRecruit More Test Participants Faster, Cheaper and Happier by Piggybacking on Existing Customer ConferencesMark Alves, Freddie Mac