Headline(Mis)Adventures In Usability

Paul O’BrianService Manager for

Portals & Mobile

Portals at CU-Boulder: A little background

• First portal went live in 2004• Connected to student data on mainframe SIS• Joined by 2 others: admitted student & prospect• Kept growing, polishing, improving• By 2009, a mature & well-liked service

• And then…

• STUDENT SYSTEM REPLACEMENT

• CU system bought PeopleSoft Campus Solutions & Enterprise Portal

• Boulder campus decided to convert portals• Portal transition occurred as part of SIS replacement• New portals went live in 2010• Last legacy portal retired in 2012

Live Green

Portals at CU-Boulder: Overview• MyCUBoulder

• Recruitment portal• Aimed at prospects, applicants, admitted students, & confirmed students• Features:

• Marketing information• Access to application

• MyCUInfo• Main portal• Aimed at continuing students, faculty, staff• Features:

• Self-service (both PeopleSoft and otherwise)• Single signon to campus resources• News/announcements• Static information

Caught up at last, but is it good?• In 2012, we’d finally recreated all the legacy functionality• But we needed to know what was working and what wasn’t

• Solution: usability testing!

The Plan: live usability testing• What is live usability testing?

• Put a user in front of the portal and ask them to perform a high-level task

• Observe where they struggle and why• Have them think out loud as they use the site• Record the session for later analysis• Report on these tests to determine worst problems, easiest problems

• Prep work required• Decide what’s in scope• Identify most critical functions• Create scenarios that capture those functions• Determine data needs for those scenarios

• And then…

A snag: Need for clean test data• Having students use their own profile complicates the scenarios• So we need test users, but realistic ones• Clone existing users?

• We must be sure we’re FERPA-safe• So after cloning, we scrub all personally identifiable info• Analysis to determine what this is

Can we build it? Yes we can!

• Test data has been a problem from Day 1 of the portals• We’ll build a tool to generate test data!• Query the database for users that fit a profile• Clone them and scrub their data• But that will take some time…

Plan B: Survey our users• How to use the time while the tool is being built?• Let’s send out a survey! • So we drafted a survey for MyCUInfo, built it in Qualtrics• Figured out incentives, everyone bought in

• And then…• A disastrous spring startup• Portal down for first two days of classes• Do we call off the survey?• We modified it• Prepared to send out on March 11

• And then…

Another snag: battling surveys

• Our campus IT office was running a concurrent survey• Very high-profile, part of a high-level evaluation effort• Consequently, we were asked to hold off our survey• After closing date of OIT survey, finals arrived

• Too late for a spring survey

• Meanwhile, test data tool developer struck with major illness

Head, meet desk

Plans C, D, E, F, and G• Screen reader demo

• Blind users show us their experience of navigating the portal

• MyCUBoulder (recruitment portal) survey• Different audience from IT survey, so no conflict

• MyCUBoulder live testing• For certain audiences (i.e. prospects), clean data is not an issue

• MyCUInfo survey, a little later• We’ll still send out the MyCUInfo survey in the fall• Removing the spring disaster adjustments

• Orientation observation• Observe students using our portal for the first time• Enlist orientation advisers as focus group