Christie HeitkampManager, User Experience & Information Architecture Team, WorldCat Local & WorldCat.org

Karen CalhounVice President, WorldCat and Metadata Services

Online Catalogs: Designing with Users in Mind

Online Catalogs: Designing with Users in MindWebinar

August 13, 2009

Polling –What Do You Think?Polling –What Do You Think?

1. Our staff understand what

our end users want from our

catalog

• A – True

• B – False

• C – Don’t know

By pirate johnnyhttp://www.flickr.com/photos/piratejohnny/2798872422/

More PollingMore Polling

2. My library’s end users are satisfied

with our online catalog

• A – True

• B – False

• C – Don’t know

By: sea turtlehttp://www.flickr.com/photos/sea-turtle/3181321172/

More PollingMore Polling

3. My library’s cataloging

practices and priorities reflect what end users want from our

catalog

• A – True

• B – False

• C – Don’t know

AN INTRODUCTIONAN INTRODUCTIONUse, User, and Usability Studies

AgendaAgenda

• Introduction to user experience

• Usability studies conducted on WorldCat.org and WorldCat Local over the past few years

• Meet the “users”

CreditsCredits

• Arnold Arcolio

• Mike Prasse

• OCLC Marketing group – Joanne Cantrell

• WorldCat Local Pilot partners

• User Experience (from wikipedia)

User experience design, most often abbreviated UX, but sometimes UE, is a term used to describe the overarching experience a person has as a result of their interactions with a particular product or service, its delivery, and related artifacts, according to their design

Usability & User ExperienceUsability & User Experience

User research methods we useUser research methods we use

• Usability Testing: 1:1 with a user completing a set

of tasks

• User Surveys: links or popups from our site

• Focus Groups: discussion groups with customers

or users

• Contextual Interviews: 1:1 observations and

interviews of users in their own environment

• Card Sorts: 1:1 activity with a user to understand

their mental model and information organization

• Soon to come at OCLC…usability testing with eye

tracking!

Wikipedia:Personas are fictitious characters created to represent the different user types within a targeted demographic that might use a site or product. Personas are useful in considering the goals, desires, and limitations of the users in order to help to guide decisions about a product, such as features, interactions, and visual design

A persona will help teams focus on designing an online catalog for a specific kind of user, instead of designing a system for any user (which usually means designing for a non-existent user)

Developing a persona for the online catalog searcherDeveloping a persona for the online catalog searcher

Who is using the catalog?Who is using the catalog?

Healthcare professional

Retired

Other (please specify)

Business professional

Teacher or professor

Student *

0% 5% 10% 15% 20% 25% 30% 35% 40%

7%

9%

12%

18%

19%

35%

Respondent TypeEnd Users Only (n=14,895)

Feb. 11 to June 27, 2009

What kind of students are these?What kind of students are these?

Graduate College Undergraduate College

High School Student

Middle School Student

Elementary School Student

Other 0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50% 48%

34%

11%

4%2% 2%

Type of Student (n=5,250)Feb. 11 to June 27, 2009

What kind of search? 80% known item searchingWhat kind of search? 80% known item searching

80%

20%

Looking for a Specific Item or Just Browsing When Searching World.org?

End Users Only (n=14,895)Feb. 11 to June 27, 2009

I searched for a specific item I was just browsing for information

What kind of item? 74% book searchesWhat kind of item? 74% book searches

74%

14%

1%2%

4%5%

Type of Specific Item Searched ForEnd Users (n=11,890)

Feb. 11 to June 27, 2009

Book Article CD DVD

Meet the “user” – primary personas of worldcat.orgMeet the “user” – primary personas of worldcat.org

Questions?Questions?

STUDY RESULTSSTUDY RESULTSOnline Catalogs: What Users and Librarians Want

With thanks to

Janet Hawk,

Joanne Cantrell,

Peggy Gallagher,

OCLC Market Research

Photo by allw3ndy

http://flickr.com/photos/allw3ndy/2757149584/

Credits: OnlineCatalogs Study

Online Catalogs: What Users and Librarians WantOnline Catalogs: What Users and Librarians Want

•End-Users expect online catalogs:•to look like popular Web sites

•to have summaries, abstracts, tables of contents

•to help find needed information

•Librarians expect online catalogs: •to serve end users’ information needs

•to help staff carry out work responsibilities

•to have accurate, structured data

•to exhibit classical principles of organizationhttp://www.oclc.org/us/en/reports/onlinecatalogs/default.htm

Objectives of our metadata quality researchObjectives of our metadata quality research

•Start over without assumptions about what “quality” is

•Identify and compare metadata expectations• End users

• Librarians

•Compare expectations of types of librarians

•Define a new WorldCat quality program …

•Taking into account the perspectives of all constituencies of WorldCat

• End users (and subgroups of end users)

• Librarians (and subgroups of librarians)

Research methodologies and demographicsResearch methodologies and demographics

• Focus groups

• Conducted by Blue Bear, LLC

• Three sessions: College students, general public, scholars

• Pop-up survey on WorldCat.org

• Conducted by ForeSee Results

• 11,000+ responses: Students (28%), educators (22%), business professionals (19%), other; mix of ages; 44% from outside U.S.

• Librarian survey

• Conducted by Marketing Backup

• 1,397 responses; North America (64%) and outside North America (36%); academic, public, special libraries; staff with roles in technical and public services, ILL, directors

What did we learn?

End-user focus group resultsWhat did we learn?

End-user focus group results

• Key observations:

• Delivery is as important, if not more important, than discovery.

• Seamless, easy flow from discovery through delivery is critical.

• Summaries and tables of contents are key elements of a description

• Improved search relevance is necessary.

What did we learn?

End-user focus group resultsWhat did we learn?

End-user focus group results

Casual Searchers

Undergraduates

Scholars

User-created lists not useful

User-created lists useful

User-created lists not useful

‘Shopper’ mentality: don’t even show me

what I can’t get now

User-contributed reviews and

ratings useful

Reviews and ratings useful if

authoritative

Strongest preference for

cover art

What did we learn?

End-user focus group resultsWhat did we learn?

End-user focus group results

•Other key findings of our study:

•Keyword searching is king, but …

• Advanced search (fielded searching) is useful

• Faceted browse is useful

• These help end users refine searches, navigate, browse, and manage large result sets

“End users enter a few short search statements into online IR systems. Generally, their queries bear two to four words.”—Karen Markey•Twenty-five years of end user searching, Part 1: Research findings. 2007. http://deepblue.lib.umich.edu/handle/2027.42/56093

•

What did we learn?

Pop-up survey suggestionsWhat did we learn?

Pop-up survey suggestions

Changes to help identify an item?

End users (n=7535)

End-user recommendationsEnd-user recommendations

• Improve search relevance

• Add more links to online full text (and make linking easy)

• Add more summaries/abstracts: Make summaries more prominent

• Add more details in the search results (e.g., cover art and summaries)

Librarian/staff survey resultsLibrarian/staff survey results

Recommended enhancements to WorldCatTotal librarian responses

Recommended enhancements to WorldCatTotal end-user responses

End-User Results: Recommended Enhancements

4

Librarian/Staff Results:

Highlighted Differences

14

1

What did we learn?

Librarians’ Perceptions of What End-Users Want

What did we learn?

Librarians’ Perceptions of What End-Users Want

Recommended enhancements to WorldCat

What did we learn?

Librarians’ Perceptions of What End-Users Want (2)

What did we learn?

Librarians’ Perceptions of What End-Users Want (2)

Recommended enhancements to WorldCat

Recommendations from librarian surveyRecommendations from librarian survey

• Merge duplicates

• Make it easier to make corrections to records (fix typos; do upgrades); “social cataloging” experiment—Wikipedia

• More emphasis on accuracy/currency of library holdings

• Enrichment—TOCs, summaries, cover art—work with content suppliers, use APIs, etc.

• More communication about what users say they want

Questions?Questions?

RESULTSRESULTSWorldCat Local Usability Testing

Usability Testing WorldCat LocalUsability Testing WorldCat Local

Test Sites from 4/2007 – present:• University of Washington• University of California

• Berkeley• Davis• Irvine

• The Ohio State University• Peninsula Library System, San Mateo, California• Free Library of Urbana• Des Plaines Public Library• Northeastern Illinois University• University of Illinois, Springfield

Usability Testing WorldCat Local – Phase 2Usability Testing WorldCat Local – Phase 2

Issues we wanted to test:

In general:• combining local, group, and global holdings;

well received by academics, needs branch information for publics

• including journal article and other licensed content;

welcomed single search for multiple kinds of content, but assumed it was comprehensive of library collection => metasearch

• simple search box; users expressed appreciation for a search box where they can “just type anything.”

• advanced search options; Used by about half of participants. Needs improvement to help user start a new search or refine an existing search.

Usability Testing WorldCat LocalUsability Testing WorldCat Local

Search Result Pages:• result ranking

Relevancy often accepted without question by most academic users. Trust library knows best how to do this for them. Sometimes surprised when an item chosen as relevant but the matching terms are not visible on search results.

• editions and FRBR, Most users do not search for a specific edition. Often the default edition chosen by WorldCat Local does not match expectations of what should be the default edition however (widest held vs. most recent).

• faceted browsing, Often noticed and praised, but not always used. Usage data indicates that they are highly used in worldcat.org. Format and Author are the two highest used facets.

• level of detail; Desire to see summary and TOC on search results. Desire to see why/where terms matched. Desire for indication of full text access, more location & availability information.

Usability Testing WorldCat LocalUsability Testing WorldCat Local

Item Details Pages:• getting print copies from local, group, and

worldwide collections; Local and group fulfillment was successful for academics. Publics were mostly concerned about what they could get at their local branch only. ILL was not of interest to publics or undergraduates, however usage indicates that ILL is highly used in academics.

• access to electronic resources; Users didn’t have a problem finding the links to electronic access, but they expect any link displayed will give them full text access. The display of WorldCat links along with local library links is problematic and confusing. When displayed with the open URL resolver, users often don’t know which is the best course of action to get to full text but are more likely to click on a link than a button to get to full text.

Usability Testing WorldCat LocalUsability Testing WorldCat Local

Item Details Pages:• collaborative and personal workflow features;

Anonymous ratings, reviews, reader recommendations, tags, and other user lists: For academics, need to know WHO is contributing this data and their reputation and identity. Reactions to anonymous user contributed data is largely negative. For publics, authority is less important. Users are more likely to contribute information.See Arnold Arcolio’s summary for more detailed information.

• navigation; Most use the browser “back” button to navigate back to results.

Usability Testing WorldCat LocalUsability Testing WorldCat Local

Item Details Pages:• Rearrangement of high-value bibliographic dataConducted surveys and card sorts to determine what information is most important when looking at an item in the catalog:

1. Title2. Author3. Summary/Abstract4. Publication Year5. Access to online content6. Publisher7. TOC8. Ability to see what is immediately avaialable9. Languages10. Editions

Detailed Record RedesignDetailed Record Redesign

Item Details Pages:

• Putting important Bib informationat the top of the page & Less important Bib Information lower on the page.

• Integration of library services on the page.

• Single place for display of electronicaccess.

• Fulfillment displayed with local/group• global availability.

Other improvements:• Grouped functionality based on user

expectations (card sorting): i.e. createda ‘toolbar’ for common functions likecite, print, email, bookmark.

• Introduced recommendations based onWorldCat data.

For more information about the usability studies we’ve done and a summary of results written by Arnold Arcolio, visit:

http://www.oclc.org/worldcatlocal/usability

Questions?Questions?

WHAT DOES IT MEAN FOR MY LIBRARY?WHAT DOES IT MEAN FOR MY LIBRARY?

Conclusion

What Does It Mean For Aligning What My Library Does with What Users Want?

What Does It Mean For Aligning What My Library Does with What Users Want?

By: David Wulffhttp://www.flickr.com/photos/dwulff/5357629/

Question 3 RevisitedQuestion 3 Revisited

•3b. My library’s cataloging practices and priorities reflect what end users want from our

catalog

• A – True

• B – False

• C – Don’t know

Two Starting PointsTwo Starting Points

1. Paying attention to what’s important about records

2. Aligning librarian and staff priorities with end user priorities• E-resources, books, media, unique digital

collections, special collection

• Redesigning work practices

Based on our two studies, what’s important about records?Based on our two studies, what’s important about records?

• Delivery information – item availability and links to content

• Basic bibliographic information (title, author, publication year, publisher, format, edition,

language)

• Evaluative content like summaries

• Fielded indexing (to support advanced search)

• Faceted browse (based on controlled forms names, topics, tags, etc.)

• Social features (for some audiences)

Follow upFollow up

• What did you think of the format of today’s session? (Webex with polling and chat)

• What did you think of the content of the presentation?

• What did you get out of the session that you can use in your work?

• What do you wish we had talked about?

Thank You!

Christie Heitkamp, heitkamc @oclc.orgKaren Calhoun, calhounk@oclc.org

Thank You!

Christie Heitkamp, heitkamc @oclc.orgKaren Calhoun, calhounk@oclc.org