Database Web Application Usability Testing

USABILITY TESTINGDatabase Web Application

Test Information

OVERVIEW

TechSmith Morae was used on a laptop computer to conduct usability testing of the newly designed WVU Libraries Database

web application. This round of usability testing was internal and focused on WVU Libraries primary target audience.

Test Information

TARGET AUDIENCE

73%* of WVU Libraries users are undergraduate students who are 48% female or 52% male, are primarily Caucasian,

African American, Hispanic, or Asian in ethnicity, are United States citizens or are international students, are 21 years in their

average age, and are 49% from the state of West Virginia or 51% from other United States, territories, or countries.

Target audience numbers are populated from both survey polls and server statistics.*

Test Information

DATABASE USE

96%* of the WVU Libraries Database web application usage is from non-Downtown Campus Library WVU employee

computer users ‒ 64% of which are returning users, and 61% of which are from ages 18 to 34 ‒ on and off Morgantown

campus, whereas 4% of Database web application usage is from WVU employee at the Downtown Campus Library. The

Geographic Breakdown of these 96% of users is as follows: 99% from the United States; 86% from the state of West

Virginia; 77% from Morgantown, West Virginia; 1% outside the United States.

Statistics are calculated by Google Analytics.*

Test Information

TASKS1. Go to WVU Libraries’ Databases.

2. Find the database PubMed.

3. What is the earliest coverage in the database SportDiscus?

4. What is a database for the subject Marketing, that is also an

encyclopedia?

5. Your assignment is to listen to an aria from an opera. What

database will you use?

6. How many databases cover the subject of psychology?

7. Find a subject guide for engineering.

8. Go to the tutorial for the database Ad$pender.

9. What are the best databases for finding sources in biology?

10. What is the starting year of coverage for the database Web of

Science?

11. How can you get help with database selection or searching?

12. You need a brief video clip on chemistry lab safety. Where

would you find it?

13. Find 2 databases that contain statistics for market research.

14. What is a database for Theater that is contains Images?

15. How many primary sources can you find for West Virginia

databases?

Test Information

*(SUS) QUESTIONS1. I think that I would like to use this system frequently

2. I found the system unnecessarily complex

3. I thought that the system was easy to use

4. I think that I would need the support of a technical person to

be able to use this system

5. I found the various functions in this system were well

integrated

6. I thought there was too much inconsistency in this system

7. I would imagine that most people would learn to use this

system very quickly

8. I found the system very cumbersome to use

9. I felt very confident using the system

10. I needed to learn a lot of things before I could get going with

this system

* System Usability Scale (SUS)

Erica Erick Kayla Rebecca Shawn

Alex Brandon Chris Christian Dave

Test Information

PARTICIPANTS

Test Results:

All Tasks

AVERAGE

TIME* Average time includes delivery of tasks.

Test Results:

All Tasks

SUCCESS

RATES* Percentages based on a 0 to 3 scoring system.

Test Results:

All Tasks

ERROR

RATE* Error rate is a standard deviation based

direct path to information as it is known to

test administrator, and the rate of the user

encountering a functionality or system error.

Test Results:

All Tasks

SUS

SURVEY* Results are based on survey results taken

after usability testing, and are concerned

with the Database web application interface

and functionality.

Test Results

TASK 1Requested Task: Go to WVU Libraries’ Databases.

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

0.74 min.

90%

0.00

10%

Participant Observations

• 10% of users couldn’t find the Databases page

• 40% of users eventually located the Databases page by utilizing the homepage

search or ‘Databases by Subject’ link

• 50% of users utilized a search engine for ‘WVU Libraries Databases’

Suggestions

• Better incorporate Databases on WVU Libraries’ homepage

Videos

• https://testing.lib.wvu.edu/usability4/task1a.mp4

• https://testing.lib.wvu.edu/usability4/task1b.mp4

https://testing.lib.wvu.edu/usability4/task1a.mp4

https://testing.lib.wvu.edu/usability4/task1b.mp4

Test Results

TASK 2Requested Task: Find the database PubMed.

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

1.17 min.

90%

0.00

10%


• 30% of users couldn’t find the ‘Database Titles’ link

• 20% of users aren’t seeing the A-Z section

• 40% of users expressed interest in / wanted to search so they either went back to

the libraries homepage, or used the browser’s default <Ctrl> + <F> to search.

• 50% of users utilized a search engine for ‘WVU Libraries Databases’

Suggestions

• Incorporate predictive search for Databases on the homepage and on the

Databases web application page

• Move the subject/title/help navigation to the left, pluralize ‘Databases’, add the word

‘by’, and make it look more like navigation options (instead of being grayed-out)

• Add ‘by Subject’ as a facet

• Add padding and emphasis to A-Z section to make it stand out more

• Eliminate “by title ‘Letter’” from results, it’s confusing to read

Videos





Test Results

TASK 3Requested Task: What is the earliest coverage in the database SportDiscus?

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

1.59 min.

60%

0.20

40%


• 100% of users don’t click or see ‘More Information’ in the databases results, 80%

never go back to this, and only 20% eventually clicked on ‘More Information’ just

because the task required them to do so

• 30% are still not seeing ‘Database Titles’

Errors

• Empty Help with a bullet under ‘More Information’

• Facets appear as a gray box for search results

• New/Trial databases don’t show in breadcrumbs

Suggestions

• Possibly employ an accordion as a button instead of a blue ‘More Information’ link

• Fix information in databases ‘More Information’

• Add facets for search, new, and trial

Videos





Test Results

TASK 4Requested Task: What is a database for the subject Marketing, that is also an

encyclopedia?

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

1.01 min.

10%

0.00

90%


• 40% of users aren’t seeing / understanding resource types as faceting

• 20% of users are not understanding the difference between Databases by subject

and titles

• 20% of students are still going back to the homepage to search

• As users spend more time with the system they are learning how to use the system

as a successful NUI (natural user interface)

Suggestions

• Facets should only populate under resource types dynamically as they are

applicable and remain to what’s pre and post breadcrumb faceting

• Add hinting for important functionality on web applications

Videos





Test Results

TASK 5Requested Task: Your assignment is to listen to an aria from an opera. What

database will you use?

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

1.07 min.

100%

0.00

0%


• 20% of users still aren’t seeing / understanding resource types as faceting

• 10% of users are still not understanding Databases by subject versus titles

• 20% went to resource type ‘Video and Sound’ first, and 30% went to the subject of

Art while 70% went to the subject of Music

• 20% of users are still going back to the homepage to search

• 20% of users used the Music tag

Suggestions

• Create visual understanding with color for facets in the left-hand, and color code

(and possibly use different icons) for tags based on resource type versus subject

• Reduce the amount of tags as is appropriate

Videos





Test Results

TASK 6Requested Task: How many databases cover the subject of psychology?

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

0.46 min.

90%

0.00

10%


• At this point all users were familiar with the faceting, subject, and title filtering of the

database web application

• 10% of users misread the pagination indicator, the other 90% performed the task

with ease

Suggestions

• Possibly add wording to pagination such as ‘Viewing results 1-10 out of 25 total’ or

incorporate how many pages / results on pages

Videos





Test Results

TASK 7Requested Task: Find a subject guide for engineering.

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

0.66 min.

80%

0.00

20%


• 40% of users don’t know what a subject guide was, or why it was needed

• 10% of users used the homepage to try and find a subject guide

Suggestions

• On the macro level the WVU Libraries website could better explain the functionality

of Research Guides living at libguides.wvu.edu (which is confusing), and that

underneath Research Guides there are Subject Guides, Course Guides, Style

Guides, Finding Guides, etc.

Videos





Test Results

TASK 8Requested Task: Go to the tutorial for the database Ad$pender.

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

1.38 min.

60%

0.00

40%


• 10% looked in ‘Help’

• 20% of users went back to the homepage to search

• 80% went straight to databases instead of ‘More Information’

• Users feel there are some databases that have too many tags

Suggestions

• Database title search states: By title ‘A’ Database Results: By title ‘A’, which is

confusing and should read instead just as ‘Database by Title: A’

Videos





Test Results

TASK 9Requested Task: What are the best databases for finding sources in biology?

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

0.99 min.

100%

0.10

0%


• 100% of users didn’t see or utilize “Librarian Top Picks’

• 40% of user utilized the ‘Primary Sources’ facet

• 20% of users went to Subject Guide for Biology

• 20% of users utilized the ‘Statistics’ facet

Error

• When users click on the ‘Statistics’ resource type facet for the database subject of

Biology there are no results

Suggestions

• ‘Librarian Top Picks’ need to be used across subject, or defaults should stick

• ‘Librarian Top Picks’ should have a different aesthetic treatment to make them

visible when they appear

Videos





Test Results

TASK 10Requested Task: What is the starting year of coverage for the database Web of

Science?

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

0.80 min.

100%

0.00

0%


• 30% of users said 1955 based on ‘More Information’

• 70% of users went straight to database and overlooked ‘More Information’, and

71% of those users said 1864 based on information in Web of Science

• 10% of users searched on the homepage

Videos





Test Results

TASK 11Requested Task: How can you get help with database selection or searching?

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

0.84 min.

100%

0.10

0%


• 10% of users would go to ‘Ask A Librarian’ green tab

• 20% of users would go ask a librarian in person

• 20% of users would go to ‘Contact Us’

• 30% of users would go to ‘Help’

Videos



• https://testing.lib.wvu.edu/usability4/task11c.mp4



https://testing.lib.wvu.edu/usability4/task11c.mp4

Test Results

TASK 12Requested Task: You need a brief video clip on chemistry lab safety. Where would

you find it?

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

1.72 min.

20%

0.67

80%


• 80% of users tried to use the ‘Video and Sound’ resource type facet

• 30% of those users eventually went out to the homepage to search, 20% stated

they would search in the subject guide, and 50% said they would go to YouTube

• 20% of users used the ‘Handbook’ tag to determine a database

Error

• 80% of users clicked on the ‘Video and Sound’ resource type facet for the

database subject of Chemistry, and there were no results

• Users can compound resource type facets and get no results

Videos



• https://testing.lib.wvu.edu/usability4/task12c.mp4



https://testing.lib.wvu.edu/usability4/task12c.mp4

Test Results

TASK 13Requested Task: Find 2 databases that contain statistics for market research.

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

0.95 min.

90%

0.00

10%


• 20% of users read database description to resolve task

• 20% of user went to homepage to search

Videos





Test Results

TASK 14Requested Task: What is a database for Theater that is contains Images?

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

0.60 min.

90%

0.00

10%

Participant Observation

• 10% of users didn’t understand faceting and filtering

Suggestions

• Possibly develop facets to work in reverse order, so by resource type then by

subject.

Videos





Test Results

TASK 15Requested Task: How many primary sources can you find for West Virginia

databases?

Average Time:

Completion Rate:

Error Rate:

Failure Rate:

0.84 min.

80%

0.10

20%


• 20% of users used tags to find primary resources or West Virginia databases

• 20% of users went to the homepage to search

Error

• Homepage search results didn’t yield database subject search results

Videos





The WVU Libraries’ database web application was redesigned as a

NUI, or Natural User Interface. In computing a NUI is the common

parlance used by designers and developers of human-machine

interfaces to refer to a user interface that is effectively invisible, and

remains invisible as the user continuously learns increasingly complex

interactions.

These usability test results demonstrates that as a user interacts with

the WVU Libraries’ database web application, the user continuously

learns how to use the interface. More so, one can see the decrease in

percentages from one task to another in regards to users not seeing or

knowing how the database web application interface functions.

Indeed, a NUI is ideal since our current analytics show that 96% of the

WVU Libraries database web application usage is from non-Downtown

Campus Library librarian/staff computer users on and off Morgantown

campus. Ergo, the database web application’s return users (ages 18 to

34) are majorly using this web application without assistance of any

kind.

Furthermore, according to this usability test 11 out of 15 tasks had a

failure rate ranging from 10% to 80%, and only 10 out of 15 tasks

had an easy completion rate ranging from 10% to 50%; these rates

can be dramatically improved.

Conclusion

SUMMARY

The following changes to analytics, content, development, and user

interface should dramatically eliminate errors, reduce the percentage of

failure rates, and decrease the amount of time for task completion

Analytics

- Install filters for librarian / staff traffic at the Evansdale and HSC

libraries

User Interface

- Better incorporate Databases on WVU Libraries’ homepage

- Move the subject/title/help navigation to the left, and possibly change

color

- Add padding and emphasis to A-Z section to make it stand out more

- Possibly employ an accordion as a button instead of a blue ‘More

Information’ link, and remove link from database title

- Create visual understanding with color for facets in the left-hand, and

color code (and possibly use different icons) for tags based on resource

type versus subject

- ‘Librarian Top Picks’ should have a different aesthetic treatment to

make them visible when they appear

- Possibly add # to simulate pagination reload

Content

- Pluralize ‘Databases’, add the word ‘by’, and make it look more like

navigation options (instead of being grayed-out)

- Fix information in databases ‘More Information’

- Add hinting for important functionality on web applications

- Reduce the amount of tags as is appropriate

- Possibly add wording to pagination such as ‘Viewing results 1-10 out of

25 total’ or incorporate how many pages / results on pages

- Database title search states: By title ‘A’ Database Results: By title ‘A’,

which is confusing and should read instead just as ‘Database by Title:

A’

Development

- Incorporate predictive search for databases on the homepage and on

the Databases web application page

- Add ‘by Subject’ as a facet

- Add facets for search, new, and trial

- Facets should only populate under resource types dynamically as they

are applicable and remain to what’s pre and post breadcrumb faceting

- ‘Librarian Top Picks’ need to be used across subject, or defaults should

stick

- Possibly develop facets to work in reverse order, so by resource type

then by subject.

Conclusion

ENHANCEMENTS

“Pay attention to what users do, not what they say.”- Jakob Nielsen

THANK YOU

Database Web Application Usability Testing

Education

Transcript of Database Web Application Usability Testing