Going Against Goliath 23 rd May 2010 Katrina Kurtz, MLIS Carrie Iwema, PhD, MLS Ansuman...

Going Against Goliath23rd May 2010

Katrina Kurtz, MLISCarrie Iwema, PhD, MLSAnsuman Chattopadhyay, PhDHealth Sciences Library SystemUniversity of Pittsburgh

http://www.hsls.pitt.edu/guides/genetics


Photo credit: http://www.uncabaret.com/node?page=5

Bioinformatics Tools

Answer biological questions

Made by informatics researchers

Freely available to the public

Published in scholarly journals


Bioinformatics Tools

Examples: Protein structure prediction

Sequence alignment

Databases of genes associated with disease

Protein interactions


Who Uses Bioinformatics Tools?

Biologists: Faculty researchers Graduate students Undergraduate students

Doctors who do biomedical research

Biomedical informatics researchers


Online Bioinformatics Resources Collection Manually curated

Searchable collection of bioinformatics tools

Freely available to the public

2696 resources

http://www.hsls.pitt.edu/guides/genetics/obrc/


Common Criticisms…

Why bother?

Everyone uses Google.

Google is taking over.


Is Google Actually Better?


Large amount of data

Popularity ranking

Biologists are not librarians

Research Project Hypothesis


OBRC is better than Google at Locating bioinformatics tools

Quickly Easily Successfully

Leaving users happier with: Results Overall experience

Research Project


10 biology graduate students

Three tasks on both OBRC and Google

Recorded search as they narrated

Search experience survey

Research Project

Measured:

Time spent on each search Number of search queries Number of sites visited Success rate Users’ ratings


Task 1: Protein Interaction


You are a pharmacologist interested in drugs that target proteins. Where would you find reliable information about how drugs interact with proteins, specifically information about the strength of the binding between the protein and ligand?

Task 2: Structure Prediction


Find a tool that performs protein structure prediction. Determine from the search engine how often it has been cited.

Task 3: Ovarian Cancer


Where would you look for information about all the published gene mutations implicated in ovarian cancer, in one place?

Is it faster?


Task 1 Task 2 Task 3 Average0

50

100

150

200

250

300

350

Average Task Completion Time

GoogleOBRC

Tim

e (

s)

ProteinInteraction

StructurePrediction

Ovarian Cancer

Is it easier to find results?



1

2

3

4

Average Number of Search Queries

GoogleOBRC

Nu

mb

er

of

Qu

eri

es

ProteinInteraction

StructurePrediction

Ovarian Cancer

Is it easier to find results?



1

2

3

4

Number of Sites Visited Before Success

GoogleOBRC

Nu

mb

er

of

Sit

es

Vis

ite

d

ProteinInteraction

StructurePrediction

Ovarian Cancer

Is it successful?


Task 1 Task 2 Task 3 Average0%

20%

40%

60%

80%

100%Rate of Success

GoogleOBRC

Pe

rce

nta

ge

of

Su

cc

es

sfu

l Se

arc

he

s

ProteinInteraction

StructurePrediction

Ovarian Cancer

Do users like it?


Appropriateness of Results

Ease of Use Overall Experience1

2

3

4

5

6

7

Survey Results

GoogleOBRC

Ra

tin

g

What do they say about Google?


“I was just frustrated [by Google].”

“Google was difficult to navigate with all the information on cancer.”

“[Google is] time consuming unless we know what we are searching for.”

“I realized that Google is not an

effective search tool.”

What do they say about OBRC?


“Wow, this is really direct!”

“Gives good results, but I don’t like the interface.”

“OBRC is an excellent resource, especially if

you don’t know exactly what you’re

searching for.”

“[OBRC] was much easier to locate

relevant information.”

What did we learn about OBRC? Time lag

Scopus and Google Scholar

Spell check

Long search queries

Unaware it exists


Limitations

Small sample size

Potential bias from tasks

Previous experience with Google


Future Work

Larger sample size

Other user groups

User-chosen task


Conclusions…


Searches using OBRC are: Faster Easier More successful

Users are more satisfied.

Users plan to use OBRC in the future.

Conclusions…


…libraries can and should

compete!


This study is IRB approved. Funding provided by: NLM Grant 3 T15

LM007059-23S1 Thanks to my collaborators, Dr. Carrie Iwema

and Dr. Ansuman Chattopadhyay Special thanks to Nancy Tannery and Dr.

Rebecca Crowley

Thank you!Any questions?

Katrina Kurtz Carrie Iwema Ansuman [email protected] [email protected] [email protected] 412-624-3508 412-648-6887 412-648-1297


mailto:[email protected]?subject=MLA

mailto:[email protected]

mailto:[email protected]


Going Against Goliath 23 rd May 2010 Katrina Kurtz, MLIS Carrie Iwema, PhD, MLS Ansuman...

Documents

Transcript of Going Against Goliath 23 rd May 2010 Katrina Kurtz, MLIS Carrie Iwema, PhD, MLS Ansuman...