Post on 05-Jan-2016
Going Against Goliath23rd May 2010
Katrina Kurtz, MLISCarrie Iwema, PhD, MLSAnsuman Chattopadhyay, PhDHealth Sciences Library SystemUniversity of Pittsburgh
http://www.hsls.pitt.edu/guides/genetics
Photo credit: http://www.uncabaret.com/node?page=5
Bioinformatics Tools
Answer biological questions
Made by informatics researchers
Freely available to the public
Published in scholarly journals
http://www.hsls.pitt.edu/guides/genetics
Bioinformatics Tools
Examples: Protein structure prediction
Sequence alignment
Databases of genes associated with disease
Protein interactions
http://www.hsls.pitt.edu/guides/genetics
Who Uses Bioinformatics Tools?
Biologists: Faculty researchers Graduate students Undergraduate students
Doctors who do biomedical research
Biomedical informatics researchers
http://www.hsls.pitt.edu/guides/genetics
Online Bioinformatics Resources Collection Manually curated
Searchable collection of bioinformatics tools
Freely available to the public
2696 resources
http://www.hsls.pitt.edu/guides/genetics/obrc/
http://www.hsls.pitt.edu/guides/genetics
http://www.hsls.pitt.edu/guides/genetics
http://www.hsls.pitt.edu/guides/genetics
http://www.hsls.pitt.edu/guides/genetics
Common Criticisms…
Why bother?
Everyone uses Google.
Google is taking over.
http://www.hsls.pitt.edu/guides/genetics
Is Google Actually Better?
http://www.hsls.pitt.edu/guides/genetics
Large amount of data
Popularity ranking
Biologists are not librarians
Research Project Hypothesis
http://www.hsls.pitt.edu/guides/genetics
OBRC is better than Google at Locating bioinformatics tools
Quickly Easily Successfully
Leaving users happier with: Results Overall experience
Research Project
http://www.hsls.pitt.edu/guides/genetics
10 biology graduate students
Three tasks on both OBRC and Google
Recorded search as they narrated
Search experience survey
Research Project
Measured:
Time spent on each search Number of search queries Number of sites visited Success rate Users’ ratings
http://www.hsls.pitt.edu/guides/genetics
Task 1: Protein Interaction
http://www.hsls.pitt.edu/guides/genetics
You are a pharmacologist interested in drugs that target proteins. Where would you find reliable information about how drugs interact with proteins, specifically information about the strength of the binding between the protein and ligand?
Task 2: Structure Prediction
http://www.hsls.pitt.edu/guides/genetics
Find a tool that performs protein structure prediction. Determine from the search engine how often it has been cited.
Task 3: Ovarian Cancer
http://www.hsls.pitt.edu/guides/genetics
Where would you look for information about all the published gene mutations implicated in ovarian cancer, in one place?
Is it faster?
http://www.hsls.pitt.edu/guides/genetics
Task 1 Task 2 Task 3 Average0
50
100
150
200
250
300
350
Average Task Completion Time
GoogleOBRC
Tim
e (
s)
ProteinInteraction
StructurePrediction
Ovarian Cancer
Is it easier to find results?
http://www.hsls.pitt.edu/guides/genetics
Task 1 Task 2 Task 3 Average0
1
2
3
4
Average Number of Search Queries
GoogleOBRC
Nu
mb
er
of
Qu
eri
es
ProteinInteraction
StructurePrediction
Ovarian Cancer
Is it easier to find results?
http://www.hsls.pitt.edu/guides/genetics
Task 1 Task 2 Task 3 Average0
1
2
3
4
Number of Sites Visited Before Success
GoogleOBRC
Nu
mb
er
of
Sit
es
Vis
ite
d
ProteinInteraction
StructurePrediction
Ovarian Cancer
Is it successful?
http://www.hsls.pitt.edu/guides/genetics
Task 1 Task 2 Task 3 Average0%
20%
40%
60%
80%
100%Rate of Success
GoogleOBRC
Pe
rce
nta
ge
of
Su
cc
es
sfu
l Se
arc
he
s
ProteinInteraction
StructurePrediction
Ovarian Cancer
Do users like it?
http://www.hsls.pitt.edu/guides/genetics
Appropriateness of Results
Ease of Use Overall Experience1
2
3
4
5
6
7
Survey Results
GoogleOBRC
Ra
tin
g
What do they say about Google?
http://www.hsls.pitt.edu/guides/genetics
“I was just frustrated [by Google].”
“Google was difficult to navigate with all the information on cancer.”
“[Google is] time consuming unless we know what we are searching for.”
“I realized that Google is not an
effective search tool.”
What do they say about OBRC?
http://www.hsls.pitt.edu/guides/genetics
“Wow, this is really direct!”
“Gives good results, but I don’t like the interface.”
“OBRC is an excellent resource, especially if
you don’t know exactly what you’re
searching for.”
“[OBRC] was much easier to locate
relevant information.”
What did we learn about OBRC? Time lag
Scopus and Google Scholar
Spell check
Long search queries
Unaware it exists
http://www.hsls.pitt.edu/guides/genetics
Limitations
Small sample size
Potential bias from tasks
Previous experience with Google
http://www.hsls.pitt.edu/guides/genetics
Future Work
Larger sample size
Other user groups
User-chosen task
http://www.hsls.pitt.edu/guides/genetics
Conclusions…
http://www.hsls.pitt.edu/guides/genetics
Searches using OBRC are: Faster Easier More successful
Users are more satisfied.
Users plan to use OBRC in the future.
Conclusions…
http://www.hsls.pitt.edu/guides/genetics
…libraries can and should
compete!
http://www.hsls.pitt.edu/guides/genetics
This study is IRB approved. Funding provided by: NLM Grant 3 T15
LM007059-23S1 Thanks to my collaborators, Dr. Carrie Iwema
and Dr. Ansuman Chattopadhyay Special thanks to Nancy Tannery and Dr.
Rebecca Crowley
Thank you!Any questions?
Katrina Kurtz Carrie Iwema Ansuman Chattopadhyaykak59@pitt.edu iwema@pitt.edu ansuman@pitt.edu 412-624-3508 412-648-6887 412-648-1297
http://www.hsls.pitt.edu/guides/genetics