Google Scholar as a research and evaluation tool
-
Upload
ec3metrics-spin-off -
Category
Education
-
view
250 -
download
1
description
Transcript of Google Scholar as a research and evaluation tool
Google Scholar as a research and
evaluation toolevaluation tool
Álvaro Cabezas-Clavijo
INDEX
• Goals
• GS as a research tool– Search engine
– Alerts
– Updates
– Library + Cite
• GS as a tool for evaluating research• GS as a tool for evaluating research– GS Citations (or Profiles)
– GS Metrics
– Bibliometric analysis• Data manipulation
• Comparison to WOS/ Scopus
• Conclusions
�To show GS possibilities to do research
�To show GS possibilities to evaluate research
�To point out strengths and weaknesses of GS as a source to perform bibliometric analysis
GOALS
�To introduce Google Scholar as a valid source for carrying out research assessments (with caution!) in Social Sciences and Humanities (SSH)
GS AS A TOOL FOR RESEARCHERSGS AS A TOOL FOR RESEARCHERS
• GS is a widespread tool for doing research
• Comprehensive source of academic literature
• Widely known by scholars and students
• Ease of use (just like google)
GS AS A TOOL FOR RESEARCHERS
•• Free and fast
• Links to primary source (when it is freely available on the
web or can be accessed from our institution)
• Non-selective source (covers all material which meet the
technical requirements)
• No restrictions on language
• No restrictions on document type
• No discipline/areas bias
GS AS A TOOL FOR RESEARCHERS (II)
• No discipline/areas bias
• Estimated size is 100 million documents100 million documents (Web of Science
� 50 million; Scopus � 53 million)
GS AS A TOOL FOR RESEARCHERS (III)
• GS is not just an academic search engine but a “family” of tools for doing and evaluating research
GS AS A TOOL FOR RESEARCHERS (IV)
ALERTS UPDATES
LIBRARY
+ CITE
• Set up by user just clicking on the “Create alert” link when
performing a search
• Delivered by 2-4 days if there is fresh info to show
• Authors or thematic searches can be configured (but not for
journals)
ALERTS
journals)
1 2
3
• Configure an alert for citations to particular papers or authors
(in this case, if they have already set up a profile)
ALERTS
Following
Christian’s new
papers or
citations
• Alerts about papers of our interest automatically generated by Google
• You only need to set up a GS profile. Algorithm takes into account:– Who we cite in our papers
– Who cite us in their papers
UPDATES
– Who cite us in their papers
– Titles of our papers
– Co-authors
– Content
• A red bell indicates the number of new papers that might be of your interest
UPDATESSome paper recommendations
Ok, I have published about
these topics, so I may be
interested in these papers
Ok, this paper cites me so it should
be related to my research
• Kind of reference manager
• Save references in one click
• Arrange references by using labels
• Edit references & export them (but one by one!)
• Can’t import references from other bibliographic software
LIBRARY + CITE
• Can’t import references from other bibliographic software
•Cite easily from GS
results: formats �
MLA, APA, Chicago
•Get the formatted
reference and
copy/paste it into a
Bibliography.
• By default; 2 folders: My citations (actually My papers) and Cited by me
• You can set up other
LIBRARY + CITE
• You can set up other labels/folders and save new references from GS searches
GS as a tool for evaluating researchGS as a tool for evaluating research
• GS provides different tools which can help in assessments of individuals, journals and more
GS AS A TOOL FOR EVALUATORS (II)
PROFILES METRICS
SOURCE FOR
BIBLIOMETRIC ANALYSES
• Individual Web page with publication list and basic
bibliometric indicators
• Papers are ranked (by default) according to their number
of citations
• Also links to co-authors (if they have set up a profile) and
PROFILES
• Also links to co-authors (if they have set up a profile) and
keywordskeywordsRanking of researchers
by keywords/areas are
generated through this
classification
TIP: there is always a field where you
can be the most cited author! ☺
• Easy to set up and edit
• Strongly recommended for young researchers as it can
increase their visibility
• You will get an alert (maximum 3 times a week) every time
you are cited
PROFILES
• 72,579 researchers with profile (March 2012)
(Estimates by Ortega & Aguillo, 2013)
• GS profiles-derived products
– Some Spanish institutions are ranking their researchers
according to GSC (Univ. Granada, CSIC,…) � stimulus to set up
a profile?
COMMUNICATION: 886 RESEARCHERS
SCIENTOMETRICS: 265 RESEARCHERS
CSIC RANKING OF RESEARCHERS
ACCORDING TO GSC
• Easy to set up (just search for
your papers)
• Terrific tool for comparing
researchers within a field or
department
• Automatically updated basic
bibliometric data
PROFILES
• Data can be easily manipulated
(a researcher can self-claim
non-authored papers)
• Can stimulate vanity and ego
• Can generate no fair
comparisons (for example, bibliometric data
• No restrictions on source,
language or area
• Everyone can measure their
performance (and their
colleagues')
comparisons (for example,
researchers from different
areas in a single univ)
• Can generate unfair analysis by
non-bibliometric experts
STRENGTHS WEAKNESSES
• GS service devoted to measure journal’s impact according to their h-index (est. April 2012)
• Rankings are provided by language (nine languages) areas (8) and subcategories (313) (just for English-language journals) and journal title’s words (i.e, oncology)
• Subject/area rankings are limited to top 20 journals and language
METRICS
• Subject/area rankings are limited to top 20 journals and language rankings are limited to top 100 journals.
• Calculations take into account output for 5 years (currently 2008-2012) and citations received until july 2013. A new update is expected to be released in July 2014.
• Journals with fewer than 100 papers published in 5 years or with no citation (h-index=0) are excluded
• arXiv and other repositories’ collections are also included
• Free product to compare and rank journals
• We can get impact information about non-JCR journals and about national and SSH publications
• Transparency: citations for every paper that contributes to the h-index can be checked
• Methodological inconsistencies such as comparing journals from different areas
• Lack of proper bibliographic control (duplicates, “dirty” data,…)
• No selection criteria
• No action against data manipulation
METRICS
can be checked
• High correlation with JCR Impact factor (0.82)
• Simple to use and understand by any scholar
• Can be easily replicated
• No action against data manipulation
• No info about new updates (expected to be updated yearly)
• Just top results are presented
• H index favours journals with high production
STRENGTHS WEAKNESSES
• GS is estimated to contain about 100 million documents (87% of scholarly papers on the web) Khabsa & Giles (2014)
• So, it is the largest source for bibliometric analysis that we can use
SOURCE FOR BIBLIOMETRIC
ANALISYS
can use
• GS-based products can also be used to help with research assessment– Publish or Perish (software for bibliometric analysis)
– H-Index Scholar (ranking of spanish researchers)
– Google Chrome add-ons
– …
PUBLISH OR PERISH
H-INDEX SCHOLAR
GOOGLE CHROME ADD-ONS
• Many things to sort out (unknown coverage, duplicates, “dirty” data) but GS can be useful for:
�Non-covered areas by WoS…
�Non-English research…
SOURCE FOR BIBLIOMETRIC
ANALISYS
�Non-English research…
�…Which means basically Social Sciences and Humanities
�institutions and individuals which lack access to WoS/Scopus
GS data can be easily manipulated
• Just need to upload an “academic” paper to a institutional
repository, university webpage or similar
• “Academic” means a paper with title, authors, abstract and of
course, bibliographic references
• GS will count the citations from these fake papers, modifying • GS will count the citations from these fake papers, modifying
indicators of all cited scholars and journals.
Delgado, Robinson &
Torres (2014)
• Small diferences between rankings generated by GS, WoS and Scopus
• Intensive Care Medicine � Spearman correlation of 0.99 forjournals and 0.93 for top researchers
Comparison to Wos / Scopus
JOURNALSJOURNALS
Cabezas & Delgado (2013)
• Spearman correlation for Communication Journals: 0.895 (GSM-WoS); 0.879 (GSM-Scopus)
• Spearman correlation for Economics & Business Journals: 0.718 (GS-WoS)
JOURNALS
Delgado & Repiso (2013)
Harzing & der Wal (2009)
• Easy to use, fast and free
• Largest source for assessment
purposes
• Provides information about areas
and materials not covered by
• Lack of bibliographic control
• Data can be manipulated
• No possibilities to massive info
download
• Time-consuming, data needs to
SOURCE FOR BIBLIOMETRIC
ANALISYS
and materials not covered by
traditional bibliometric products
• Rankings are very similar to
those offered by WoS / Scopus
• Possibility to build new products
based on GS data
• Time-consuming, data needs to
be “cleaned”
• GS and their services could be
closed down in any moment
STRENGTHS WEAKNESSES
ConclusionsConclusions
• Bibliometricians need to find other products to complement WoS / Scopus-based evaluations (GS, ERIH, national products,…)
• What if subscriptions to traditional databases are cancelled?– Spain paying 4-6 million Euros to Thomson Reuters yearly for
national access to WoS from public research organizations
CONCLUSIONS
– Spain paying 4-6 million Euros to Thomson Reuters yearly for national access to WoS from public research organizations
• SSH deserve to be measured (prominent rankings such as Shanghai or Leiden ignore Humanities and a huge part of Social Sciences!)
• GS tools are far from perfect but provide us with some valuable information
• Bibliometricians take usually the “easy way”; only measure what can be easily measured with databases such as WoS or Scopus
• A challenge for bibliometricians is to get the most out of the existing tools to evaluate SSH (and to develop new ones!)
• GS can be used as a complementary or alternative source to
CONCLUSIONS
• GS can be used as a complementary or alternative source to shed some light on SSH
• Use with extreme caution, especially when assessing individuals
• Keep an eye on the future: Institutions might be demanding GS-based analysis soon.
Further readingsFurther readings
• Delgado-López-Cózar, E., & Cabezas-Clavijo, Á. (2013). Ranking journals: could Google Scholar Metrics be an alternative to Journal Citation Reports and Scimago JournalRank?. Learned publishing, 26(2), 101-113.
• Delgado López-Cózar, E., Robinson-García, N., & Torres-Salinas, D. (2014). The Google Scholar Experiment: how to index false papers and manipulate bibliometric indicators. Journal of the Association for Information Science and Technology, 65(3), 446-454.
• Harzing, A. W., & Van Der Wal, R. (2009). A Google Scholar h-index for journals: An
FURTHER READINGS
• Harzing, A. W., & Van Der Wal, R. (2009). A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business. Journal of the American Society for Information Science and Technology, 60(1), 41-46.
• Khabsa, M., & Giles, C. L. (2014). The Number of Scholarly Documents on the PublicWeb. PloS one, 9(5), e93949.
• Ortega, J. L., & Aguillo, I. F. (2013). Institutional and country collaboration in an online service of scientific profiles: Google Scholar Citations. Journal of Informetrics, 7(2), 394-403.
• BLOG: Google Scholar Digest: analysis of papers dealing with GS http://googlescholardigest.blogspot.com.es/
• TWITTER: Follow the #googlescholar hashtag
• SOFTWARE: Publish or Perish http://www.harzing.com/pop.htm
• PRODUCTS:
FURTHER MATERIALS
• PRODUCTS:
• GOOGLE SCHOLAR CITATIONS http://scholar.google.com/citations
• GOOGLE SCHOLAR METRICS http://scholar.google.com/citations?view_op=top_venues&hl=en
Google Scholar as a research and
evaluation toolevaluation tool
Álvaro Cabezas-Clavijo
Thank you!