What’s the Difference? Analysing the relative performance of Impact Metrics
description
Transcript of What’s the Difference? Analysing the relative performance of Impact Metrics
![Page 1: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/1.jpg)
What’s the Difference?Analysing the relative performance of Impact Metrics
David [email protected] and Computer Science
![Page 2: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/2.jpg)
2
Overview Bibliometrics Why Impact Metrics Publishing Paradigms Metrics Comparison Bibliometrics as an aid
![Page 3: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/3.jpg)
3
Citation Analysis
“Citation analysis is the examination of the frequency, patterns and graphs of citations
in articles and books.”Impact Factor
REF Impact Factor?
Article Impact Factor
PageRankCitation Count
![Page 4: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/4.jpg)
4
Why use Impact Metrics?
LibrariesResearchers
InstitutionsFunding Bodies
REF
Serials Crisis
Publishing Practices Publication
Impact
PerformanceResearch Quality
![Page 5: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/5.jpg)
5
Why/Where Researchers Publish?
Build NetworksEnhance ScienceGain Citations
Job SafetyPromotion Opportunities
Publish or Perish?
CostTime Constraints
Effort
Metrics
Target Audience
![Page 6: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/6.jpg)
6
Metrics (1)Citation Count Total number of citations towards an item
4
![Page 7: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/7.jpg)
7
Metrics (2)(Journal) Impact Factor
No of citations in year X to articles published in year X - 1 & year X – 2
Total number of citable articles from year X - 1 & year X - 2
Average No. of times an article published in the last 2 years has been cited this year
![Page 8: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/8.jpg)
8
Metrics (3)Eigenfactor (EF)
The EF of a citing JournalNumber of Citations in JournalΣEF(journal)
Weighted citations, like a weighted impact factor. Also takes into account reader time.
![Page 9: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/9.jpg)
9
Metrics (4)Article Influence Score
Journals Eigenfactor ScoreFraction of Articles Published by Journal
Designed to measure the relative importance of a JOURNAL on a per article basis. Not an indicator of
article influence.
![Page 10: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/10.jpg)
10
Similarity in ISI Metrics - JCR 2008
Citations Impact Factor No. Of Articles EigenfactorImpact Factor 0.70 No. of Articles 0.71 0.34 Eigenfactor 0.68 0.78 0.28 Article Impact Score 0.94 0.97 0.70 0.75
-1 = Complete Inverse Correlation 0 = No Relation 1 = Complete Correlation
![Page 11: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/11.jpg)
11
“15% of articles, account for 50% of the citations, and the most
cited 50% of the papers account for 90% of the citations”
P.O. Seglen. Why the impact factor of journals should not be used for evaluating research. British medical journal, 314(7079):497, 1997.
![Page 12: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/12.jpg)
12
Metrics (5)REF Pilot Metric
If a Journal is in a subject area, then the REF denominator is essentially the total Journal Impact Factor for all journals in an
area for one year only
Can still suffer from the long tail effect.
Centile distribution would help alleviate this.
No. of citations to article AAverage no. of citations to similar articles
published in the same year
![Page 13: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/13.jpg)
13
Social Science The study of citations is a social science Whenever you measure something, you
are measuring human behavior. You have to be very careful with how you
view the outputs of such a study. If you can find the typical behavior you
may be able to use a metric to represent it.
![Page 14: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/14.jpg)
14
Social Science - Article Interest Might solve a highly complex problem
which only 9 others understand Might open up new science but won’t be
the primary cited article if another gives a usage, or reviews it.
1/3rd of citations are copied verbatim (arXiv)
![Page 15: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/15.jpg)
15
Digital Repositories 900+ OA Repositories 18 Million Records In Pilot Study 73% of
Institutions have an IR registered with ROAR
46% could have sourced all their data from the repository.
46%
27%
27%
All Resources In Digital RepositorySome Resources in a Digital RepositoryUnknown or No Digital Repository
![Page 16: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/16.jpg)
16
Digital Repositories Telling your own story Find out research directions Who are the big collaborations More evidence for the expert
panel
46%
27%
27%
All Resources In Digital RepositorySome Resources in a Digital RepositoryUnknown or No Digital Repository
![Page 17: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/17.jpg)
17
More on metrics Bibliometrics gets a much wider use than
simply publications, depending on what you call it:
L. Bjorneborn and P. Ingwersen. Toward a basic framework for webometrics. Journal of the American Society for Information Science and Technology, 55(14), 2004.
![Page 18: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/18.jpg)
18
Metrics (6)PageRank
PR = PageRankL = Number of Linksn = Our Nodep = Citing Nodesα = Dampning Factor (Random Surfer)
Offers a weighted citation to help prevent against false positives. More applicable when there is no peer review.
![Page 19: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/19.jpg)
19
More Correlations (AIF)
Citation Count PageRankDownload Count (~0.4*)PageRank 0.58Hits (Authority) 0.73 0.10
* T. Brody and S. Harnad. Earlier web usage statistics as predictors of later citation impact. Journal of the American Society for Information Science and Technology, 57 (8):1060-1072, 2006.
Correlation of Rank Order Based upon data obtained from citebase.org
![Page 20: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/20.jpg)
20
Metrics Similarity
J. Bollen, H. Van de Sompel and M. A. Rodriguez. ,Towards usage-based impact metrics: first results from the MESUR project.
![Page 21: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/21.jpg)
21
Complimentary Bibliometric Methods Bibliographic Coupling– The idea of being able to relate things via links– e.g. A Paper’s field can be determined:
· From the authors departmental affiliation. · From that papers citations.· From that papers usage. · From which papers it is most often co-cited with.
The last point is the only one related to impact
![Page 22: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/22.jpg)
22
Finding Connections Usage Data Citation Data
J. Bollen. et al. Clickstream data yields high-resolution maps of
science. PLoS One, February 2009. Featured in Butler, D.
Nature News, 9 March; 2009. doi:10.1038/458135a.
![Page 23: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/23.jpg)
23
Co-Citations A bridge paper can be found if it is Co-
Cited with Publications in many disparate areas.
What I want to know is can you use these Co-Citations as an indicator of Impact?
![Page 24: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/24.jpg)
24
Summary Advise on Human Behavior Many metrics tell you different things Important to match the best metric to
your study Examine the landscape
![Page 25: What’s the Difference? Analysing the relative performance of Impact Metrics](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816449550346895dd61135/html5/thumbnails/25.jpg)
25
Summary (2) Researcher & The Institution Gathering Data
More usage of metrics– Building your institutional story– Discover new research collaborations– Early indicators