December 10, 2009

Eunice Wong – Subject Librarian of Engineering

Diana Chan – Associate University Librarian

Analyze and Improve

Your Citation Counts

Part II:

Recent Developments in Citation Metrics

BibliometricsTimeline

The Metrics

Average Citation Rate for Papers Published by Field

Sources on Citation Counts

Comparing WS, Scopus and GS

Knowledge Harvesting and Scopus Author ID Form

Metrics on Quantity and Impact

How to Improve Your Citation Counts

2

Citation Indexing and Analysis

3

Garfield, E. (1970). Citation indexing for studying science. Nature, 227(5259), 669-671.

Bibliometrics Time Line

4

Source: Research Trends (2007) no. 1 & 2

Google Scholar2007

The Metrics

Publications (#, joint AUs)

Citations (minus self-citations)

Journal Impact Factor

H-Index

G-Index

Other metrics

5

A Classic Paper

6

Marks technological, methodological or theoretical advances

Receives many more than the expected number of citations

for their area

May persist for many years

Laemmli, U. K. (1970). Cleavage of structural proteins

during the assembly of the head of bacteriophageT4. Nature,

227(5259), 680-685. – cited over 85,114 times since 1996

What is a citation worth?

Gomez-Mejia and Balkin (1992) estimated that the marginal

dollar value of a single citation to articles in 21 top-tier

management journals

at US$192 in 1988,

with a future value of US$1,522, and

a cumulative annuity of US$13,350 (HK$100,000) in 2011.

7

Average Citation Rates for Papers

published by field, 1999-2009

Papers Published in: 1999 2003 2006 2009 All Years

All Fields 18.24 13.24 6.52 0.26 10.06

Biology & Biochemistry 29.24 20.79 9.62 0.36 16.35

Chemistry 16.68 13.31 7.13 0.32 10.1

Computer Science 6.84 4.32 1.54 0.1 3.25

Economics & Business 10.67 7.56 3.08 0.13 5.37

Engineering 7.28 5.66 2.86 0.14 4.18

Mathematics 6.38 4.26 2.13 0.12 3.17

Physics 14.15 10.35 5.73 0.25 8.45

Psychiatry/Psychology 19.97 14.11 6.3 0.23 10.21

Social Sciences, general 8 5.95 2.91 0.14 4.26

Source: Essential Science Indicators8

Sources with Citation CountsTypes Examples

Commercial

Citation Databases

Web of Science –SCI, SSCI, A&HI

Scopus

Commercial

Databases

Academic Search Premier

ACM Digital Library

JSTOR

ProQuest

Science Direct

SciFinder Scholar,

MathSciNet,

Chinese Citation Db, Chinese Science Citation Db (not

available at HKUST)

Web Tools Google Scholar

Scholarly Websites RePec

ACM Portal

Citeseer9

Other Commercial Databases

ScienceDirect

ACM Digital Library

PsycINFO

10

Other Commercial Databases

11

Academic Search

Premier (Ebscohost)

ProQuest

Other Commercial Databases

JSTOR

Google Scholar12

Scholarly Websites

13

RePEc

14

Citebase

15

Studies on Citation Counts from

Different Sources

16

Studies Comparing Citation Counts Findings

Whitley (2002) Compared 30 chemistry researchers’ works

indexed in Chemical Abstracts and SCI

during 1999-2001

23% unique to CA

17% unique to SCI

Goodrum et.al.

(2001) and

Zhao & Logan

(2002)

Compared citations of the top-25 cited

authors in computer science indexed in

CiteSeer with SCI

44% overlap between the

two and concluded

CiteSeer and SCI were

complementary to each

other

Pauly and

Stergiou (2005)

compared citation counts between WoS and

GS for papers in math, chem, physics,

computer sciences, molecular biology,

ecology, fisheries, oceanography,

geosciences, economics and psychology

A good correlation in

citation counts between

the 2 sources and

suggested GS can

substitute for WoS

Not to rely on a single counting source

Comparing WoS, Scopus and GSMeho & Yang ‘s Study

(2007)

• Covering citations of over

1,000 scholarly works of

15 faculty members in

Information Science at

Indiana University between

1996 and 2005

17

Author Web of Science 1991-

(1970-)

Scopus Google Scholar

Author A

Business

Articles = 27 (36)

Cited by = 519 (626)

H-index=14 (16)

Articles = 30

Cited by =508

H-index=14

Articles = 77

Cited by = 2457

H-index= 23

Author B

Business

Articles = 17 (40)

Cited by = 131 (912)

H-index= 7 (17)

Articles = 25

Cited by = 190

H-index= 8

Articles = 107

Cited by = 2018

H-index= 22

Author C

Engineering

Articles = 62 (162)

Cited by = 1345 (1747)

H-index= 18 (23)

Articles = 228

Cited by = 1786

H-index= 18

Articles = 259

Cited by = 9288

H-index= 39

Author D

Engineering

Articles = 19 (45)

Cited by = 608 (2090)

H-index= 13 (19)

Articles = 73

Cited by = 1649

H-index= 9

Articles = 90

Cited by = 3506

H-index= 24

18

1. Many publication and citation counts of HKUST researchers are missed out

from Web of Science due to a shorter subscription period (1991-).

2. SBM and HSS researchers are under-represented in Scopus and Web of Science

3. Junior researchers are under-represented in Scopus and Web of Science

Knowledge Harvesting and

Citation Counting

A comprehensive way of doing is to harvest from:

Web of Science (done)

Scopus (next step)

OCGA and TTC databases

Other Databases (Ebscohost, IEEE Xplore, ProQuest, etc.)

19

HKUST Scholarly Publications Index

HKUST Scholarly Publications Index

http://lbapps.ust.hk/hkir/handle/9999/116999

20

21

1. Fill in Scopus Author ID

2. Retrieve articles

3. Identify problems if any

4. Use the submission form for

missing articles

5. Select other databases that you

may find your publications

http://lbxml.ust.hk/ir/scopusid.pl

Metrics on Output Quantity

22

Hirsch Index Egghe Index

Quantity

measure

H-index – e.g. 7

H (7) papers of the author that

have H (7) citations or more

G-index – e.g. 20

G (20) papers that received

GxG (400) citations or more

• H-index: Publications ranked in descending order

by "Times Cited". H is equal to the number of

papers in the list that have h or more citations.

• Problems with h-index -Lutz Bornmann (2008) :

o Insensitive to highly cited papers

o Dependent on career length

o Will never go down, so cannot indicate periods of

inactivity

Rank Times Cited Yr

1 48 2000

2 44 1999

3 43 2000

4 30 2004

5 13 2000

6 12 2002

7 11 2004

8 6 2005

19 0 …

Metrics on Impact Measure

23

A- index M-index

Impact measure A-index – e.g. 35

The average number of

citations of publications

in the h-index of 25 is

35

M-index – e.g.

The median number of

citations received by

papers in the h-index of

25 is 38

H index and its variants

Source: Lutz Bornmann

24

Improve Your Citation Counts

25

Publish in high-impact journals -Judge (2007), Glass (1955),

Frank and Cook (1995)

Make your publications visible

Your name

Your affiliation

Your abstract

Your keywords

Your English

Your publication list

Make them openly accessible

Ways to Improve Counts Not As Good Recommended

Publish in Top journals

(Journals with high IF, or a

high average citation rate)

IF of Bio-medical Materials and

Engineering=0.446

IF of Annual Review of Biomedical

Engineering=10.789

Use a middle initial Chan, C. (30 authors in Scopus

affiliated with HKUST)

Chan, C.M. (4 authors in Scopus affiliated

with HKUST)

Use a consistent name Chan, Philip Ching Ho

Chan, P. C.

Chan, P. C. H.

Chan, P.

Chan, Philip C.

Chan, Philip C.

Register at databases or sites

with author authority lists

Scopus Author ID=7406034643

Web of Science - ResearcherID

Add HKUST and department

name

UST, Hong Kong Hong Kong University of Science &

Technology,

Department of Electrical and Electronic

Engineering,

Hong Kong,

Peoples Republic of China

26

Ways to Improve Counts

“Good keywords in abstracts” Brian Faith, Editor-in-Chief , Ecological Modelling

“People are using search engines to find papers, so…”

“Well-written, clear English

especially in abstracts”

Brian Faith, Editor-in-Chief , Ecological Modelling

“People are using search engines to find papers, so…”

Add papers to open access

repository such as HKUST IR

or SSRN

Will be harvested by search engines such as GS and be included in

their citation counts

Post a list of your full

publication on your website

Self promotion

Subscribe to Citation Alerts Web of Science: Create Citation Alert to a particular article

Scopus: Email Alert when an author/article is cited

Make corrections Inform WS and Scopus for errors in affiliation & citations

Use other citation sources for

items not indexed in WOS &

Scopus

Google Scholar, SciFinder Scholar, MathSciNet, Chinese Citation

Database, Chinese Science Citation Database

Maintain an updated file of

citations as they are discovered

Keep a file of your own citations. Only you can tell if those citations

are to your work

27

28

Make your publication openly accessible

in HKUSTIR

Open Access on

Increasing Readability and Citations

29

http://repository.ust.hk

30

31

Finland has climbed up the rankings

Source: ResearchTrends, May 2009 32

Kuopio University in Finland The University has improved its ranking by:

Focusing on strategic research and supporting this with funding(e.g. biological sciences and medicine)

Making publishing papers in international and high-quality journals a priority Using bibliometric tools to find out where to publish (e.g. Annals of

Internal Medicine, Cell, Nature, Nature Genetics, Lancet, Proceedings of the National Academy of Sciences of the United States of America)

Using ranking lists as tools in evaluation

Library in acquiring the best possible e-journal collections and promoting the use to researchers

Adopting international collaboration throughout the university According to Jamo Saarti, Kuopio University

33

The Case of Hong Kong

Source: Incites.com <http://in-cites.com/countries/hongkong.html>34

A Subject Guide on Research Impact

35http://lbguides.ust.hk/subject-guide/2-Research-Impact

For any questions or feedback, please email lbeunice@ust.hk or dianachan@ust.hk

References Balaram, P. (2004) “The Shanghai Rankings”, Current Science, Vol.86, No.10

Bornmann, L., Mutz, R., and Daniel, H.D. (2008) “Are there better indices for evaluation purposes than the h

index? A comparison of nine different variants of the h index using data from biomedicine”, Journal of the American

Society for Information Science and Technology, Vol. 59, No. 5, pp. 830–837.

Garfield, Eugene. (1970) “Citation Indexing for Studying Science.” Nature 227, pp. 669-671

Goodrum, A.A., McCain, K.W., Lawrence, S. & Giles, C.L. (2001) “Scholarly publishing in the Internet age: A

citation analysis of computer science literature” , Information Processing & Management, Vol.37 No.5, pp. 661-675

Gomez-Mejia, L.R., & balkin, D.B. (1992) “Determinants of faculty pay: An agency theory perspective”, Academy

of Management Journal,Vol. 35:, pp.921-955.

Harzing, A.K.andWal, R.D. (2008) “Google Scholar as a new source for citation analysis”, Ethics in Science and

Environmental Politics, Vol.8, pp. 61-73.

Jacso, P. (2005) “As we may search – comparison of major features of the Web of Science, Scopus, and Google

Scholar citation-based and citation-enhanced databases” Current Science Vol. 89, No. 9, pp.1537-1547

Jacso, P. (2005) “Google Scholar: the pros and the cons” Online Information Review Vol. 29, pp.208-214.

Judge, T.A. et.al. (2007) “What causes a management article to be cited – article, author or journal?” Academy of

Management JournalVol.50 No.3, pp.491-506

Leung, K. (2007) “The glory and tyranny of citation impact: an east Asian perspective” Academy of Management

JournalVol. 50 No. 3 pp.510-513

36

References Meho, L.& Yang, K. (2007) “Impact of data sources on citation counts and rankings of LIS faculty: Web of Science

versus Scopus and Google Scholar”, JASITVol. 58, No. 13, pp.2105-2125

Pagell, R. (2008) “From Bean Counting to Academic Accountability”

http://library.smu.edu.sg/aboutus/Bibliometricscomplete.pdf

Pauly, D., & Stergiou, K.I. (2005) “Equivalence of results from two citation analyses: Thomson ISI’s citation index

and Google’s Scholar service”, Ethics in Science and Environmental Politics, 33-35

Research Trends http://info.scopus.com/researchtrends

ScienceWatch Online http://sciencewatch.com

Soong, S.C. (2009) “Measuring citation advantages of open accessibility” D-Lib Magazine Vol.15 No.11/12, 3p.

Whitley, K.M. (2002). “Analysis of SciFinder Scholar and Web of Science citation searches.” Journal of the American

Society for Information Science and Technology , Vol.53, No.14, pp.1210-1215

Zhao, D.Z., & Logan, E. (2002). “Citation analysis using scientific publications on the web as data source: A case

study in the XML research area.” Scientometrics , Vol. 54 No.3, pp. 449-472

37