Habib NISO Altmetrics Dec 2013

13
Researcher Awareness + Perception: A year in review NISO Altmetrics Project Meeting Washington, D.C. – December 11, 2013 Michael Habib, MSLS Sr. Product Manager, Scopus [email protected] Twitter: @habib http://orcid.org/0000-0002-8860-7565

Transcript of Habib NISO Altmetrics Dec 2013

Page 1: Habib NISO Altmetrics Dec 2013

Researcher Awareness + Perception:A year in review

NISO Altmetrics Project MeetingWashington, D.C. – December 11, 2013

Michael Habib, MSLSSr. Product Manager, [email protected]: @habibhttp://orcid.org/0000-0002-8860-7565

Page 2: Habib NISO Altmetrics Dec 2013

About 1 year ago… October 2012

Background & approach:

54,442 individuals were randomly selected from Scopus

3,090 respondents completed

Representative response by country and discipline.

Error margin ± 1.5%, at 90% confidence levels

2

Adrian Mulligan, Gemma Deakin and Rebekah DuttonElsevier Research & Academic Relations

Page 3: Habib NISO Altmetrics Dec 2013

3

Impact Factor 82%

H-Index 43%

Journal Usage Factor 10%

SJR 4%

Altmetrics ???

Impact Factor is published by Thomson Reuters, Altmetrics were least well known

10/12: Most widely known by researchers

Page 4: Habib NISO Altmetrics Dec 2013

4

10/12: Most widely known by researchers

Impact Factor 82%

H-Index 43%

Journal Usage Factor 10%

SJR 4%

Altmetrics 1%

Page 5: Habib NISO Altmetrics Dec 2013

5

Impact Factor 88%

H-Index 70%

Journal Usage Factor 14%

SJR 14%

Altmetrics ???% Awareness of quality metrics (n=326, Q3 13) – From internal study by Elsevier Research & Academic Relations - Mingxin Zhou / Cat herine Fielding-Huda - October 2013

One year on? Most widely known by researchers in Q3 (n=326)

Page 6: Habib NISO Altmetrics Dec 2013

6

Impact Factor 88% (+6)

H-Index 70% (+27)

Journal Usage Factor 14% (+4)

SJR 14% (+10)

Altmetrics 5% (+4)

One year on? Most widely known by researchers in Q3 (n=326)

Page 7: Habib NISO Altmetrics Dec 2013

7

Generally metrics with the highest awareness are also considered to be the most useful

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%0%

10%

20%

30%

40%

50%

60%

70%

Altmetrics

F1000

Journal Usage Factor

h-index

EigenfactorSJR

SNIP

Impact factor

R² = 0.697191374481422

Percentage of respondents that are aware of the metric

Perc

enta

ge o

f aw

are

resp

onde

nts

that

cho

se t

he

met

ric

as o

ne o

f the

mos

t us

eful

The trendline shows the linear trend for the relationship between awareness and usage of metrics

Metrics above the line have lower levels of awareness, but are more likely to be rated as useful than the typical awareness-usage relationship

Metrics below the line have higher levels of awareness, but are less likely to be rated as useful than the typical awareness-usage relationship

* This is the 2012 data again See appendix for background and approach. Research by Elsevier Research & Academic Relations. Impact Factor is published by Thomson Reuters,

Page 8: Habib NISO Altmetrics Dec 2013

n = 326 (Q3 13) Yes10%

No90%

% Awareness of Altmetric for Scopus

32 (10%) Scopus users stated they know of Altmetric for Scopus. 25 of them think it very useful or somewhat useful.

Page 9: Habib NISO Altmetrics Dec 2013

Assessing the usefulness of potential quality metrics: by age

Significant difference between subset and total (subset higher)

Significant difference between subset and total (subset lower)

Under 36 (n=540) 36-45 (n=920) 46-55 (n=819) 56-65 (n=507) Over 65 (n=242) TOTAL (n=3,090)

Article views/downloads (for

articles) 43%

Citations from materials that are in

repositories 43%

Share in social network mentions (for

articles) 16%

Number of readers (for articles) 40%

Number of followers (for researchers) 31%

Votes or ratings (for articles) 24%

A metric that measures the contribution an

individual makes to peer review (for researchers)

28%

A score based on reviewer assessment (for

articles) 28%

Q3 Thinking about possible new measures of research productivity, how useful do you think the below would be in assessing the quality of a researcher or a research article?(By age) % Think it would be extremely/very useful

43%

49%

21%

42%

38%

35%

34%

33%

44%

45%

18%

41%

33%

24%

29%

28%

45%

41%

15%

39%

28%

22%

27%

27%

44%

41%

12%

41%

30%

22%

26%

27%

36%

37%

13%

35%

30%

19%

24%

27%

Page 10: Habib NISO Altmetrics Dec 2013

Assessing the usefulness of potential quality metrics: by region (1 of 2)

Significant difference between subset and total (subset higher)

Significant difference between subset and total (subset lower)

Africa (n=72)

APAC (n=803)

Eastern Europe (n=183)

Latin America (n=182)

TOTAL (n=3,090)

Article views/downloads (for articles) 43%

Citations from materials that are in repositories 43%

Share in social network mentions (for articles) 16%

Number of readers (for articles) 40%

Number of followers (for researchers) 31%

Votes or ratings (for articles) 24%

A metric that measures the contribution an individual makes to peer review (for researchers) 28%

A score based on reviewer assessment (for articles) 28%

Q3 Thinking about possible new measures of research productivity, how useful do you think the below would be in assessing the quality of a researcher or a research article? (By region, slide 1 of 2) % Think it would be extremely/very useful

56%

51%

26%

49%

36%

33%

40%

44%

50%

55%

27%

46%

46%

29%

35%

36%

50%

49%

19%

45%

41%

30%

28%

26%

50%

49%

21%

45%

34%

24%

32%

35%

Page 11: Habib NISO Altmetrics Dec 2013

Assessing the usefulness of potential quality metrics: by region (2 of 2)

Significant difference between subset and total (subset higher)

Significant difference between subset and total (subset lower)

Middle East (n=47) North America (n=770) Western Europe (n=1,033) TOTAL (n=3,090)

Article views/downloads (for articles) 43%

Citations from materials that are in repositories 43%

Share in social network mentions (for articles) 16%

Number of readers (for articles) 40%

Number of followers (for researchers) 31%

Votes or ratings (for articles) 24%

A metric that measures the contribution an individual makes to peer review (for

researchers) 28%

A score based on reviewer assessment (for articles) 28%

Q3 Thinking about possible new measures of research productivity, how useful do you think the below would be in assessing the quality of a researcher or a research article? (By region, slide 2 of 2) % Think it would be extremely/very useful

40%

40%

19%

43%

32%

28%

32%

34%

41%

42%

10%

36%

23%

19%

26%

26%

36%

32%

11%

36%

23%

22%

23%

22%

Page 12: Habib NISO Altmetrics Dec 2013

Michael Habib, MSLS

Sr. Product Manager, Scopus

[email protected]

Twitter: @habib

http://orcid.org/0000-0002-8860-7565

Thank you!

Page 13: Habib NISO Altmetrics Dec 2013

Background & approach

Who & when: 54,442 individuals were randomly selected from Scopus. They were approached to complete the study in October 2012. To ensure an unbiased response Elsevier’s name was only revealed at the end of the survey.

Responses: The online survey took around 15-20 minutes to complete. 3,090 respondents completed it, representing a response rate of 5.7%.

Data has not been weighted. There was a representative response by country and discipline.

Statistical testing: Error margin ± 1.5%, at 90% confidence levels. When comparing the score for main group and sub-groups we have used a Z test of proportion to identify differences between the overall average and the sub-group (90% confidence levels), when there are 30 or more responses.

13

Adrian Mulligan, Gemma Deakin and Rebekah DuttonElsevier Research & Academic Relations