Post on 12-Nov-2014
description
https://www.asis.org/rdap/
LastName, Title
The impact of data reuse: a pilot study of five measures
Kathleen FearApril 5, 2013
What is reuse impact?
• Scholarly contribution through producing data– Recognized and rewarded through publications and
publication metrics• Scholarly contribution through sharing data– Recognized and rewarded through
…
?
What can we do to measure and communicate the scholarly contribution a data producer
makes when their data is reused?
Pilot study of 5 measures
• Identify a set of social science datasets
• Find out how much and in what contexts they have been reused
• Demonstrate a variety of measures– Do they all come out the same?– Or do different measures highlight different data?
Sample set: 273 studies
38%
45%
11%
3% 3%Author Type
Single author
Two or more authors
Government
Non-governmental institution
Media organization
81%
19%
Processed vs. unprocessed
Processed
Unprocessed
38%
32%
30%
Release Date
2000
2001
2002
Reuse citations
• How many times has the data been reused?
• ICPSR Bibliography of Data-Related Literature– Excluded: publications by study authors and
research team members, literature reviews, commentary
Some data is reused a lot
Lots of data is reused a little
Even more is reused not at all
Study ID Study Name Reuse count
6693 National Comorbidity Survey: Baseline (NCS-1), 1990-1992 175
2884 National Treatment Improvement Evaluation Study (NTIES), 1992-1997 32
3160Project on Policing Neighborhoods in Indianapolis, Indiana, and St. Petersburg, Florida, 1996-1997
34
2851
Hispanic Established Populations for the Epidemiologic Studies of the Elderly, 1993-1994: [Arizona, California, Colorado, New Mexico, and Texas]
24
2258 Drug Abuse Treatment Outcome Study (DATOS), 1991-1994: [United States] 19
How high-quality are the data’s reuse publications?
Study ID Study Name Secondary Impact
6693 National Comorbidity Survey: Baseline (NCS-1), 1990-1992 83
2258 Drug Abuse Treatment Outcome Study (DATOS), 1991-1994: [United States] 21
2851
Hispanic Established Populations for the Epidemiologic Studies of the Elderly, 1993-1994: [Arizona, California, Colorado, New Mexico, and Texas]
20
2884 National Treatment Improvement Evaluation Study (NTIES), 1992-1997 19
2778 Gambling Impact and Behavior Study, 1997-1999: [United States] 18
How broadly or narrowly is the data reused?
Diversity
• Variety, balance, disparity among reuse publications + disparity between citing disciplines and data discipline
€
δ = pi∑ pjdij + pkdk∑
DataID Study Title Diversity
3190 National Organizations Survey (NOS), 1996-1997 2.5000
3337Evaluation of the Gang Resistance Education and Training (GREAT) Program in the United States, 1995-1999
2.2230
2976Police Stress and Domestic Violence in Police Families in Baltimore, Maryland, 1997-1999
2.1794
3334Aging, Status, and Sense of Control (ASOC), 1995, 1998, 2001 [United States]
2.0313
2993 Reintegrative Shaming Experiments (RISE) in Australia, 1995-1999 2.0000
How large is the publication network stemming from the data?
Downloaders
• How many individuals download the data?
• Unique users identified by email address or IP address
Study ID Study Name Downloaders
6693 National Comorbidity Survey: Baseline (NCS-1), 1990-1992 3787
2790World Values Surveys and European Values Surveys, 1981-1984, 1990-1993, and 1995-1997
3393
2778 Gambling Impact and Behavior Study, 1997-1999: [United States] 2637
3088 Alcohol and Drug Services Study (ADSS), 1996-1999: [United States] 2478
3355 Recidivism of Prisoners Released in 1994 2209
Comparing metricsReuse Citations Sec. Impact Diversity Downloaders
6693 6693 3190 66933160 2258 3337 27782884 2851 2976 30882851 2884 3334 28332258 2778 3052 33342976 3385 2993 33372833 3160 3323 22582778 3337 2778 29763337 2833 3163 32123385 3023 3002 3002
Reuse count
Secondary impact Diversity
Downloaders
Study ID Study Name Diver-
sityReuse Count
Sec. Impact
Down-loaders
3052
Risk Factors for Violent Victimization of Women in a Major Northeastern City, 1990-1991 and 1996-1997
5 18 26 27
3355Recidivism of Prisoners Released in 1994
29 18 18 4
3450Pennsylvania Sentencing Data, 1998
30 14 6 29
https://www.asis.org/rdap/
LastName, Title
Thank you!
Questions?
Kathleen Fearkfear@umich.edu