Measuring Impact
description
Transcript of Measuring Impact
![Page 1: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/1.jpg)
Measuring ImpactEd Baker & The Scratchpads Team
![Page 2: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/2.jpg)
ViBRANT and eMonocot
![Page 3: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/3.jpg)
ViBRANT and eMonocot
• Projects that focus on data collection, curation and reuse
![Page 4: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/4.jpg)
ViBRANT and eMonocot
• Projects that focus on data collection, curation and reuse
• Would like metrics to know how well we are doing
![Page 5: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/5.jpg)
ViBRANT and eMonocot
![Page 6: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/6.jpg)
ViBRANT and eMonocot
Projects based around communities
![Page 7: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/7.jpg)
ViBRANT and eMonocot
Projects based around communities
Provide tools for communities
![Page 8: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/8.jpg)
ViBRANT and eMonocot
Projects based around communities
Provide tools for communities• Quantify user contribution to multi-user
projects
![Page 9: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/9.jpg)
ViBRANT and eMonocot
Projects based around communities
Provide tools for communities• Quantify user contribution to multi-user
projects• Quantify how content generated by the
community is used/reused
![Page 10: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/10.jpg)
Citizen Science
![Page 11: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/11.jpg)
Citizen Science
• COMBER
![Page 12: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/12.jpg)
Citizen Science
• COMBER
• anymals+plants
![Page 13: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/13.jpg)
Citizen Science
• COMBER
• anymals+plants
• Citizen Science profile
![Page 14: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/14.jpg)
Citizen Science
• COMBER
• anymals+plants
• Citizen Science profile
Metrics allow for more competitive activities and for users to quantify their involvement
![Page 15: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/15.jpg)
Who needs impact metrics?
![Page 16: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/16.jpg)
Who needs impact metrics?
• The authors of data
![Page 17: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/17.jpg)
Who needs impact metrics?
• The authors of data
• People who employ data authors
![Page 18: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/18.jpg)
Who needs impact metrics?
• The authors of data
• People who employ data authors
• People who reuse the data
![Page 19: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/19.jpg)
Who needs impact metrics?
• The authors of data
• People who employ data authors
• People who reuse the data
Am I useful?
![Page 20: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/20.jpg)
Who needs impact metrics?
• The authors of data
• People who employ data authors
• People who reuse the data
Am I useful?
Do we have our priorities right?
![Page 21: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/21.jpg)
Who needs impact metrics?
• The authors of data
• People who employ data authors
• People who reuse the data
Am I useful?
Do we have our priorities right?
![Page 22: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/22.jpg)
Who needs impact metrics?
• The authors of data
• People who employ data authors
• People who reuse the data
Am I useful?
Do we have our priorities right?
Hidden gem?
![Page 23: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/23.jpg)
Who needs impact metrics?
• The authors of data
• People who employ data authors
• People who reuse the data
Am I useful?
Do we have our priorities right?
Hidden gem?
Avoid like the plague?
![Page 24: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/24.jpg)
Our premise
![Page 25: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/25.jpg)
We want people to share data…
![Page 26: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/26.jpg)
…people want to please the “powers that be” …
![Page 27: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/27.jpg)
… the powers that be like statistics.
![Page 28: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/28.jpg)
So we made some
![Page 29: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/29.jpg)
But the people weren’t happy.
![Page 30: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/30.jpg)
But the people weren’t happy.
Of all the things I do you only care about papers?
![Page 31: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/31.jpg)
But the people weren’t happy.
Of all the things I do you only care about papers?
The only way you value my papers is by citation in other
papers?
![Page 32: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/32.jpg)
But the people weren’t happy.
Of all the things I do you only care about papers?
The only way you value my papers is by citation in other
papers?
The impact of my career is measured by one number?
![Page 33: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/33.jpg)
So some people made their own
![Page 34: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/34.jpg)
But there are still problems
![Page 35: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/35.jpg)
But there are still problems
This STILL only deals with my publications!
![Page 36: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/36.jpg)
But there are still problems
This STILL only deals with my publications!
How do I get credit for data?
![Page 37: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/37.jpg)
But there are still problems
This STILL only deals with my publications!
How do I get credit for data?
I wrote some useful biodiversity software – what
about that?
![Page 38: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/38.jpg)
Are alt metrics useful?
![Page 39: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/39.jpg)
Are alt metrics useful?
![Page 40: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/40.jpg)
Are alt metrics useful?
These are fairly scholarly
![Page 41: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/41.jpg)
Are alt metrics useful?
These are fairly scholarly
These measure broader public impact
![Page 42: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/42.jpg)
Are alt metrics useful?
These are fairly scholarly
These measure broader public impact
• A good start
![Page 43: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/43.jpg)
Are alt metrics useful?
These are fairly scholarly
These measure broader public impact
• A good start
• Much broader than just scholarly citations
![Page 44: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/44.jpg)
Are alt metrics useful?
These are fairly scholarly
These measure broader public impact
• A good start
• Much broader than just scholarly citations
• Actually ignores traditional citations
![Page 45: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/45.jpg)
Are alt metrics useful?
These are fairly scholarly
These measure broader public impact
• A good start
• Much broader than just scholarly citations
• Actually ignores traditional citations
• Still only applied to papers
![Page 46: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/46.jpg)
What are we doing?
![Page 47: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/47.jpg)
What are we doing?
![Page 48: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/48.jpg)
What are we doing?
![Page 49: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/49.jpg)
What are we doing?
![Page 50: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/50.jpg)
Biodiversity Data Journal
Data papers (descriptions of data)
![Page 51: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/51.jpg)
Biodiversity Data Journal
Data papers (descriptions of data)
• Allows data to become citable
![Page 52: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/52.jpg)
Biodiversity Data Journal
Data papers (descriptions of data)
• Allows data to become citable
• Allows data to participate in standard credit/metric systems
![Page 53: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/53.jpg)
Biodiversity Data Journal
Data papers (descriptions of data)
• Allows data to become citable
• Allows data to participate in standard credit/metric systems
• … allows data to participate in alt metrics
![Page 54: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/54.jpg)
Scratchpads Statistics Module
What content exists on a Scratchpad site?
![Page 55: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/55.jpg)
Scratchpads Statistics Module
What content exists on a Scratchpad site?
• Filter by user or taxonomic term
![Page 56: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/56.jpg)
Scratchpads Statistics Module
What content exists on a Scratchpad site?
• Filter by user or taxonomic term
• How much content is there?
![Page 57: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/57.jpg)
Scratchpads Statistics Module
What content exists on a Scratchpad site?
• Filter by user or taxonomic term
• How much content is there?
• What kind of content is it?
![Page 58: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/58.jpg)
Scratchpads Statistics Module
What content exists on a Scratchpad site?
• Filter by user or taxonomic term
• How much content is there?
• What kind of content is it?
• How many registered users?
![Page 59: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/59.jpg)
Scratchpads Statistics Module
What content exists on a Scratchpad site?
• Filter by user or taxonomic term
• How much content is there?
• What kind of content is it?
• How many registered users?
• How often is the content viewed?
![Page 60: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/60.jpg)
Scratchpads Statistics ModuleAvailable now:
http://antkey.org/stats
[scratchpad.url]/stats
![Page 61: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/61.jpg)
Scratchpads Metrics Module
Puts the Scratchpad content in a broader context
![Page 62: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/62.jpg)
Scratchpads Metrics Module
Puts the Scratchpad content in a broader context
• Users and content
![Page 63: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/63.jpg)
Scratchpads Metrics Module
Puts the Scratchpad content in a broader context
• Users and content
• Opt-in (not enabled by default)
![Page 64: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/64.jpg)
Scratchpads Metrics Module
Puts the Scratchpad content in a broader context
• Users and content
• Opt-in (not enabled by default)
• Modular (pick and choose what to include)
![Page 65: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/65.jpg)
Scratchpads Metrics Module
Puts the Scratchpad content in a broader context
• Users and content
• Opt-in (not enabled by default)
• Modular (pick and choose what to include)
• Available by end of the year
![Page 66: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/66.jpg)
Scratchpads User Metrics
A partial implementation of the Scholar Factor proposed by Bourne and Fink
![Page 67: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/67.jpg)
Scratchpads User Metrics
A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink
Citations
![Page 68: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/68.jpg)
Scratchpads User Metrics
A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink
Citations
Scholar Factor
![Page 69: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/69.jpg)
Scratchpads User Metrics
A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink
Citations
Scholar Factor
Software / Data
![Page 70: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/70.jpg)
Scratchpads User Metrics
A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink
Citations
Scholar Factor
Software / Data Grant / manuscript reviews
![Page 71: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/71.jpg)
Scratchpads User Metrics
A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink
Citations
Scholar Factor
Software / Data Grant / manuscript reviews
![Page 72: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/72.jpg)
Scholar Factor
Scratchpads User Metrics
![Page 73: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/73.jpg)
Scholar Factor
Scratchpads User Metrics
Scratchpads User Metrics Module
![Page 74: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/74.jpg)
Scholar Factor
Scratchpads User Metrics
Scratchpads User Metrics Module
Google Scholar (h-index)
![Page 75: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/75.jpg)
Scholar Factor
Scratchpads User Metrics
Scratchpads User Metrics Module
Google Scholar (h-index) Scratchpad
![Page 76: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/76.jpg)
Scholar Factor
Scratchpads User Metrics
Scratchpads User Metrics Module
Google Scholar (h-index) Scratchpad GitHub
![Page 77: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/77.jpg)
Scholar Factor
Scratchpads User Metrics
Scratchpads User Metrics Module
Google Scholar (h-index) Scratchpad GitHub ?
![Page 78: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/78.jpg)
Scholar Factor
Scratchpads User Metrics
Scratchpads User Metrics Module
Google Scholar (h-index) Scratchpad GitHub ?
Modular
![Page 79: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/79.jpg)
Scholar Factor
Scratchpads User Metrics
Scratchpads User Metrics Module
Google Scholar (h-index) Scratchpad GitHub ?
Modular
![Page 80: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/80.jpg)
Scholar Factor
Scratchpads User Metrics
Scratchpads User Metrics Module
Google Scholar (h-index) Scratchpad GitHub ?
Modular
![Page 81: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/81.jpg)
Same modular structure for content
![Page 82: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/82.jpg)
Same modular structure for content
• Number of web links to page
![Page 83: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/83.jpg)
Same modular structure for content
• Number of web links to page
• Scholarly links (Google Scholar)
![Page 84: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/84.jpg)
Same modular structure for content
• Number of web links to page
• Scholarly links (Google Scholar)
• Social links
![Page 85: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/85.jpg)
Summary
We’re not there yet.
![Page 86: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/86.jpg)
Summary
We’re not there yet, but we have …
![Page 87: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/87.jpg)
Summary
We’re not there yet, but we have…
• Modular framework to add services as they become available
![Page 88: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/88.jpg)
Summary
We’re not there yet, but we have…
• Modular framework to add services as they become available
• multi-dimensional – not just papers and citations
![Page 89: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/89.jpg)
Summary
We’re not there yet, but we have…
• Modular framework to add services as they become available
• multi-dimensional – not just papers and citations
• Not a one size fits all approach
![Page 90: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/90.jpg)
Summary
We’re not there yet, but we have…
• Modular framework to add services as they become available
• multi-dimensional – not just papers and citations
• Not a one size fits all approach
![Page 91: Measuring Impact](https://reader036.fdocuments.in/reader036/viewer/2022062323/56816449550346895dd61267/html5/thumbnails/91.jpg)