Research evaluation: is it our business? Librarians in the brave new world of research evaluation...
-
Upload
jack-potter -
Category
Documents
-
view
221 -
download
2
Transcript of Research evaluation: is it our business? Librarians in the brave new world of research evaluation...
![Page 1: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/1.jpg)
Research evaluation: is it our business? Librarians in the brave new world of research evaluation
Andria McGrathSenior Information Specialist, Research Support, King’s College London
![Page 2: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/2.jpg)
What is research evaluation?
• External evaluation – in the UK the RAE now REF3 major elements of the RAE: – publication outputs, – grant income awarded, – research students
• Vital importance – link to income• Benchmarking – Rankings• Internal evaluation
![Page 3: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/3.jpg)
The players in institutions
• Research Office• PVC Research• Research group/dept heads• Administrators• Librarians – publications expertise• Institutional repository managers
3
![Page 4: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/4.jpg)
Evolving interest
• REF events at King’s 2008 -2010• 1st event – Beyond the RAE – representatives
from 75 universities – 1/3 librarians• Survey –
– biggest challenges – verifying publication lists– Biggest concerns for REF – bibliometrics – data
sources, subject differences, skilling up to cope
• http://www.kcl.ac.uk/iss/support/ref
4
![Page 5: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/5.jpg)
Events Nov 08 and June 09
• Focus on bibliometrics – pilots• Pre-event survey before Nov event:
– Only about half of institutions had a centralized publications data collection system
– One or two mentioned Symplectic– One mentioned something called Pure
5
![Page 6: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/6.jpg)
Data collection systems
• Some – only concerned with collecting RAE pubs• Others – aimed at collecting all publication info• Wider uses – profile pages / internal evaluation• Some creating internal ‘home grown’ systems
– Separate systems for 3 RAE data elements– Integrated system, taking feeds from other systems
(eg Research Gateway at King’s)
![Page 7: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/7.jpg)
REF focus changed
• Bibliometrics element toned down• Impact (4th King’s REF event in June 2010)• Not publication impact – practical impacts of
research• Research leaders had seen the light on
– data collection systems – bibliometrics
7
![Page 8: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/8.jpg)
CRIS Current Research Information SystemsAlready common in mainland Europe
8
![Page 9: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/9.jpg)
CERIF
• Common European Research Information Format
• A standard – uses XML• Objects or entities with attributes
eg project, person, organizational unit; • Relationships – 'linking relations’ • Rich semantics – roles and time
9
![Page 10: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/10.jpg)
R4R project - Ready for REF
• JISC project – King’s and Southampton• CERIF4REF XML schema created• Plugins for Eprints, Dspace and Fedora• Transfer data to REF collection system• Research admins, IR managers, systems staff
and librarians involved• http://r4r.cerch.kcl.ac.uk/
10
![Page 11: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/11.jpg)
Librarians and CRIS systems
• St Andrews experience• If research office led – get in on the ground
floor• Librarians have plenty of expertise to offer
11
![Page 12: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/12.jpg)
Which librarians?
• Senior managers• Subject/liaison librarians• Research support librarians• Instutional repository managers• Information resources depts. – subs• Enquiries staff
12
![Page 13: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/13.jpg)
Strategic directions for libraries
• RLUK strategic themes• 1st – redefining the research library model• New services / new roles for staff• Value of libraries for research and researchers
(RIN/RLUK report)• RLUK – Reskilling for Research (forthcoming)• Research support librarians
13
![Page 14: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/14.jpg)
Some new areas for librariansassociated with research evaluation• CRIS systems & CERIF• New ways of interacting with bibliographic /
citation databases• APIs – for publication or citation data• Discovering other institutional systems
eg grants DBs / student systems that feed into CRISs
• Bibliometrics principles and new products14
![Page 15: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/15.jpg)
New opportunities
• Showing our value– To administrators / research office– To academics and senior research leaders– To Graduate School
• Opportunities to advocate Open access
15
![Page 16: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/16.jpg)
Bibliometrics – the new buzz word
Awareness growing
•By librarians
•By research leaders in institutions
•By academics
16
![Page 17: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/17.jpg)
Librarians – natural synergy
• Citation databases• Citation reports• Training offered – eg by Thomson Reuters• Training points up limitations of basic
measures• Normalizing of measures crucial but difficult
without specialised products
17
![Page 18: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/18.jpg)
By research leaders
• A few initiates who understand the issues leading the way
• Percentiles tables in Essential Science Indicators used for evaluations
• H-index being used by some• At least one institution has employed a
bibliometrician
18
![Page 19: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/19.jpg)
Academics
Some beginning to take an interest
•Discovering:– ResearcherID– Citation reports in citation databases– Google Scholar
•Graduates – training opportunity – “Making an impact”
19
![Page 20: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/20.jpg)
Holy grail
• Research information system• Dashboard for research managers – eg for
grant information• Captures data with minimum effort• Integrated with bibliometrics• For internal & external purposes – evaluation
and promotional
20
![Page 21: Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research.](https://reader036.fdocuments.in/reader036/viewer/2022062404/5515fb5f550346d46f8b58b8/html5/thumbnails/21.jpg)
Is it our business?
21