Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title...
-
Upload
duongtuyen -
Category
Documents
-
view
217 -
download
4
Transcript of Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title...
![Page 1: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/1.jpg)
| 1
1
2017 SRA International - Annual MeetingOctober 14-18, 2017
Daniel Calto, Global Director of Solution Services, Research IntelligenceMartin Kirk PhD, Director ORS & SPARC, University of British Columbia
Research Metrics: Informing Institutional Strategy and Demonstrating Research Excellence / Impact
![Page 2: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/2.jpg)
| 2
2
Society of Research Administrators 16 October 2017Vancouver
Using Advanced Metrics to Identify Unique Institutional Research Strengths, Recruit and Retain Talent, and Drive More Productive University‐Industry Partnerships
Daniel CaltoGlobal Director of Solution ServicesResearch Intelligence
![Page 3: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/3.jpg)
| 3
Elsevier – A Unique Vantage Point on the Global World of Research
• Founded 130+ years ago
• Serving 30 million+ scientists, students, health and information professionals in 180+ countries
• 2,500 E-journals, 2,000 E-books published each year, dozens of research-oriented databases
• RELX (Reed Elsevier), the largest digital company in Europe
Elsevier – From publisher to solution provider Each year• 1.3 million manuscripts submitted to 2,500+ Elsevier journals
• 350,000+ articles published
• 900 million digital article downloads delivered
• 22,000+ journals from 5000+ publishers, >2 million articles per year tracked by Scopus (>69M articles in total)
• Terabytes of data in the Elsevier Research Intelligence suite
• Interactions with every university and government
![Page 4: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/4.jpg)
| 4
What Data Do We Bring to the Table?
61M
13M
800M/Y
80K
11M1M/Y
50M/Y
FTA downloads
FTA click-through
3M
Patent data
(>89M)
![Page 5: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/5.jpg)
| 5| 5| 5
Scopus Coverage Summary
21,951 peer-reviewed journals280 trade journals
• Full metadata, abstracts and cited references (refs post-1970 only)
• Funding data from acknowledgements
• Citations back to 1970
Physical Sciences7,441Health Sciences7,133SocialSciences8,698LifeSciences4,601
100K conference events8M conference papers
Mainly Engineering and Computer Sciences
562 book series
150K stand-alone books1.2M items
Focus on Social Sciences and A&H
World’s largest Abstract and Citations Database69M records from 22K serials, 100K conferences and 150K books
from more than 5000 publishers and 105 countries• Updated daily• Records back to 1823• “Articles in Press” from > 8,075 titles• 40 different languages covered• 3,643 active Gold Open Access journals indexed
BOOKSCONFERENCESJOURNALS
27M patents
From 5 major patent offices- WIPO- EPO- USPTO- JPO- UK IPO
PATENTS*
![Page 6: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/6.jpg)
| 6| 6| 6
Scopus Data ModelThe Scopus data model is designed around the notion that articles are written by authorsthat are affiliated with institutions. Visually and rather simplistically, this relational model is represented below. What is the value of this structured data? This relational data model means that Scopus can tell you who is doing what in global literature and where they are doing it with higher accuracy than anyone else
article
author
Scopus Data Model Simplified
affiliation
Over 69M Journal, Conference, & Book
records
Over 12M author profiles (active)
Over 9M institutional profiles, incl. 80000 major output entities
![Page 7: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/7.jpg)
Elsevier Text and Data Mining | 7
Fingerprints Can Be Created from Any Text, or Group of Texts
• Any text can be Fingerprinted, from grant applications to publications • Fingerprints are generated from the title and abstract • Natural Language Processing techniques are applied
2
1
Concepts are derived from the text. Each concept is found in an underlying thesaurus suitable for the scientific area of the text. Concepts are weighted to create a precise summary of the text’s meaning.
![Page 8: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/8.jpg)
Elsevier Text and Data Mining |
Noun Phrases Concepts
8
How Does the Elsevier Fingerprint Engine Work?
FingerprintThesauriSource Text
Compendex
GESIS
MeSHGeoTree
Domain Thesaurus/Vocabulary Life Sciences MeSH thesaurus Physics NASA thesaurus Agriculture NAL thesaurus Economics STW thesaurus, Eco Humanities vocabulary Social Sciences Gesis thesaurus Mathematics Cambridge Math thesaurus, Math vocabulary Geosciences Geobase thesaurus Engineering Compendex thesaurus Humanities Humanities vocabulary Compounds (Chemistry) Compendex thesaurus, MeSH thesaurus
![Page 9: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/9.jpg)
Elsevier Text and Data Mining | 9
Natural Language Processing Modules Applied to Text text
Natural Language via NLP Processing to Structured Semantic Machine‐Readable Text
![Page 10: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/10.jpg)
| 10
10
FPE In Action--Expert Lookup—Global Expertise Search
![Page 11: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/11.jpg)
| 11
11
Expert Lookup—Global Expertise Search—Top Scholars
![Page 12: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/12.jpg)
Elsevier Text and Data Mining |
FPE in Action: SciVal
12
![Page 13: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/13.jpg)
Elsevier Text and Data Mining |
FPE in Action: Pure
13
![Page 14: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/14.jpg)
| 14
Network Graphs – Production, Collaboration, Impact, Visualization..
![Page 15: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/15.jpg)
| 15
15
Topics of Prominence—A Planning Solution
• We have identified ~100.000 global research topics and ranked them by Prominence.
• Prominence is a new indicator that shows the current momentum of a topic by looking at citations, views and CiteScore values.
• Prominence highly correlated with funding –helps researchers and research managers identify topics in which funding will increase.
• Going way beyond what the competition can do…
![Page 16: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/16.jpg)
| 16
16
First Solution of its Kind
The first truly global detailed research portfolio analysis –this has never been done before – we use all of Scopus to form topics.
• Who’s leading the way – we can identify emergent topics with high momentum to understand who is currently leading the way.
• What’s related – We can tell you how the topics are related to your research portfolio.
• A better reflection of reality – topics are an excellent reflection of reality since they are based on citation patterns and not journal categories and therefore truly multidisciplinary.
![Page 17: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/17.jpg)
| 17
17
A groundbreaking concept
• Researchers in topics with high prominence receive more funding on average – We have evidence that researchers in prominent topics receive more funding (per researcher) than their peers in other topics.
• Help improve grant applications – we can truly help researchers to increase their grant success rate by focusing on high prominence topics.
• Topics resonate with researchers – researchers recognize them intuitively and agree with the level of granularity.
![Page 18: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/18.jpg)
| 18
18
Mapping Research Topics--History and Competition
• 1985 – ISI (now Clarivate) develops Research Frontso A bibliometric way to identify research opportunities
• 1988 – CRP (now SciTech) develops Research Communitieso Same algorithms and lower thresholds to increase coverage
• 2007 – SciTech develops Distinctive Competencieso Clusters research communities using University strengths
• 2015 – SciTech develops Topicso Significantly increases coverage and accuracy
• 2017 – SciTech develops Topic Prominence indicatoro Uses citations, downloads and journal impacto First time a bibliometric indicator is used to predict funding patterns
![Page 19: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/19.jpg)
| 19
19
Mapping Research Topics--History and competition
• Research Fronts (1985) 2% coverage 10,000 clusterso (Clarivate is still using this!)
• Research Communities (1988) 4% coverage 35,000 clusters
• Distinctive Competencies (2007)15% coverage 200,000 clusters
• Topics (2015) 95% coverage 100,000 clusters
• Topic Prominence (2017) 95% coverage Predicts fundingo Full coverage, accurately models supply/demand for science
![Page 20: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/20.jpg)
| 20
20
Topics of Prominence—Accuracy
• NEEDS- Accuracy: Accurate topics that
contain the right papers
- Comprehensive analysis at scaleshows that topics based on directcitation are far more accuratethan those based on bibliographiccoupling or co-citation
- Also, they are much more accuratethan journal categories
- Use topics identified using directcitation
Klavans, R. and K.W. Boyack, Which type of citation analysis generates the most accurate taxonomy of scientific and technical knowledge? JASIST, 2017. 68(4): p. 984-998.
![Page 21: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/21.jpg)
| 21
21
Topics of Prominence – Variance and Dynamics
• Stability: Topics with realistic dynamics• Topics can be new or old, large or small,
growing, emerging, declining, interdisciplinary, etc., and have varied histories
• Topics have persistent dynamics; low birth and death rates, s-curve histories
Boyack, K.W. and R. Klavans, R. Creation and analysis of large-scale bibliometric networks. Springer Handbook of Science and Technology Indicators, 2018 (to appear).
![Page 22: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/22.jpg)
| 22
22
Topic Prominence—How is It Calculated?
• Factor 1 has an eigenvalue of 2.33 (very high), suggesting that the composite indicator should include Citations, Views, Citescore
• Other formulations with more features were tested, but they did not have greater explanatory power than the 3-feature indicator
• Pj = 0.495 (Cj – mean(Cj))/stdev(Cj) + 0.391 (Vj – mean(Vj))/stdev(Vj) + 0.114 (CSj – mean(CSj))/stdev(CSj),
Table 2. Factor loadings and scoring coefficients used to calculate topic prominence. Factor 1 Factor 2 Normalized Score L:Citations 0.837 - 0.244 0.495 L:Views 0.812 - 0.262 0.391 L:CiteScore 0.653 0.154 0.114 L:Authors 0.593 0.334 (not used) Vitality 0.441 0.269 (not used)
![Page 23: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/23.jpg)
| 23
23
Example model and map
• Using 2013-10 datacut (source data 1996-2012)
• 582 million citing-cited pairs, 24.6 million source EID, 23.8 million cited non-indexed EID
• Calculated relatedness for 582 million pairs
• Ran SLM using resolution of3 x 10-5
• A few clusters with <50 items were merged with larger clusters
• Result – 91,726 clusters (topics)
Klavans, R. and K.W. Boyack, Research portfolio analysis and topic prominence. Journal of Informetrics, 2017 (under review).
![Page 24: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/24.jpg)
| 24
24
Single Topic Characterization for 92,000 Topics
![Page 25: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/25.jpg)
| 25
25
Topics of Prominence—Highly Correlated with Funding
• Funding divided into two time periods (2008-10, 2011-13)• Prominence was calculated as of 2010
• Funding in two time periods is extremely highly correlated• Prominence is highly correlated with funding in both time periods
Table 4. Correlation matrix for variables considered in the funding prediction analysis. L:Fund1113 L:Fund0810 Prominence Vitality L:Authors L:Fund1113 1.000 L:Fund0810 0.837 1.000 Prominence 0.606 0.616 1.000 Vitality 0.166 0.162 0.314 1.000 L:Authors 0.160 0.171 0.242 0.202 1.000
![Page 26: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/26.jpg)
2017 SRA International - Annual MeetingOctober 14-18, 2017
Martin Kirk PhD, Director ORS & SPARC, University of British Columbia
Research Metrics: Informing Institutional Strategy and Demonstrating Research Excellence / Impact
![Page 27: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/27.jpg)
All About Impact
27
![Page 28: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/28.jpg)
28
ROI:• Need to convince stakeholders of research ROI• Research as economic development activity
Collaboration:• Research teams…moving away from individual researcher enterprise• Interdisciplinary research• Increased partnership with industry• Large scale collaboration between research institutions
Social Sciences and Broader Impacts:• Having social science and GELS included in holistic research approach • Improved recognition and evaluation of social sciences impact• Knowledge mobilization or broader impact plans as funding success criterion
HQP:• High Quality Personnel (HQP) productivity
Internationalization:• Growing internationalization of research• Performance in THE and ARWU global ranking systems
Interoperability and Integration:• Seamless interoperability (CCV)• Reducing administrative burden• Integrated researcher performance management and reporting
Scarcity of Research Funding:• Low granting agency success rates
WHY RESEARCH METRICS
![Page 29: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/29.jpg)
29
Retrospective:• What is the economic ROI of the research …• What is the direct societal impact of our research• What is the productivity and impact of a specific research cohort: Individual PI,
Group, Department, Institute, Faculty, institution…performance management, competitiveness etc.
• What have been our most productive collaborations• How competitive are we compared to peer institutions• How do we move up the global rankings
Foresight:• Who should we reward for their research excellence (retention)• Who should we hire to grow our impact, build out strategic research themes• Who should we cooperate with in the future for biggest bang for buck?• Who should we (broad) strategically align with (U 21)
BIG QUESTIONS UNIVERSITIES ARE TRYING TO ANSWER
![Page 30: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/30.jpg)
Canada 2014 to 2016
SciVal
30
![Page 31: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/31.jpg)
Canada 2014 – 2016 Field-Weighted Top 10 Institutions
31
![Page 32: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/32.jpg)
UBC 2014-2016 Overall Research Performance
![Page 33: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/33.jpg)
Researchers at the UBC have contributed to 12,038 topics between 2014 to 2016
Top 5% of worldwide Topics of Performance
![Page 34: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/34.jpg)
Competencies / Topics at UBC
![Page 35: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/35.jpg)
1. Most active Institutions – Aortic, Valve; Aortic Valve Stenosis; severe symptomatic T.32
Top 3 UBC AuthorsWebb, John G
Leipsic, Jonathon AvromDvir, Danny
Top 3 worldwideWebb, John G
Rodés-Cabu, Josep ( U Laval)Colombo, Antonio (U Vita‐Salute Sam Raffaele Italy)
![Page 36: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/36.jpg)
2. jets; production; parton showerT.1026
![Page 37: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/37.jpg)
3. microwaves; polarization; angular power
![Page 38: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/38.jpg)
Benchmarking Institutions – Field Weighted Citation Impact 2011 - 2016
![Page 39: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/39.jpg)
Benchmarking Institutions – Publications in top Journals 2011 - 2016
![Page 40: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/40.jpg)
Research Area: Quantum UBC 2016
![Page 41: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/41.jpg)
Quantum UBC cont.…
![Page 42: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/42.jpg)
PI: Marco Marra – Overview 2014 - 2016
![Page 43: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/43.jpg)
Collaboration – Marco Marra2014 - 2016
Harvard top 10 collaborators
![Page 44: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/44.jpg)
Citing-Patents count - Marco Marra 2014 - 2016
![Page 45: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/45.jpg)
UBC collaboration with Harvard 2014 - 2016
![Page 46: Research Metrics: Informing Institutional Strategy and ... Data Model Simplified ... from the title and abstract •Natural Language Processing techniques ... 100,000 clusters •Topic](https://reader033.fdocuments.in/reader033/viewer/2022051801/5ae4229f7f8b9a90138e88d2/html5/thumbnails/46.jpg)
46
• Evaluation of Research in Canada is spotty but becoming more significant
• Current tools are good but need to be better i.e. more flexibility, better integration, more scope of metrics etc.
• Need clean, comprehensive data sets • Need better, more nuanced impact metrics (Snowball+)
Bottom line: Need to transform institutional use of metrics and tools from interesting to mission critical
CONCLUSIONS