UNICA Core Group on Evaluation in Research and Higher Education

 29 October 2010

Vienna

1

I. Context

2

Data “mysteriously” aggregatedMultidimensionality of research excellence neglected unreliable data etc.

Current rankings not very useful for benchmarking or to develop research in one university

II. Objectives

3

to provide to the UNICA network a toolbox of research evaluation methodologies adapted to different “cases” identified as interesting by the UNICA members

to pool resources in learning from each other about best practices in the context of benchmarking activities

to develop a methodology accepted by Unica members, and based on expertise sharing, complementarities and harmonization of databases

III. Working Group

4

10 universities 4 meetings

Carte d’europe et universitésLeurs classements ectuels

IV. Methodology (1)

5

1. To list the possible objects to evaluateProjectsResearchers Teams, faculties, centres, ……

2. To list the possible goals of an evaluationTypologyRankingSwot analysis …

3. To list the possible users of an evaluationRectorsResearchersGovernments, funding agencies Media, public, …

One case is defined by a combination of answers to each of the 3 questions

IV. Methodology (2)

Focus on Case 2

Case study number

Object Goals Users

1 Faculties Strong and weak points Rectors

2 Group/department (smaller groups than “faculties”)

Strong and weak points Rectors

3 Teams, groups, faculties

Strong and weak points Public

6

III. Methodology (3)

7

Agreement to limit the number of indicators per dimension Dimensions: what do we want to take into account Indicators: how do we want to take into account

Two first fields: Economic/business, Physics

5 Dimensions and 20 indicators D1: scientific output (7 indicators) D2: Attractiveness/internationalization (5 indicators) D3: Research training (3 indicators) D4: Transfer of technology (5 indicators) D5: Links to education (indicators still to be selected)

+ Basic Data on university (Global budget, staff, etc)

8

DIMENSION 1: SCIENTIFIC OUTPUT

1. Publications in Scopus, WoF

2. Peer review chapters of books

3. Peer review books

4. # Citations in databases

5. # Patents

6. # ERC grants

7. # Highly scientists

III. Methodology (4)

IV. What has been done ?

9

Agreement on dimensions and indicatorsData collection from several members of the group Analysis of difficulties

IV. Challenges and next steps

10

To get data from all membersAnalysis and graphical representationNeed for common definitions of indicatorsTry to use existing dataTo have a meeting with Unica members involved in Multirank.