New trends in university rankings 2011
-
Upload
uoc-universitat-oberta-de-catalunya -
Category
Education
-
view
109 -
download
1
description
Transcript of New trends in university rankings 2011
www.che.de
Gero Federkeil,CHE Centre for Higher Education, Germany
New Trends in University Rankings
1st International Seminar on Higher Education Rankings and E-Learning,
22-23 September 2011, Barcelona
The CHE – A Short Introduction
The traditional ranking approach
Überschrift Nummer 3New trends: Multi-dimensional rankings
Presentation
2Barcelona, September 2011
The U-Multirank Project
Conclusions
Private, not-profit organisationFounded in 1994 by Bertelsmann Foundation and German Rectors Conference
The CHE - Centre for Higher Education
Goal: Initiate and promote of reforms in German higher education
Activities:HE policy issues (e.g. Bologna, funding, …)ConsultingCommunication & trainingRanking
3Barcelona, September 2011
The CHE - Centre for Higher Education
Ranking of German universities among founding tasks of CHEFirst ranking published in 1998
Extension of fields and indicatorsContinuous further development of methodology
InternationalisationExtension of CHE Ranking: Austria, Switzerland, NetherlandsCooperation with Fundación CYD to develop a ranking in SpainU-Multirank project to “develop the concept and test the feasibility of a global multi-dimensional university ranking” Founding member of IREG –Observatory on Academic Rankings and Excellence (“Berlin Principles”)
4Barcelona, September 2011
The CHE – A Short Introduction
The traditional ranking approach
Überschrift Nummer 3New trends: Multi-dimensional rankings
Presentation
5Barcelona, September 2011
The U-Multirank Project
Conclusions
Barcelona, September 2011 6
The growing importance of rankings
Rankings have become a wide spread phenomenon now:about 10 global rankingsnational rankings in about 50 countries
Rankings have an impact onstudentsstrategies of universities (e.g cooperations, mergers)politics (e.g. Excellence / exchnage programmes)
Yet they remain controversial
The Traditional Ranking Approach
Barcelona, September 2011 7
Ranking of wholeuniversities
Composite indicator
League table
Barcelona, September 2011 8
The critique of the traditional ranking model
Ranking of whole institutions
Composite overall indictor
League table approach
• Composite indicators blur profiles and strengths & weaknessesThere are neither theoretical nor empirical arguments for assigning specific pre-defined weights to single indicators
Most users are interested in information about “their” field”Institutional rankings give misleading averages across fields/units
Barcelona, September 2011
Shanghai Jiaotong Ranking QS Indicator Weight Indicator Weight SCI publications 20 % Reputation among scholars 40 %
Publications Science & Nature 20 % Reputation among employers 10 %
Highly cited authors 20 % Citations 20 %
Nobel Prizes & Field Medals 20 % Student-staff-ratio 20 %
Alumni with NobelPrizes 10 % International students 10 %
Size 10 % International staff 10 %
World Rankings: Indicators & Weights
9
Barcelona, September 2011 10
The critique of the traditional ranking model
Ranking of whole institutions
Composite overall indictor
League table approach
• Composite indicators blur profiles and strengths & weaknessesThere are neither theoretical nor empirical arguments for assigning specific pre-defined weights to single indicators
• Small differences in the scores of indicators lead to big differences in league tableGive false impression of exactness (“Number 123 is better than number 127”)
Most users are interested in information about “their” field”Institutional rankings give misleading averages across fields/units
The CHE – A Short Introduction
The traditional ranking approach
Überschrift Nummer 3New trends: Multi-dimensional rankings
Presentation
11Barcelona, September 2011
The U-Multirank Project
Conclusions
Barcelona, September 2011 12
New trends in rankings
Although this is still the dominant model there are new trends:
Alternative approaches: Multi-dimensional rankingsChanges within existing rankings
Barcelona, September 2011 13
Changes in existing rankings
Introduction of field based-rankings in addition to institutional rankings
At least in lower part of ranking: broad groups instead of league tables (e.g. 51 – 75, 76 – 100 ...),
Barcelona, September 2011 14
A new approach: Multi-dimensional rankings
There is a number of rankings with a different approach:
National rankings, e.g.:CHE ranking (Germany, Austria, Switzerland, Netherlands)College Navigator TaiwanGood University Guide AustraliaRanking Project Spain (Fundación CYD – CHE)
International rankings, e.g.Leiden RankingU-Multirank Project
Barcelona, September 2011 15
Multi-dimensional rankings; Basic concept
Basic assumption: There is no objective ranking as there is no single, objective concept of quality (“Quality is in the eye of the beholder”)Different users of rankings have different ideas about quality & different preferences with regard to indicatorsEach ranking reflects the views of the producers of rankingsAs the producers of rankings have to define a set of indicators, butthis set should be broad and be based on stakeholder consultation, and,the decision about the relevance of indicators should be left to users
Multi-dimensional rankings do not calculate
composite indicatorswith pre-defined weights
of single indicators !
HE Conference 2011 | Gero Federkeil | Canberra 2011/03/02 16
Multi-dimensional ranking : CHE
Field based ranking
Multi-dimensional ranking
Rank group approach
• ~20 to 25 indicators, each stands for itself• Show profiles, allow for an anaylsis of strengths
& weaknesses at relevant levelRefer to a user-focused concept of good performance /quality
• Top, Middle, Bottom group• Avoids false impressions of exactness• Takes serious limitations in data (quality)
Inclusion of 34 fieldsMore meaningful information to usersReflects internal heterogeneity of universities
+
+
+
The CHE – A Short Introduction
The traditional ranking approach
Überschrift Nummer 3New trends: Multi-dimensional rankings
Presentation
17Barcelona, September 2011
The U-Multirank Project
Conclusions
The project• Commissioned by the European Commission• 2‐year project, 2009 – June 2011• Report now available:
http://ec.europa.eu/education/higher-education/doc/multirank_en.pdf
• Ján Figel, the former European Commissioner for Education, Training, Culture and Youth:
“‐ to allow stakeholders to make informed choices;‐ to help institutions to position themselves andimprove their performance”
• Two phases:o Design of new instrumento Testing the feasibility of new instrument
18Barcelona, September 2011
Specification of U‐Multirank
• Five dimensions:o Teaching & learningo Researcho Knowledge transfero International orientationo Regional engagement
• Long list of indicators to be tested in pilot project
• development of data collection tools and processes (question‐naires, definitions, FAQs, communication + feedback processes)
• methods for building ranking groups instead of league tables
19Barcelona, September 2011
Testing U‐Multirank
• Two levels:• Institution (FIR)• Field (FBR)
• Global sample of higher education and research institutions: 159 (target: 150), 2/3 Europe, 109 completed institutional questionnaires
• Two fields: • Business studies• Engineering (electrical and mechanical)
20Barcelona, September 2011
Classification and Ranking: Mapping diversity
Diversity of higher education institutions in Europe & theworld
Identifying comparableinstitutions that can be
compared in one ranking
Description of horizontal diversity
Types/profiles
Assessment of verticaldiversity
Performance
Complementary instruments of transparency
+
M lti di i l l b l i it ki
Barcelona, September 2011 22
First element: Selection of a comparable set of universities based on institutional profiles
Teaching andlearning
Research involvement
Knowledgeexchange
Regional engagement
International orientation
Student profile
Example:
• Comprehensive, teaching oriented institution• Mainly undergraduate education• Low research orientation• Some activities with regard to knowledge transfer• Low international orientation• Regionally embedded (e.g. recruiting)
Comparison / ranking among this particularprofile of institutions
Barcelona, September 2011 23
Second element: ranking – Multi-dimensional ranking
Barcelona, September 2011 24
Second element: Selection of indicators according to user‘s preference
Selection of (upto) 5 indicators
Priorisation ofindicators
Barcelona, September 2011 25
Second element: Selection of indicators according to user‘s preference
The result is a personalised
ranking
Barcelona, September 2011 26
Second element: Selection of indicators according to user‘s preference
Looking different with different
indicators
Barcelona, September 2011 27
Dimension Total # indicators
After pilot
A: need no/minor modification
B: need further work C: discarded
Teaching & Learning
FIR 5 0 1 0
FBR 8 4 4 0
FBR (student survey) 13 13 0 0
Research
FIR 9 3 5 1
FBR 7 6 1 0
Knowledge transfer
FIR 8 3 5 0
FBR 7 1 2 4
International orientation
FIR 8 6 2 0
FBR 9 6 3 0
Regional engagement
FIR 4 1 3 0
FBR 5 1 4 0
feasibility of dimensions and indicators
Barcelona, September 2011 28
Teaching & Learning Teaching & Learning
U‐Multirank ‐ Indicators
Barcelona, September 2011 29
Research Knowledge Transfer
U‐Multirank ‐ Indicators
Barcelona, September 2011 30
Regional EngagementInternational Orientation
U‐Multirank ‐ Indicators
Barcelona, September 2011 31
EU Commission: Communication on the Modernisation of Higher Education (20 Sept.2011):
Future Prospects
http://ec.europa.eu/education/higher-education/doc/com0911_en.pdf
The CHE – A Short Introduction
The traditional ranking approach
Überschrift Nummer 3New trends: Multi-dimensional rankings
Presentation
32Barcelona, September 2011
The U-Multirank Project
Conclusions
Conclusions
Barcelona, September 2011 33
Rankings are here to stay, their relevance will rather increaseThere is a need for transparency about European Higher Education Aera
Multi-dimensional rankings are a new approach which
is user-driven, taking into account that there is no single objective ranking, and,
is able to make visible the diversity of higher education institutions by showing different profiles, and
is looking beyond research excellence only (teaching & learning, transfer ...)
Conclusions
Barcelona, September 2011 34
Within the context of U-Multirank there could be one particular ranking among Open Universities /distance education institutions.
Some indicators of U-Multirank are applicable,
the appropriateness of others would have to be discussed, and
there is surely a need for new, particular indicators
Barcelona, September 2011 35
There might be some limits to rankings in general
„You‘re kidding! You count publications?“
37
More information:
www.che-ranking.dewww.u-multirank.eu
www.che.de
Gero Federkeil,CHE Centre for Higher Education, Germany
New Trends in University Rankings
1st International Seminar on Higher Education Rankings and E-Learning,
22-23 September 2011, Barcelona