Rankings and the Emergence of New Forms of ......Rankings and the Emergence of New Forms of...
Transcript of Rankings and the Emergence of New Forms of ......Rankings and the Emergence of New Forms of...
www.dit.ie/researchandenterprise
RankingsandtheEmergenceofNewFormsofAccountability
ProfessorEllenHazelkornPolicyAdvisortotheHigherEducationAuthority(Ireland)Director,HigherEducationPolicyResearchUnit(HEPRU)
130th NEASCAnnualConferenceandMeetingDecember2015
Boston
www.dit.ie/researchandenterprise
WhicharetheBestUniversities&Colleges?
Are the best universities those which best match the criteria established bythe different rankings OR those that help the majority of students earn thecredentials for sustainability living and employment?Are the best universities those that contribute to new scientific discoveriesand highly trained PhDs OR those that “emphasize the obligations studentshave to serve their communities and the nation at large”?Are the best universities those that reinforce an elite knowledge society(where progress depends on the cutting-edge knowledge of the few) ORthose that help build-up a mass knowledge society (where progress dependson the “wisdom of the many”)?Should we define quality according to reputation/prestige, studentsatisfaction, salary level of graduates OR how students have beentransformed by their educational experience, e.g. learning gain?
www.dit.ie/researchandenterprise
Themes
• RankingsandtheirLegacy
• NewFormsofAccountability
• ImplicationsforHigherEducation
www.dit.ie/researchandenterprise
1.RANKINGSANDTHEIRLEGACY
www.dit.ie/researchandenterprise
Rankings’Legacy
• Global rankings have become a game-changer;• Immediately – and subsequently – attracted attention of policymakers and
the academy:– Choice of indicators has set parameters for what constitutes quality;– Visible measure of global competitiveness and multi-polar character;– “Top 100” has transformed “world-class” into a strategy, a language, a
topic of study;– Driven profound transformation of our HEIs and HE systems;– Placed HE/R&D investment high on political and policy agenda.
• Today, less about student choice and more about geopolitical positioning.And in the process, a whole industry has been created.
www.dit.ie/researchandenterprise
WorldOrderAccordingtoRankingsTop100,2004-2015
www.dit.ie/researchandenterprise
MeasuringanElite
• Over18,000HEIsworld-wide,butglobalrankingsfocusoncharacteristicsandperformanceofthetop100universities;- lessthan0.5percentoftotal;- ~0.4percentoftotalworldpopulationof196mtertiarystudents(UIS,
2012).
• AsdemandforHEgrowsworldwide,selectivityisaccelerating.
www.dit.ie/researchandenterprise
InfluenceofRankings
• Restructuringofnationalsystems;• Reshapingofnationalpriorities;• Refocusingofinstitutionalpriorities;• ReorganisingtheHEI,institutionaldepartmentsandhierarchyof
disciplines;• Emphasisonresearchvs.teaching;postgraduatevs.undergraduate– with
implicationsfortheacademicprofession;• Changesinresearchpractice:language,publication,orientation,
basic/applied,etc.• Influenceonstakeholders– students,governments,business/employers,
investors,public,etc.
www.dit.ie/researchandenterprise
MeasuringQuality
• Despite all the criticism, rankings have succeeded in placing HE within awider comparative and international framework;
• Quality and excellence are key differentiators in national/global market;– National geo-political positioning and pride;– Beacon to attract/retain investment, business and talent;– Institutional reputation and status;– Performance assessment of scientific-scholarly research;– Link between qualification and career opportunities and life-style;– Value-for-money and return-on-(public) investment;
• “All the things wrong with the rankings matter considerably less than theplain fact that the rankings matter” (Locke, 2011, 226)
www.dit.ie/researchandenterprise
GlobalSignificance
Global economy critical to understanding significance and impact:• Early years associated with tail-end of long period of economic growth
driven by unregulated finance capital;• Latter years marked by lingering effects of global economic crisis – and
search for sustained recovery and economic growth:– Intensification of competition for a greater share of mobile capital and
talent;– Noticeable shifts in “world order”;– Significant demographic changes;– R&D investment patterns and geographic imbalances;– HE is big business for nations and institutions;
• Internationalisation• Deregulation of HE market: for-private/cross-border providers.
www.dit.ie/researchandenterprise
QualitynowaGeopoliticalIssue
• As globalisation accelerates and market principles intrude further, growingneed, arguably out of necessity, to regulate the marketplace;– Supra-national governments, national governments/US states, HE
agencies, commercial media, HE organisations;– Global intelligence business leading to common international data set;– In global labour market, quality credentials are essential for
international recognition.• Greater government involvement either directly in the process and/or use
of outcomes in policy and decision making;- Signals extent to which HE is losing role as primary guardian of quality
(Harman, 2011, 51;Dill and Beerkens, 2010, 313-315).
– Explains why global rankings have assumed such significance at ageopolitical level.
www.dit.ie/researchandenterprise
2.NEWFORMSOFACCOUNTABILITY
www.dit.ie/researchandenterprise
AlternativeRankings/AlternativestoRankings
• Rankings have been the dominant instrument but there are a range ofother tools being developed:– Alternative rankings by competitors, as new products and services;– Alternatives to rankings by governments, agencies, HE, and others.
• Traditionally,qualitywasmeasuredininput(e.g. studententry,academicqualifications,budget/income,libraryresources) andreputation;
• Asthe“private”and“public”goodrolesofhighereducationcomeunderincreasingscrutiny,thefocushasshiftedtooutputs, outcomes,impact,benefitandrelevance;
• Shows different ways to measure quality.
www.dit.ie/researchandenterprise
Transparency&AccountabilityInstruments
Rankings
Classification&Profiling
QualificationsFrameworks
QA&LearningOutcomes
Scorecards&Websites
Globalintelligence
SocialNetworking
www.dit.ie/researchandenterprise
Rankings
Rankings
Classification&Profiling
QualificationsFrameworks
QA&LearningOutcomes
Scorecards&Websites
Globalintelligence
SocialNetworking
www.dit.ie/researchandenterprise
GlobalRankings
• AcademicRankingofWorldUniversities (ARWU)(ShanghaiJiaoTongUniversity,China),2003
• Webometrics(SpanishNationalResearchCouncil,Spain),2004• NationalTaiwanUniversityRankings(formerlyPerformanceRankingof
ScientificPapersforResearchUniversities, HEEACT),2007• LeidenRanking(CentreforScience&TechnologyStudies,Universityof
Leiden),2008• SCImagoJournalandCountryRank(SJR)(Spain),2009• UniversityRankingbyAcademicPerformance(URAP) (InformaticsInstitute
ofMiddleEastTechnicalUniversity,Turkey),2009• QSWorldUniversityRankings(QuacquarelliSymonds,UK),2010• THE WorldUniversityRanking(TimesHigherEducation,UK),2010• U-Multirank (EuropeanCommission, Brussels),2014• BestGlobalUniversitiesrankings (USNWR,US),2014
www.dit.ie/researchandenterprise
SelectNationalRankings(red=governmentsponsored)
INSTITUTIONALDISCIPLINE/
SUB-CATEGORIESSPECIALIST
• UniversityRankingSystem(Bulgaria)• CHE-HochschulRanking (Germany)• ExpertUniversityRanking(Russia)• GoodUniversityGuide(Australia)• GuardianUniversityGuide(UK)• UniversityRankingsofIslamicCountries(Iran)• HigherEducationCommissionRankings(Pakistan)• LaRepubblicaGrandeGuidaUniversità (Italy)• Maclean’s OnCampus(Canada)• NationalRankingsof BestUniversities(Kazakhstan)• Netbig ChineseUniversityRanking(China)• NigeriaUniversities CommissionRanking• OHEC(Thailand)• PerspektywyUniversityRanking(Poland)• RankingU-Sapiens(Colombia)• Sunday TimesGoodUniversityGuide(Ireland)• TimesHigherEducationUniversityGuide(UK)• Top200UniversityRankings(Ukraine)• URANK-rank(Sweden)• USNewsandWorldReport(USNWR)CollegeRankings(US)
• Dataquest(India)• India Today(India)• Outlook(India)• LeNouvelObservateur (France)• SherifMagazine(Iran)• NationalResearchCouncilRankingof
Doctoral Programmes(US)• Toplawschools.com(US)• AmericanUniversities Admission
Programme:UndergraduateAmericanUniversities Rankingsfor InternationalStudents (US)
• USNewsandWorldReport(USNWR)TopMed Schools(US)
• WPROSTMBA (Poland)
• CollegeNET SocialMobilityIndexRanking(US)
• GeorgetownPublicPolicy ReviewPlacementEfficiencyRanking(US)
• Metroversities (US)• NewYorkTimesMost
Economically DiverseTopColleges(US)
• OnlineStudyAustraliaOnlineUniversityRankingList(Australia)
• PrincetonReview(US)• SavioursofOurCities (US)• SocialMobilityIndex(CollegeNet
andPayscale,US)• WashingtonMonthlyCollege
Guide(US)• WashingtonMonthlyRankingof
CommunityColleges(US)
www.dit.ie/researchandenterprise
U-Multirank
www.dit.ie/researchandenterprise
www.dit.ie/researchandenterprise
Classification&Profiling
Rankings
Classification&Profiling
QualificationsFrameworks
QA&LearningOutcomes
Scorecards&Websites
Globalintelligence
SocialNetworking
www.dit.ie/researchandenterprise
U-Map
• Profilingtooltofacilitateeasycomparisonbetweendifferent institutionsandinformstudentchoiceorstrategicdecisionmakingbyinstitutionsorgovernments.• SimilarinitiativeshavebeendevelopedbyNorway,Ireland,Australia
www.dit.ie/researchandenterprise
www.dit.ie/researchandenterprise
QualificationsFrameworks
Rankings
Classification&Profiling
QualificationsFrameworks
QA&LearningOutcomes
Scorecards&Websites
Globalintelligence
SocialNetworking
www.dit.ie/researchandenterprise
www.dit.ie/researchandenterprise
QA&LearningOutcomes
Rankings
Classification&Profiling
QualificationsFrameworks
QA&LearningOutcomes
Scorecards&Websites
Globalintelligence
SocialNetworking
www.dit.ie/researchandenterprise
www.dit.ie/researchandenterprise
LessonsfromBologna
• Bologna places quality within broader educational framework;• Big idea is the “accountability loop”:
– Qualification framework facilitates flexible learning paths and sets out clearstatement of what students must demonstrate at each qualification level
– “Tuning” aids (re-)design, development, implementation, evaluation andenhancementof quality degree programmes within context
– Formalized concept of learning outcomes rather than measuring inputs (e.g.credit hours, classroom teaching, entry grades)
– QA systems sets evaluation standards and guidelines for institutional self-assessment and quality processes, and externalmonitoring
– Diploma Supplement provides description of academic career and thecompetencies acquired
www.dit.ie/researchandenterprise
Measuring&ComparingAchievementsofLearningOutcomes
• CALOHEEisaEUfundedprojectdrawingontheTuningexperiencebeginning2017;
• SeekstostudywhetherstudentsenrolledinHEaroundEuropedeveloptheappropriatecompetencesbylookingatstudyprogrammes,andwhethertheydeliver theirpromises;– UnlikeAHELOwhichlooksatinstitutionallevel,thisprojectlooksat
theprogrammelevel• Willfocuson5domain:Engineering (CivilEngineering),HealthCare
(Nursing),Humanities(History),NaturalSciences(Physics),andSocialSciences(Education).
www.dit.ie/researchandenterprise
AHELO
• AssessmentofLearningOutcomesdevelopedbyOECDinresponse tocriticismofrankings;
• Attentiononoutputsoflearning,– Genericskills commontoallstudents
(suchascriticalthinking,analyticalreasoning,problem-solving,andwrittencommunication).
– Discipline-specificskills ineconomicsandengineering
– Contextualquestionnaires arealsobeinggivenatthestudent,facultyandinstitution leveltolinkthedatatostudentbackgroundsandlearningenvironments.
www.dit.ie/researchandenterprise
TeachingExcellenceFramework(UK)
• TEFwillreplaceQAA;• NewOfficeofStudent(OfS);• Spotlightonteaching,andencourage
excellentteaching;• Provideextensive informationto
studentsandemployers;• Metrics:
– Teachingquality– Learningenvironment– Studentoutcomesandlearninggain
• InstitutionswhichperformagainstTEFcanraisetuitionfee;– Institutionswhichdonot- willfail.
www.dit.ie/researchandenterprise
Scorecards&Websites
Rankings
Classification&Profiling
QualificationsFrameworks
QA&LearningOutcomes
Scorecards&Websites
Globalintelligence
SocialNetworking
www.dit.ie/researchandenterprise
www.dit.ie/researchandenterprise
www.dit.ie/researchandenterprise
www.dit.ie/researchandenterprise
QualityIndicatorsforLearningandTeaching(Australia)
• PurposebuiltwebsiteknownasQILT(www.qilt.edu.au).– ReplacesMy-Uni
• Helpsstudentsandfamiliesmakeinformedchoices;
• Dataon:– Overallsatisfactionofcurrentstudents– Overallsatisfactionofrecentgraduates– Outcomesofstudentsmovingintofull-
timeemploymentaftergraduation– Mediansalaryreceivedbyrecent
graduates.
www.dit.ie/researchandenterprise
GlobalIntelligence
Rankings
Classification&Profiling
QualificationsFrameworks
QA&LearningOutcomes
Scorecards&Websites
Globalintelligence
SocialNetworking
www.dit.ie/researchandenterprise
GlobalInitiatives
• GlobalInstitutionalProfilesProject– ThomsonReuters- Dataprofilesof700+globallysignificant institutions, combiningreputational
assessment, scholarlyoutputs,fundinglevels,facultycharacteristicsandmuchmoreinonecomprehensivedatabase.
• GlobalResearchUniversityProfiles(GRUP)– ARWU- Comprehensivedatabaseandbenchmarkingtoolcovering1200research
universitiesacross40indicators,allowinguserstolearnaboutperformanceoflocalandinternationalpeersforvariouspurposes.
• TimesHigherEducationGlobalProfilesProject– THE- PreviouslyassociationwithTR,nowwithScopus– planstodevelopitsown
globaldatabase.
• EuropeanTertiaryEducationRegister(ETER)– EuropeanUnion- DatabaseofEuropeanHEIs ,36countriesand2,673HEIs;underheadingse.g.
staffandstudents,subjectdomains,researchactivityandexpenditures.
www.dit.ie/researchandenterprise
SocialNetworking
Rankings
Classification&Profiling
QualificationsFrameworks
QA&LearningOutcomes
Scorecards&Websites
Globalintelligence
SocialNetworking
www.dit.ie/researchandenterprise
Student“TripAdvisor”
www.dit.ie/researchandenterprise
3.IMPLICATIONSFORHIGHEREDUCATION
www.dit.ie/researchandenterprise
ShapeofThingstoCome
Brief survey of recent developments shows what is coming down the track –what is happening internationally, and puts US experience in broader context.
• HE has always been competitive, but it must now learn to survive in thismore complex, and globally competitive environment in which:
- Quality and Excellence are key mantra;- Greater public scrutiny and transparency are normalised;- National prominence is no longer sufficient;- Survival no longer guaranteed.
• There is likely to be a “savage sorting of winners and losers” (Sassen, 2011)
www.dit.ie/researchandenterprise
WouldtheDebateHappenedOtherwise?
• Academy has been slow to engage meaningfully in discussion aboutquality;- Efforts at obfuscation, “gaming” and boycott have not helped;- Toomuchemphasisonprocess;- Complaintsaboutexcessivebureaucracy,form-filling,andcostof
compliance;- Pointedtothel’Oreal factor:becausewe’reworthit;
• Information deficit has created opportunities for the public andgovernments – and commercial interests – to define quality for their ownpurposes.
www.dit.ie/researchandenterprise
BeCarefulWhatYouAskFor
• Publicdiscourseinrecentyearshasfocusedattentionon:– Meetingneedsofnationaleconomies,– Concernsaboutgraduateemployabilityandaffordability– Response toconcernsaboutvalueandimpact;
• Noticeableshifttomeasuringteachingandlearningoutcomesthatallowpublictojudgewhethergraduateshavethethresholdqualitiesexpected;
• Rankingsinitiallyfilledthisgapbuttheyfocustoonarrowlyoneliteuniversitiesandresearch;
• Arguably– manyofthenewdevelopmentsarearesponse totheover-emphasisonresearch ratherthanT&L.
www.dit.ie/researchandenterprise
StartDrivingtheBus
• Institutional and country-based evaluations are rapidly being overtaken byinternational efforts.
• Public/political support for universities and colleges only maintained byassurances of quality because society, students and others have a right toknow whether the institutions are capable of meeting expectations.
• Little doubt that determining and measuring quality is complex;- Lots of “good practice” but no agreed definition;- Difficult to compare diverse institutions across jurisdictions;
• BUT – however worthy, criticisms of government/international efforts are– they miss the ‘politics’ of the situation.
• Challenge is how does HE respond? Can we afford to wait?
Ifdon’t cookowndinner– don’tcomplainwhat’sservedtoyou.
www.dit.ie/researchandenterprise
HigherEducationPolicyResearchUnit(HEPRU)
[email protected]://www.dit.ie/hepruhttps://www.researchgate.net/profile/Ellen_Hazelkorn