Quality Management, Performance
Measurement and Indicators in Higher
Education Institutions: Between Burden,
Inspiration and Innovation
Cláudia S. SarricoHigher Education TeamOECD Directorate for Education and Skills
SQELT International Evaluation Workshop1-2 July 2019Danube University Krems
Rationale for performance and quality management in higher education
2
Higher education is rapidly expandingShare of 24-34 year-olds with a tertiary degree across OECD and G20 countries
Note: The figures in these graphs are estimates based on available data. The population estimations are based on the OECD annual
population projections.
Source: OECD (2015), "How is the global talent pool changing (2013, 2030)?”, Education Indicators in Focus, No. 31,
https://doi.org/10.1787/5js33lf9jk41-en.
China, 17%
United States, 14%
India, 14%
Russian Federation, 10%
Japan, 6%
Indonesia, 4%
Brazil, 4%
Korea, 4%
Mexico, 3%
United Kingdom, 3%
France, 2%
Germany, 2%
Canada, 2%
Turkey, 2%
Spain, 2%Poland, 2%
Argentina, 1%Italy, 1%
Australia, 1%Saudi Arabia, 1% Other, 6%
137 million 25-34 year-olds with tertiary
education
China, 27%
United States, 8%
India, 23%
Russian Federation, 4%
Japan, 3%
Indonesia, 5%
Brazil, 5%
Korea, 2%
Mexico, 2%
United Kingdom, 2%
France, 1%Germany, 2%
Canada, 1%
Turkey, 2%Spain, 1%Poland, 1%
Argentina, 2%Italy, 1%Australia, 1%
Saudi Arabia, 3% Other, 6%
300 million 25-34 year-olds with tertiary
education
2013 2030
3
4
But there are quality challenges…Percentage of graduates with low literacy and numeracy
0 20 40 60 80
JapanNetherlands*
Czech RepublicFinland
SwedenFlanders*Norway*Australia
Slovak RepublicUnited StatesNew Zealand
AustriaNorthern Ireland…
PolandFrance
DenmarkEngland (UK)
GermanyKorea
IrelandAverageCanada
Estonia*Slovenia
LithuaniaItaly
SpainIsrael
GreeceTurkey
Chile
Literacy skills
0 10 20 30 40 50 60 70 80
Czech RepublicFlemish Community*
Netherlands*Slovak Republic
JapanSwedenAustria
Norway*Finland
DenmarkGermany
FranceLithuaniaSloveniaAustraliaAverage
Northern Ireland (UK)New Zealand
PolandEstonia*
United StatesEngland (UK)
KoreaIreland
CanadaItaly
SpainGreece
IsraelTurkey
Chile
Numeracy skills
Below level 1 Level 1 Level 2
Source: OECD (2016), Skills Matter: Further Results from the Survey of Adult Skills, https://doi.org/10.1787/9789264258051-en.
5
And equity challenges…Access rate gaps for 18-24 year-olds
-80
-70
-60
-50
-40
-30
-20
-10
0
Panel A - Bachelor’s or long first degree programmes Percentage change in the probability to enter a higher education programme for 18-24 year-olds whose
parents did not attain higher education and for those whose parents are foreign-born
-100
-80
-60
-40
-20
0
20
40
60
Chile Slovenia Norway* Average France Sweden Latvia Portugal
Panel B - Short-cycle programmesPercentage change in the probability to enter a higher education programme for 18-24 year-olds whose
parents did not attain higher education and for those whose parents are foreign-born
Parents without higher education Foreign-born parents
How to read this chart: Panel A: In Slovenia, 18-24 year-olds without tertiary educated parents are about 40% less likely to enter a bachelor’s or long first degree programme than other
18-24 year-olds. Panel B: In Chile, 18-24 year-olds without tertiary educated parents are about 40% more likely to enter a short-cycle programme than other 18-24 year-olds.
Source: Indicators of Education Systems (INES) Survey on Equity in Tertiary Education.
6
And relevance challenges…Businesses collaborating on innovation with higher education or research institutions
0
5
10
15
20
25
30
Universities or other higher education institutions Government or public research institutes Private research institutes
53 42
Source: Adapted from Eurostat (2018), Community Innovation Survey, Science, Technology and Innovation Indicators,
http://ec.europa.eu/eurostat/web/science-technology-innovation/data/database.
7
And the costs of higher education are becoming
increasingly difficult to manage…
90
100
110
120
130
140
2005 2010 2015
Expenditure on education institutions and number of students by education level, 2005=100
Total expenditure, higher education Number of students, higher educationExpenditure per student, higher education Total expenditure, primary to post-secondary non-tertiary educationNumber of students, primary to post-secondary non-tertiary education Expenditure per student, primary to post-secondary non-tertiary education
Source: Adapted from OECD (2018[3]), Education at a Glance 2018, https://doi.org/10.1787/eag-2018-en.
From Burden to Inspiration and Innovation
8
Questions
Erosion of trust
Accountability mechanisms
Attempt to restore trust
Quality?
Equity?
Relevance?
Financial sustainability?
Normative perspective on trust
Rationalist-instrumental perspective on trust
(Stensaker and Harvey 2011)
(Stensaker and Gornitzka 2009)
Burden
9
10
Benchmarking Higher Education Systems
Highlights
https://doi.org/10.1787/be5514d7-en
Higher education provides graduates with favourable economic and social outcomes, but the low basic skills of some graduates is a cause for concern
Inequity of access by socio-economic and migration background is a persistent challenge
Only 4 in 10 bachelor’s students are able to complete on time, and 2 in 10 do not complete at all
Young doctorate holders in higher education employment find less job security than their predecessors and their peers in other sectors
Higher education research and development relies heavily upon public funding, and establishes limited collaboration with businesses on innovation, especially for small and medium enterprises
There is an increasing focus on engagement activities, but frameworks for measuring activities do not yet exist
Open access to scientific documents remains limited
Although quality is difficult to measure, governments are increasingly trying to link funding and other policies to the quality of teaching and research
Data limitations prevent comprehensive performance assessment of higher education systems, but improvements in measurements are possible
11
Benchmarking Higher Education Systems
Source of inspiration and innovation
https://doi.org/10.1787/be5514d7-en
Benchmarking approach
Higher education
system benchmark
Higher education
system
Policy
benchmarking
Comparison of
policies
New ideas, new policies,
new practices
Creativity and innovation
Peer learning
Comparison of
metrics
Metric
benchmarking
Inspiration
Innovation
12
Country reviews
The mobilisation, allocation, and management of financial resources: public funding for teaching, research, and infrastructure; private revenues raised by higher education institutions; student fees and student financial support
Human resources management: attracting, recruiting and selecting the higher education workforce, the structuring of the higher education workforces, and inducing the desired performance from the higher education workforce
Resource governance and coordination: coordinating demand and supply (study places, programmes, and institutions), the network of provision (institutional collaboration, alliance, and mergers), and student pathways.
New Higher Education Resources Project
Country reviews and benchmarking policy briefs
Attracting, recruiting and selecting the
HE workforce
Recruitment process
Staff profile
Staff numbers
Structuring the higher education
workforce
Employment status of academic staff
Academic roles and working time
Digitalisation of teaching and learning
Inducing the desired performance from the HE workforce
Staff appraisal
Promotion
Remuneration
Professional development
Mobility
Retirement
14
New Higher Education Resources Project
Human Resources
What is happening to quality and performance measurement and
management?
15
Performance Model
objectives inputs activities outputs
intermediate outcomes
final outcomes
needssocio-
economic situation
efficiency
cost-effectivenesseconomy
value and sustainability
effectivenessrelevance
higher education system
Context
(Talbot 2010, Bouckaert and Halligan 2008)
(Sarrico, 2018) 16
• Different measurement and management instruments
• Steering mechanisms: professional, state and market
regulation
• Performance indicators
– Economy
– Efficiency
– Quality: from internal to external quality, from intrinsic to
extrinsic motivations, from improvement to accountability
17
Towards integration and squaring the circle
Towards integration and squaring the circle
• Integrative frameworks, at system, institution and unit
level
– National and supra-national (ENQA, INQAAHE, OECD,
UNESCO, World Bank)
– Institutional – increasingly integrated:
• In wider management and governance arrangements
• Different missions: learning and teaching, research and scholarship,
engagement
• Different organisational levels: institution, sub-units
• Different quality and performance dimensions
18
What has resulted from the quest for performance in higher education?
19
More and better?
• Research productivity and quality up
– Economic and social impact?
• Rankings, reputation and the quest for world-class universities
– Lack of attention to the quality of teaching and meaningful engagement
with the wider world
• Poor education provision less likely
– Study success increasingly addressed
– Learning outcomes and learning gain?
– Graduate labour market outcomes?
– Wider social outcomes of graduates?
20
More and better?
• Engagement
– Emphasis on technology transfer and commercialisation of research
– Less on wider civic and social engagement
– Difficulty in measuring ‘valorisation’
• Performance-based funding in addition to basic government allocations
– Ex-post – reward for good past performance
– Ex-ante – performance agreements
• Growing importance of third-party funding in addition to core funding and student fees
– Continuing Education
– Knowledge and technology transfer
– Service provision
– Endowments and other philanthropic donations
21
What does the future hold?
22
Some possible trajectories
• Self-accountability -> societal accountability -> societal engagement(Hazelkorn, 2016)
• Bias towards research addressed
– Measurement of learning outcomes and learning gain
– Initiatives to improve the quality of teaching
• More attention to human resources management and professional
development
• Valorisation agenda
– More emphasis on the social impact of higher education
– More engaged graduates, knowledge exchange, and civic and social
engagement
• From ‘world-class universities’ to ‘world-class systems’
23
Top Related