Evaluating the multiple

missions of higher

education

Richard YellandOECD Directorate for Education

20th EURASHE Conference, Tallinn, 14 October 2010

U-Multirank

• Multi-dimensional; global

• Feasibility study

• A range of indicators

– Teaching and learning

– Research

– Knowledge transfer

– International orientation

– Regional engagement

What we know about HE quality…

Proxies of higher education quality exist, but none are perfect

- Rankings often biased towards input factors and

research

- Subjectivity of reputation factor

- Cultural sensitivity of satisfaction factor

- Labour market outcomes sensitive to conjuncture and

local economic conditions

So

what?Learning outcomes as a promising direction

- Defining them (Tuning process in Bologna area)

- Incorporating them in quality assurance processes

- Measuring them (AHELO)

The massification of participation in higher education has meant

much more heterogeneous abilities and expectations of students

than in the past

The OECD AHELO feasibility

studyWhat is AHELO?

Why undertake the study?

Why is AHELO important?

A ground-breaking initiative to assess HE learning outcomes on an

international scale, by creating measures that would be valid:

• For all cultures and languages

• And also for the diversity of HE institutions

• Employs a wide range of measures

• Provides faculties, students and government agencies with a more

balanced assessment of HE quality – not just research-driven rankings!

• No sacrifice of HEIs’ missions or autonomy in their subsequent efforts to

improve performance

After decades of quantitative growth in HE, consensus on the need to ensure

quality for all (Athens, 2006)… but information gap on learning outcomes

Carry out a feasibility study to provide a proof of concept (Tokyo, 2008)

AHELO: 4 strands of work

Initial work on defining expected

learning outcomes

through ‘Tuning’ approach.

+ contextual data

Discipline strand

in Engineering

Initial work on defining expected

learning outcomes

through ‘Tuning’ approach.

+ contextual data

Discipline strand

in Economics

Several perspectives to

explore the issue of value-

added (conceptually,

psychometrics), building on

recent OECD work at

school level.

Research-based “Value-

added” or “Learning gain”

measurement strand

International pilot test of the US

Collegiate Learning Assessment (CLA),

to assess the extent to which problem-

solving or critical thinking can be validly

measured across different cultural,

linguistic and institutional contexts.

+ contextual data

Generic skills strand

Work to be undertaken in 2

phases

Generic Skills

Framework

EconomicsFramework

EngineeringFramework

Project management,survey operations and

analyses of results

Contextual dimension surveys

Frameworks

Instrument development &

small-scale validation

Generic Skills

Instrument

EconomicsInstrument

EngineeringInstrument

Implementation

Phase 1 -Initial proof

of concept

Phase 2 -Scientific feasibility & proof of

practicality

OECD’s tool to mobilise Higher Education for Economic, Social and Cultural Development of their Cities and Regions and to enhance regional partnership building:

… the Reviews of Higher Education in Regional and City Development

Reviews of Higher Education in

regional & city development 2008-10

2005-07: 14 regions in 12 countries

2008-10: 15 regions in 11 countries

OECD countries: 12 regions

Australia State of Victoria

Chile Bío Bío Region

Germany City of Berlin

Israel The Galilee

Italy Lombardy

Mexico State of Veracruz

Netherlands AmsterdamRotterdam

Spain Andalusia & Catalonia

US Southern Arizona

US-Mexico Paso del Norte Region

Non-member economies: 2 regions

Brazil State of Paraná

Malaysia Penang

Review visit to Free State, South Africa in October 2010

2011-12: 3rd round: recruitment

How are the reviews done?

• Regional Self-evaluation report lead and owned by the regional Steering Committee using OECD guidelines

• Visit by the OECD-led international expert team

• OECD Review Report tailored for the region

• Analysis and synthesis by OECD

• Dissemination of outcomes (roundtable meetings, national and regional seminars, international meetings)

What is being reviewed?

Partnership building

The OECD reviews draw

together HEIs and public

and private agencies to

identify strategic goals and

to work towards them.

Importance of context

The reviews take account

of different national and

regional contexts within

which the HEIs operate.

HEIs

Regional capacity building

Human capital and skills development

Social and cultural development

Regional innovation

National and regional context

Focus of analysis of the OECD review

Issues arising

• New ways of approaching SMEs

• Widening innovation concept

• Widening access an emerging or revitalised policy

agenda

• Green Growth and sustainability gaining momentum

• Higher education serving urban regeneration and

community development

• Initiatives to combine aspirations for world class and

regional engagement

Thank you

richard.yelland@oecd.org

Twitter: @RichardJYelland

www.oecd.org/edu/higher