What can we learn from other school systems about improvement?

Comments on How the world’s most improved school systems keep getting

better

David Raffe, University of Edinburgh

Parliamentary Research Enquiry, 28 June 2011

TWO APPROACHES TO CROSS-NATIONAL POLICY ANALYSIS

Policy borrowing: search for ‘best practice’ which is unique and transferable

Policy learning: using foreign experience in a variety of ways to improve understanding and to support policy development at home

THE McKINSEY APPROACH

A sophisticated version of policy borrowing: Focus on change (improvement) Relate ‘best practice’ to the starting point

(‘improvement journey’) Contextualise interventions

… but still a ‘policy borrowing’ or ‘best practice’ approach (just one improvement journey)

THE McKINSEY APPROACH Identify improving systems … … at different stages of ‘improvement journey’ Ask system leaders about their interventions Abstract common features … … define ‘intervention clusters’ for each stage, and

across all stages (see Exhibit 8 in report)

Approach begs three questions …

1. ARE THESE ACTUALLY ‘IMPROVING’ SYSTEMS?

Probably, but remember: There are other criteria of good performance (and

therefore of improvement) Issues of comparative measurement Goodhart’s Law

2. ARE THEY ACTUALLY MAKING THE INTERVENTIONS DESCRIBED IN THE REPORT?

Not as obvious as the report suggests, because: Information is obtained from within the ‘policy

bubble’ ‘Bottom-up’ and critical perspectives are not invited Claims to have decentralised power, promoted

professionalism, etc are not challenged or tested

3. IS THE IMPROVEMENT DUE TO THESE INTERVENTIONS?

Doubtful, because: Methodology probably downplays variation in

interventions of different ‘improving’ countries Wrong comparison – with ‘improvers’ at other stages

of improvement journey not with ‘non-improvers’ Choice of interventions largely dictated by

circumstances: ‘non-improvers’ probably made same interventions

Processes and contexts treated as secondary

McKINSEY: AN OVERVIEW Not a reliable guide to choosing policy interventions Flawed example of a ‘policy borrowing/best practice’

approach … but even a better designed ‘best practice’ study

would (probably) have been inconclusive The alternative is a ‘policy learning’ approach … … for which McKinsey provides useful material

A ‘POLICY LEARNING’ APPROACH: SOME EXAMPLES

Use cross-national comparisons: To understand own system better To explore policy options and examples of good (not

best) practice To understand processes, dynamics of change,

issues, etc Not to identify ‘best practice’ to be taken off-the-peg

A ‘POLICY LEARNING’ APPROACH (CONTINUED)

Learn from your own history It’s probably been tried before Look for (national) path dependence rather than

(shared) improvement journey

Identify strengths and build on them Avoid deficit thinking Eg: tradition of local innovation

STRENGTHS OF ENGLISH SYSTEM? (14-19)

“Institutional flexibility and responsiveness; a tradition of local process-based innovation; ‘alternative routes’ and second-chance pathways; the absence of over-extended pathways and the opportunity for relatively early entry to the labour market; a tradition of pastoral care; a tradition of learning in and through employment; a labour market which supports relatively high levels of early career mobility; and young people with a strong sense of agency.” (Raffe 2002)

“To which I would add: a growing respect for cultural diversity, an openness to new ideas and a capacity for institutional innovation.” (Coffield 2002)

A ‘POLICY LEARNING’ APPROACH (CONTINUED)

Facilitate horizontal learning McKinsey’s ‘intermediate layer’

Facilitate vertical learning Learn from implementation research: bottom-up and

top-down Don’t filter out unwelcome messages ‘Double loop’ as well as ‘single loop’ learning

A ‘POLICY LEARNING’ APPROACH (CONTINUED)Very diverse and fragmented systems don’t learn well Less horizontal learning ‘Other people’s children’ syndrome Inherent problems (contextual effects, etc)

Teachers, classrooms and pedagogies matter more than schools

So why do so many improvement strategies focus on schools?

A ‘POLICY LEARNING’ APPROACH (CONTINUED)Professionalism and devolution are important, but; Hard to ‘mandate’ Needs trust, experience, cultural change – and time Undermined by mixed or changing policy messages Don’t assume they’ve been achieved because the

policy says so

Don’t exaggerate potential impact of policy interventions Be humble!