The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework

Targets & Performance ManagementWorld Bank’s Public Sector Performance

Global Expert Team 9 May

Gwyn Bevan

Models of delivery of public services

Knights / Knaves

Econs / Humans

Prospect theory

Trust & Altruism (T&A)

Knights Humans Reward failure?

Hierarchy & Targets (H&T)

Knights & Knaves

Econs Sack staff?

Public Ranking (PR) Knights Humans Reputation?

Choice & Competition (C&C)

Knights & Knaves

Econs Winners & losers?

UK’s natural experiment: before & after devolution (1999)

NHSs Schools

Before After Before After

EnglandT&A

H&T

PR

C&C

PRWales T&A C&C

PR

C&C

H&T+PR: ‘naming & shaming’ spotlight + prospect theory

Staff in zero-rated hospital ‘devastating …hit right

down to workforce -- bad reports usually hit senior management upwards

nurses demand changing rooms .. because being accosted in streets’

Source: Mannion et al (2005)

‘dirty dozen’ zero-rated CEOs(English NHS)

Hospital waiting times: T&A H&T+PR

Numbers waiting elective admissions (England) (‘000s)

H&T+PR

Hospital waiting times: T&A

0

10

20

30

40

50

60

70

1997 1999 2001 2003 2005

England Wales

No/‘000 waiting > 3 months GP 1st outpatient appointment

Hospital waiting times. Natural experiment: T&A vs H&T+PR

051015202530354045

1997 1999 2001 2003 2005

England Wales

No/‘000 waiting > 3 months GP 1st outpatient appointment

H&T+PR

Ambulances:T&A

45

50

55

60

65

70

75

80

1999 2001 2003 2005 2007

England Wales

Source: Bevan & Hamblin (2009)

% responses life-threatening emergencies < 8 minutes Target

E & W

Ambulances. Natural experiment: T&A vs H&T+PR

45

50

55

60

65

70

75

80

1999 2001 2003 2005 2007

England Wales

Source: Bevan & Hamblin (2009)

% responses life-threatening emergencies < 8 minutes H&T+P

RTarget E

Target W

35

45

55

65

1994 1996 1998 2000 2002 2004 2006 2008

England Wales

% pupils achieving > 5 good grades at 16

Note: data are missing for 1998, 2001 & 2005 & these have been estimated as the mean values from adjacent years

Schools: C&C+PR

35

45

55

65

1994 1996 1998 2000 2002 2004 2006 2008

England Wales

Schools. Natural experiment: C&C+PR vs C&C

Note: data are missing for 1998, 2001 & 2005 & these have been estimated as the mean values from adjacent years

Wales stops PR

% pupils achieving > 5 good grades at 16

H&T & PR: design

Residual domain R

Gages on P

No measures on P

Tin openers on P

Selected domain P

H&T & PR : problems of gaming

Selected domain T: incentives apply to measures M[T]

Excluded domain N: unimportant or cannot be measured

Holmstrom & Milgram (1991)

H&T & PR: types of gaming

N? Worsen

performance? M[T]?

Hit the target & miss the point

Misreporting & cheating

CollusionTarget M[T]

Domain N

Domain T

For discussion

Theory of contingency mix of models

T&A performance measurement no gaming ineffective

PR + H&T powerful gaming Design to counter gaming

C&C choice attractive in principle Characteristics influence effectiveness

For discussion C&C: Requirements

Universities Schools Hospitals

Money follows choice Supply-side flexibility Information Transaction costs Management freedom

COFEPRIS: Commission for Protection against Sanitary Risks

9,000 registrations (value $1bn) required 150 registered in 2010

Elements for transformation of failing organisations

English NHS COFEPRIS

Spotlight effect

Naming & shaming in media

Inform media of failure

‘Broken windows’

Waiting times Registrations

Prospect theory

Sack ‘failing’ CEOs Sack underperforming middle managers

Rewards Name elite organisations

Promote young able managers

COFEPRIS: Sanitary registrations

Source: Mikel Arriola Peñalosa (2012) Federal Commissioner for the Protection against Sanitary Risks