Post on 13-Feb-2016
description
VIII KONFERENCJA EWALUACYJNA
METHODOLOGICAL DEVELOPMENTS AND CHALLENGES IN UK POLICY EVALUATION DANIEL FUJIWARA
VIII Evaluation Conference‘Methodological Developments and Challenges in UK Policy Evaluation’
Daniel FujiwaraSenior Economist Cabinet Office & London School of EconomicsWarsaw, November 2012.
Policy evaluation in the UK
Analysts in government have a long history and tradition of policy evaluation.
The Magenta Book
Assessing whether the policy had a causal impact on the outcomes of interest.
The Green Book
Cost-benefit analysis is the key tool for policy assessment. CBA enters at the Appraisal stage
Appraise policy interventions in terms of their social costs and benefits. Further supported through the Social Value Act (2012)
Challenges to policy evaluation
● Some constraints on use of impact evaluation findings:
i. Are the impact and valuation estimates robust?
ii. Have we acknowledged and included all possible impacts?
iii. Are there some things that are considered separately – eg, merit goods, distribution (of income, health etc).
Challenges to policy evaluation: Impact measurement (1)
Identifying and measuring thecounterfactual
Many public sector organisations moving to the Maryland Evaluation scale.
‘What Works’ institutes like Washington State Institute for Public Policy.
Scale ranks howwell counterfactuals have been measured in the analysis.
Level Design Statistical method
5 Randomised trials Evaluations with well implemented random assignment of treatment to subjects in treatment and control groups.
4 Quasi-Experiments
Evaluations that use a naturally occurring event (that makes the treatment assignment as good as random)
3 Matchingtechniques;Regressionanalysis
Non-experimental evaluations where treatment and comparison groups are matched on observable characteristics
2 Simplecomparisons
Studies with a treated and comparison group, but with no attempt made to control for differences among the groups.
1 Pre- and postanalysis
Studies where no comparison group is used. Outcomes are measured pre and post-treatment.
Challenges to policy evaluation: Impact measurement (2)
Level 5 study designs are being increasingly used for public policy interventions.
RCTs used in tax policy, environment policy and labour market interventions.
Challenges to policy evaluation: Valuation
Supplementary Green Book guidance provides guidelines on how to value social impacts
Preference methods or Wellbeing Valuation ?
Different methods are increasingly allowing us to value and include a broad range of impacts
Challenges to policy evaluation: Summary
These developments and new guidelines for policy evaluation help us to understand the impacts of a policy better and to continuously improve the success of policy interventions.
Policy interventions are enhanced during this process through the use of evidence from behavioural science.