EuropeAidEvaluation

of Budget Support:

Methodological

Approach and Results

Jurgen LovaszDEVCO B/2

EU Expert Group on Results, April 2012

EuropeAid

1.Issue paper (May 2008);

2.Common methodological documents approved by the OECD/ DAC Steering Group;

3.Three tests of the methodology in Tunisia, Mali (finalised) and Zambia (under finalisation);

4.Short synthesis of the findings of the 3 tests for Busan HLF;

5.First International Workshop in Brussels on the refined methodological approach in March 2012

6.Dutch Government finances drafting of Standard Methodological Approach for Evaluation of BS based on former docs and results of pilot studies. Finalisation in April 2012

Where we are

EuropeAidNext Steps

• Pilot 2 days internal EC “Familiarisation” Workshops (June/July 2012)

• 3 days Training Workshops mainly for potential evaluators of BS Evaluations; as of October 2012

• Next joint evaluations of budget support:Confirmed:

• Tanzania (EC in lead, launched in Jan.2012);• Burundi (Belgium in lead);

To be decided:• Asia: Nepal, Philippines (?) (EC in lead)• Morocco (EC in lead)• Mozambique (interest from EC, Italy, Sweden, Finland

and Netherlands);• Ghana (in 2013?)• Sierra Leone (in 2013;EC/DFID in lead?)

EuropeAid

EXISTING BUDGET SUPPORT EVALUATIONS

2004 Tanzania

2006 Joint Evaluation of 7 countries from 1994 to 2004 budget support

2007 Ghana

Evaluations addressing mainly THE PROCESSAnd not enough the CHAIN OF RESULTS

IMPACTS AT THE GLOBAL LEVEL

IMPACTS ON SECTOR POLICIES

EuropeAid

Other context factors

Other context factors

BUDGET SUPPORT

GOVERNMENT OUTPUTS

ECONOMIC and SOCIAL

IMPACTS

EuropeAid

EuropeAid

What can we see behind, what is easily visible?

Main Findings and Results

EuropeAidTunisia

•7 out of 12 operations financed by the European Commission over the period 1996-2009.

•Four GBS and three SBS, covering secondary and tertiary education and vocational / educational training

•GBS: European Commission, World Bank, AfDB

•SBS: European Commission

EuropeAid

•32 out of 33 operations financed by 11 donors over the period 2003-09,

•GBS: AfDB, WB-IDA, Canada, Denmark, European Commission, France, Netherland, Sweden

•SBS - PFM: Canada.

•SBS – social sectors: Belgium, Canada, France, The Netherlands, Sweden-Norway, Spain

•SBS – decentralisation: EC

Mali

EuropeAidZambia

•Poverty Reduction Strategy Budget Support and 3 Sector Budget Support (roads, public financial management, health) over the period 2002-2009

•PRSBS: European Commission, Finland, Germany, The Netherlands, Norway, Sweden and United Kingdom

•SBS: European Commission and United Kingdom

EuropeAid

Contribution of Budget Support to the aid framework and partner country policies

Contribution to the implementation of the Paris Declaration and AAA

Increased financial resources and strengthened budget management

Adapted design, positive dialogue frameworks and diversified use of TA

Main findings

EuropeAid

But ...

Although harmonisation and alignment improved, they far from

completed :- donors often support less coordinated initiatives

- unrealistic PAF conditions and targets (donor driven indicators)

- multiplication of coordination structures

Main findings

EuropeAid

BS increased discretionary expenditure and allocative efficiency

And

Effective support to implementation of Public Financial Management & other reforms when government and society are committed:

Budget Support cannot buy reforms

Main findings

EuropeAid

BS increased public investment and reform implementation that produced significant

outcomes in education and health.

In the three countries, the reforms and related investments supported by higher levels of public expenditure produced significant outcomes and some measurable impacts as well

Main findings

EuropeAid

BS contributed partly to sound macro economic policies, growth and private sector development

Tunisia: “Strong Opportunity Framework” including BS contributed to growth and private sector developmentMali/Zambia: While BS funds contributed to sound macro economic policies, other important factors (regional integration, export prices, debt relief, etc) also played an important role

Main findings

EuropeAid

BS contribution to growth and social policies improved income and non-income poverty

But

Rural poverty has not been tackled adequately by government (Zambia)

Main findings

EuropeAidKey Lessons

- BS is an effective tool in a country with robust policies

- BS cannot determine major policy changes but can provide sound support for policy reforms, that is not possible with only projects

- BS is more beneficial with high degree of harmonisation and alignment

- BS is strengthened and multiplied when it is linked to wider political and economic partnerships

EuropeAid

Cont.: Key Lessons

- BS has been an instrument conducive to increasing transparency and accountability of budget processes and public expenditure management and may be used to support anti-corruption policies

- BS benefits if it is complementary with other forms of aid, especially for capacity building

- The choices between GBS and SBS should be flexible and based on the context.

- Improved documented policy dialogue, data production and policy assessments of development results is required

EuropeAid

THE

METHODOLOGY AND FINDINGS

ON

EuropeAid

THANK YOU!

EuropeAid

EuropeAidMali: Example of application of methodology

EuropeAidMali: Example of application of methodology

EuropeAid

GBS/SBS COMPREHENSIVE EVALUATION FRAMEWORK

Step 1

Step 2

Step 3

EuropeAid

1.Inventory of the inputs provided

2.Assessment of the performance of these inputs in terms of direct outputs and induced outputs.

3.Identification of the changes which have occurred during the period in the partner country and the key causal factors.

4.Assessment of the extent to which budget support (may) have contributed to the results identified at the outcome and impact levels.

Expected Results of the Budget Support Evaluation

EuropeAid

Formulation/Design

Evaluation period

« Planning » for Evaluation as

regards:-Data devel.-- surveys

-- indicator devel.

Dev + gather info on:-data

-PFM assessments-Suveys

Document PD

Evaluation based on material developed and

gathered before;Additional material may

be required

Learn from evaluation

resultsFollow up

recommendations

How to reflect ‘Evaluations’ in PCM

Implementation