Evaluation of Country Programmes Between Finland and Nepal,

Nicaragua and Tanzania

Oral presentationFebruary 7, 2012

Julian CALDECOTT, Team Leader,Julian CALDECOTT, Team Leader,Michael HAWKES, Nepal Country expertMichael HAWKES, Nepal Country expert

Fred van SLUIJS, Nicaragua Country expertFred van SLUIJS, Nicaragua Country expertArto VALJAS, Tanzania Country expertArto VALJAS, Tanzania Country expert

Bernadeta KILLIAN, Tanzania national expertBernadeta KILLIAN, Tanzania national expert

Introduction to the evaluationIntroduction to the evaluationJulian CALDECOTTJulian CALDECOTT

AimsAims

FocusFocus on 2002-2011: 1998, 2001, 2004, 2007 policy on 2002-2011: 1998, 2001, 2004, 2007 policy frameworks.frameworks.

InfluencesInfluences: policy, actors, events: policy, actors, events MechanismsMechanisms: programming, dialogue, management: programming, dialogue, management Aid effectivenessAid effectiveness, Finnish added value, cross-, Finnish added value, cross-

cutting themescutting themes StrengthsStrengths, weaknesses, lessons learned., weaknesses, lessons learned.

2

MethodsMethods

A team A team consisting of a Team leader, 3 International consisting of a Team leader, 3 International country experts, 3 National experts and a country experts, 3 National experts and a ResearcherResearcher

Interviews Interviews (300 people) in Finland and field(300 people) in Finland and field Literature: Literature: reports, archives, publicationsreports, archives, publications Feedback Feedback from embassies and expertsfrom embassies and experts.. SelectedSelected field visits field visits 14 criteria 14 criteria (Relevance, Efficiency, Strategic

effectiveness, Impact, Sustainability, Coordination, Complementarity, Compatibility, Connectedness, Coherence, FAV, Partner satisfaction, Programming logic, Replicability).

3

ReportsReports

3 Country reports and a Policy brief3 Country reports and a Policy brief Context: Context: history, governance, needs, donorshistory, governance, needs, donors Evolution: Evolution: sectors, projects, circumstancessectors, projects, circumstances Performance: Performance: the 14 criteriathe 14 criteria Highlights: Highlights: influences, policy-into-practice, aid influences, policy-into-practice, aid

effectiveness, FAV, etc.effectiveness, FAV, etc. Lessons learnt and key recommendations.Lessons learnt and key recommendations.

4

Nepal country evaluationNepal country evaluationMike HAWKESMike HAWKES

The evaluation period encompasses the The evaluation period encompasses the introduction of the 2004 and 2007 Finnish introduction of the 2004 and 2007 Finnish development policies and the climax, resolution development policies and the climax, resolution and aftermath of the decade-long armed conflict in and aftermath of the decade-long armed conflict in NepalNepal

5

Volume and sources of aidVolume and sources of aid

2009: overall ODA almost US$ 500 million. 2009: overall ODA almost US$ 500 million. (nearly 3.5 per cent from Finland).(nearly 3.5 per cent from Finland).

Top bilateral donors (94% total): UK, Top bilateral donors (94% total): UK, United States, Germany, Norway, Denmark, United States, Germany, Norway, Denmark, EC, Japan, Switzerland and Finland.EC, Japan, Switzerland and Finland.

These figures exclude aid from India and These figures exclude aid from India and China.China.

7

8

9

Spatial data capture analysis and Spatial data capture analysis and interpretationinterpretation

Topographic mapping. Topographic mapping. Forest Resource Assessment.Forest Resource Assessment.

10

Water Sanitation and HygieneWater Sanitation and Hygiene

Consistent investment.Consistent investment. Focus on genuine needs. Focus on genuine needs. Concerns.Concerns.

11

EducationEducation

Consistent support to SWAp.Consistent support to SWAp.

Modest financial but major Modest financial but major intellectual contribution.intellectual contribution.

Nepal enthusiastic.Nepal enthusiastic.

12

ForestryForestry

Withdrew 1999.Withdrew 1999. Reengaged 2007. Reengaged 2007. Forest Resource Assessment.Forest Resource Assessment. IFAD leasehold forestry project. IFAD leasehold forestry project. Multi-Stakeholder Forestry Multi-Stakeholder Forestry

Programme with UK and Programme with UK and Switzerland.Switzerland.

13

Environment and Climate changeEnvironment and Climate change

Environmental Administration and Environmental Administration and ManagementManagement

Solid Waste ManagementSolid Waste Management Natural disastersNatural disasters Regional Flood Information Regional Flood Information

SystemSystem

14

Peace-building, governance and human Peace-building, governance and human rightsrights

Basic Operating GuidelinesBasic Operating Guidelines The Nepal Peace Trust FundThe Nepal Peace Trust Fund Office of the High Commissioner for Office of the High Commissioner for

Human Rights Human Rights The National Human Rights Commission The National Human Rights Commission UN Security Council Resolutions 1325 UN Security Council Resolutions 1325

of 2000 and 1820 of 2008 on Women, of 2000 and 1820 of 2008 on Women, Peace and Security Peace and Security

15

Country programme scores for Country programme scores for evaluation criteriaevaluation criteria

Criterion Notes and scores (where ‘a’ = very good, ‘b’ = good, ‘c’ = some problems, and ‘d’ = serious deficiencies)

Relevance Overall score (b). Problems in strategic relevance.Efficiency (a).Strategic effectiveness

(a).

Impact (a).Sustainability Overall score (c). Problems in environmental

sustainability.Coordination (a).Complementarity

Overall score (b).

Compatibility (a).Connectedness (c). Problems in internal politics, external

relations, climate change.Coherence (a).Finnish added value

(a).

Partner satisfaction

(a).

Programming logic

(b).

Replicability (b).

16

Key findingsKey findings – past programming – past programming

Very strong positive influence by:Very strong positive influence by: Embassy-government relations.Embassy-government relations. Policy dialogue and country Policy dialogue and country

consultations.consultations. Embassy participation in multi-donor Embassy participation in multi-donor

mechanisms.mechanisms. Good results in peace building and Good results in peace building and

constitution building.constitution building. Mainstreaming of cross-cutting themes Mainstreaming of cross-cutting themes

was effective and strongly influential.was effective and strongly influential.

17

Key findingsKey findings – future programming – future programming

Rapid growth and Rapid growth and fragmentation.fragmentation.

Limited strategic ambition on Limited strategic ambition on climate proofing.climate proofing.

18

Nicaragua country evaluationNicaragua country evaluationFred van SLUIJSFred van SLUIJS

Introduction Context: history, governance, needs, donor activity.Evolution: sectors, interventions, circumstances.Performance: the 14 criteria, research questions, lessons learnedHighlights: political landscape, global issues, good governance  

19

Context Regions and peoples Politics and polarization (1979

FSLN, 1990-2006 Conservative Restoration, 2006 FLSN)

Donor landscape: old friends, new friends.

20

21

Evolution International influences (SWAp, SAP,

PRSP, Paris Declaration) Timeline (1989 project policy, 2001

operationalization, 2004 policy streamlining, 2005 decentralization pilot.

Architecture (four pronged approach) Challenges of transition 2008-2011

22

PerformanceApplying the evaluation

criteriaResearch questions

23

Highlights and issues Nicaragua as a special case Re-balancing of powers (ALBA, China) Global issues (e.g. climate change,

security): old problems, new challenges.

The meaning of good governance. Values in development cooperation:

presence in difficult times.

24

Lessons learned

Good practices (e.g. sustainable institutionalisation, donor coordination)

Bad practices (e.g. vulnerable basket funds, weak dialogue)

25

Tanzania country evaluationTanzania country evaluationArto VALJAS and Bernadeta KILLIANArto VALJAS and Bernadeta KILLIAN

Development ContextDevelopment ContextCountry overview Country overview Political contextPolitical contextEconomic growthEconomic growthDevelopment strategiesDevelopment strategiesDual development approachDual development approach

26

Donor community Donor community

Donor darlingDonor darling Front runner in coordination and Front runner in coordination and

harmonisation harmonisation Front runner in aid effectiveness Front runner in aid effectiveness Donor response to the dual Donor response to the dual

development approachdevelopment approach

27

Development Results Development Results

MDGs Income poverty Public consumption in health

and education High growth areas FDI and trade (in particular

from/with China)

28

Evolution of the country Evolution of the country programmeprogramme

2005-09 period of growth in volume2005-09 period of growth in volume 2010 unplanned downturn in 2010 unplanned downturn in

volumevolume Areas of cooperation: Areas of cooperation:

Beginning Beginning MidwayMidway EndEnd

29

Main areas of cooperation Main areas of cooperation

GBSGBS Local government reformLocal government reform Forestry/land useForestry/land use

Including targeted projects Including targeted projects SMOLE and LIMASSMOLE and LIMAS

30

Other areas of cooperationOther areas of cooperation

New areas of cooperation New areas of cooperation NGO support and the LCF NGO support and the LCF Regional programmes Regional programmes Economic cooperationEconomic cooperation

31

Influence of policy until 2007 Influence of policy until 2007

Match between policy, Match between policy, programme and contextprogramme and context

ConsequencesConsequences

32

Influence of policy 2008-2011 Influence of policy 2008-2011

Changes in contextChanges in context Ministerial involvementMinisterial involvement The 2008 Country Assistance The 2008 Country Assistance

PlanPlan Country consultations and the Country consultations and the

2009-2011 Development 2009-2011 Development Cooperation PlanCooperation Plan

33

Influence of FAV and PDInfluence of FAV and PD

Different Different definitions/interpretations of FAVdefinitions/interpretations of FAV

FAV influence strong after 2007FAV influence strong after 2007 Finnish PD performance above Finnish PD performance above

averageaverage Difference between PD rhetoric Difference between PD rhetoric

and practiceand practice

34

Evaluation criteria and Evaluation criteria and questionsquestions

Variations in relevanceVariations in relevance Variations in sustainabilityVariations in sustainability Cross-cutting themesCross-cutting themes Coordination Coordination CompatibilityCompatibility

35

Strengths & WeaknessesStrengths & Weaknesses Strength: Reservoir of trust Strength: Reservoir of trust Strength: Influence amongst Strength: Influence amongst

donors donors Weakness: fragmentation & Weakness: fragmentation &

incoherenceincoherence Best practice: NAFORMABest practice: NAFORMA

36

Worst practices Worst practices

Absence of comprehensive Absence of comprehensive reportingreporting

Quality of dialogue in Mtwara/LindiQuality of dialogue in Mtwara/Lindi Decision-making process on SMOLE Decision-making process on SMOLE Decision-making on SDIDecision-making on SDI DRPS strategic value and DRPS strategic value and

compatibilitycompatibility

37

ConclusionsConclusions Consequences of ambiguous Consequences of ambiguous

policy and application of slippery policy and application of slippery conceptsconcepts

Need for consolidationNeed for consolidation Timeframe for consolidationTimeframe for consolidation

38

RecommendationsRecommendations Agreeing principles for Agreeing principles for

consolidationconsolidation CCTs and logical frameworks CCTs and logical frameworks Guidance from HelsinkiGuidance from Helsinki Reconsideration of modalities Reconsideration of modalities

and sectorsand sectors

39

Overview of findingsOverview of findingsJulian CALDECOTTJulian CALDECOTT

InteractionsInteractions with Governmentwith GovernmentNepal Nicaragua Tanzania

In the field during armed conflict, 1996-2006 (WASH, environment, education).

Development of SWAps, debt relief & GBS, 2002-2006 (2004 policy influence: strong).

Dialogue mainly through multi-donor channels, and around SWAps and GBS.

Dialogue and programming suspended, 2005-2006 (2004 policy influence: weak).

Friction with government from 2007, governance SWAp and GBS closed (2007 policy influence: weak).

National capacity constraints lead donors back to projects, but weaken dialogue with government.

Excellent relations since 2007, focused on government priorities (2007 policy influence: strong).

Rural development and health/gender SWAps continue as effective, pro-poor and constructive.

From 2007 Finnish bilateral activities multiply, contributing to good high-level relations.

40

Interactions with other DonorsInteractions with other Donors

All 3 countries:All 3 countries: Low-cost access to multi-donor Low-cost access to multi-donor

forums.forums. Embassy staff influence forums. Embassy staff influence forums. Challenge from new donors, Challenge from new donors,

uncoordinated government-to-uncoordinated government-to-government support and investment.government support and investment.

41

Interactions with civil societyInteractions with civil society

MFA support for NGOs & embassy MFA support for NGOs & embassy (LCF) support for local CSOs, but:(LCF) support for local CSOs, but:LCF focus on fewer and larger LCF focus on fewer and larger CSOs in Nepal and Tanzania.CSOs in Nepal and Tanzania.Closure of LCF in 2008 in Closure of LCF in 2008 in Nicaragua, funds diverted to other Nicaragua, funds diverted to other modalities that in effect weakened modalities that in effect weakened NGOs.NGOs.

42

Implementing the Paris DeclarationImplementing the Paris DeclarationNepal Nicaragua Tanzania

Donors resist commitment to programmatic approach due to fears over corruption.

GBS and all SWAps were in line with the Declaration.

GBS and all SWAps were in line with the Declaration.

Education sector SWAp is a success.

Problems after 2007 because of conflict over government efforts to own decentralisation, human rights, and civil society interventions.

Leaves education SWAp, forestry SWAp fails, weak consensus around good governance and anti-corruption measures linked to GBS.

Increasing attention to investments with shared responsibility (e.g. WASH and forestry sectors).

Sectoral budget support largely replaced GBS and the share paid through government increased after 2007.

Proliferation of project modalities by donors including Finland.

43

Promoting Cross-cutting ThemesPromoting Cross-cutting ThemesNepal Nicaragua Tanzania

Peace-building, governance and human rights mainstreamed in the country programme.

Gender and health-related CCTs fully integrated within the health/gender SWAp.

CCTs in multilateral and bilateral dialogue, in bilateral programmes, and key objectives of LCF.

GESI embedded in education SWAp and in environmental and WASH interventions.

Since 2007, mainstreamed support for democratic accountability, rule of law, human rights & gender equity.

Results on governance and anti-corruption mixed (lack of interest by government); other CCTs largely neglected.

LCF used to support CSOs that work on caste exclusion and human rights among women and children.

LCF promoted rights of vulnerable groups; last grants supported CSOs on cultural values and special needs of youth and women.

LCF is where the CCTs are best taken into account, and where paying attention to them contributes most to success.

44

Putting policy into practicePutting policy into practiceAll 3 countries :All 3 countries : Joint programming by embassy and MFA.Joint programming by embassy and MFA. Little influence of 2008 CAP.Little influence of 2008 CAP. Diverse unplanned influences on choices.Diverse unplanned influences on choices. Embassy staff adapt & use opportunities.Embassy staff adapt & use opportunities. Weak guidance on CCTs, Paris Weak guidance on CCTs, Paris

Declaration and climate change.Declaration and climate change. FAV: special interests, technical fields FAV: special interests, technical fields

and ways of working.and ways of working.

45

StrengthsStrengthsNepal Nicaragua Tanzania

Finnish role in donor coordination

Finnish role in donor coordination

Finnish role in donor coordination

Small financial contributions to obtain maximum leverage and influence in multiple forums.

The health/gender and rural development SWAps.

Finnish influence in the donor community with embassy personnel as chairs or co-chairs of the Development Partner Group and its working groups.

GESI strategies in the WASH and environment sector projects.

Keeping ways to explore new ideas through innovative and strategic projects.

Partnership with FAO has great potential in DeNRM, forestry and climate change.

46

WeaknessesWeaknessesNepal Nicaragua Tanzania

Ambition on strategic issues

Ambition on strategic issues

Ambition on strategic issues

Opinions differ, but no failure or worst practice was found.

Pause in consultations before and after 2009 session, when continual dialogue was needed.

Disordered programming process after 2007.

Refocusing of the governance programme towards a rights agenda, while creating aid dependency and political isolation among NGOs.

Weak quality of dialogue and decision-making around several interventions.

Willingness for the embassy to be considered hostile to government.

Opaque means by which certain funding decisions were made.

47

Key influencesKey influences Opportunities seen by embassies (all).Opportunities seen by embassies (all). Government needs (Nepal).Government needs (Nepal). Paris Declaration (Tanzania).Paris Declaration (Tanzania). Policies (2007 Nepal, 2004 Nicaragua).Policies (2007 Nepal, 2004 Nicaragua). Government plans (Tanzania).Government plans (Tanzania). Links with particular locations (all).Links with particular locations (all). Changed circumstances (Nepal, Changed circumstances (Nepal,

Nicaragua).Nicaragua). Personal decisions (MFA Tanzania, Personal decisions (MFA Tanzania,

embassy Nicaragua, government embassy Nicaragua, government Tanzania).Tanzania).

48

Lessons from strengthsLessons from strengths

Make small contributions to multi-donor Make small contributions to multi-donor enterprises and earn maximum influence.enterprises and earn maximum influence.

Use well-formulated strategies to Use well-formulated strategies to promote participation, inclusion and promote participation, inclusion and equity.equity.

Participate with government and donors Participate with government and donors in serious, long-term sectoral in serious, long-term sectoral programmes.programmes.

Preserve a complementary role for Preserve a complementary role for innovative and strategic experiments and innovative and strategic experiments and partnerships.partnerships.

49

Lessons from weaknessesLessons from weaknesses Actively build agreement with government Actively build agreement with government

that strategic issues should be addressed.that strategic issues should be addressed. Maintain active dialogue with government Maintain active dialogue with government

especially at difficult times.especially at difficult times. Understand consequences of supporting Understand consequences of supporting

CSOs that compete with government.CSOs that compete with government. Maintain at all times the role of critical Maintain at all times the role of critical

friend, partner and counsellor of friend, partner and counsellor of government.government.

Base programming on rigorous, Base programming on rigorous, transparent analysis and collective transparent analysis and collective decision making.decision making.

50

Thank you!Thank you!

51