November 2010 - OECD · 2016-03-29 · November 2010. 2 Preface This ... UNFPA United Nations...
Transcript of November 2010 - OECD · 2016-03-29 · November 2010. 2 Preface This ... UNFPA United Nations...
1
JOINT EVALUATION OF THE PARIS DECLARATION PHASE 2
COOK ISLANDS COUNTRY EVALUATION
VOLUME of APPENDICES BOOKLET
November 2010
2
Preface
This Volume of Appendices contains the collation of the terms of reference, tools used for the study, research information and data gathered as a result of the study. The volume of Appendices also includes the survey summary findings matrix assessments, tracer studies, list of participants, and summaries of interview findings. All of this information contained in the Volume of Appendices, provides the evidence that is used in the Paris Declaration Evaluation Phase II Study for the Cook Islands Report.
3
Table of Contents Preface 2
Acronyms and Abbreviations 4
Evaluation Tools
Appendix 1: Terms of Reference 6
Appendix 2: Pacific Aid Effectiveness Principles 23
Appendix 3: Matrix Evaluation Questions Template 28
Appendix 4: Interview template 61
Appendix 5: Survey template 68
Appendix 6: List of participants 75
Evaluation Assessments/result data
Appendix 7: National Rated matrix 80
Appendix 8: Infrastructure rated matrix 116
Appendix 9: Health rated matrix 125
Appendix 10: Summary of interviews 134
Appendix 11: Summary of survey results 155
Appendix 12: Summary of six project profiles 176
Appendix 12a: Health ‐HSV 177
Appendix 12b: Health ‐ Pandemic Plan 178
Appendix 12c: Health ‐ GF – TB 179
Appendix 12d: Infrastructure ‐ Sports Stadium 180
Appendix 12e: Infrastructure – CRRP 181
Appendix 12f: Infrastructure – PPTA 182
4
Acronyms and Abbreviations
AAA Accra Agenda for Action
ADB Asian Development Bank
AMD Aid Management Division
AusAID Australian Agency for International Development
CISO Cook Islands Statistic Office
CIGFPPM Cook Islands Government Financial policies and procedures manual.
CIGovt Cook Islands Government
CIMDGs Cook Islands Millennium Development Goals
CITV Cook Islands Television
CS Civil Society
CSO Civil Society Organisation
DAC Development Assistance Committee
DPs Development Partners
EU European Union
FAO Food and Agriculture Organisation
FAP Forward Aid Programme
GAVI Global Alliance for Vaccines and Immunisation
GDP Gross Domestic Product
GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria
GNI Gross National Income
GPs Government Partners
HIES Household Expenditure Survey
HOM Head of Ministry
INTAFF Ministry of Internal Affairs
LDCs Least Developed Countries
MDGs Millennium Development Goals
MFAI Ministry of Foreign Affairs and Immigration
MFEM Ministry of Finance and Economic Management
MFDR Managing for Development Results
MTBF Medium Term Budget Framework
MOE Ministry of Education
5
MOH Ministry of Health
MOIP Ministry of Infrastructure and Planning
NGO Non‐Government Organization
NRG National Reference Group
NSDC National Sustainable Development Commission
NSDP National Sustainable Development Plan
NZAID New Zealand Agency for International Development
NZAP New Zealand Aid Programme
ODA Official Development Assistance
OECD Organization for Economic Cooperation and Development
OPM Office of the Prime Minister
OPSC Office of the Public Service Commissioner
PBAs Programme‐Based Approaches
PD Paris Declaration
PEFA Public Expenditure Framework Assessment
PERCA Public Expenditure Review Committee Act
PFTAC Pacific Financial Technical Assistance Centre
PIFS Pacific Islands Forum Secretariat
SPC Secretariat for Pacific Countries
SWAp Sector‐Wide Approach
TC Technical Cooperation
UNDP United Nations Development Programme
UNFPA United Nations Population Fund
UNDAF United Nations Development Assistance Framework
WB World Bank
WHO World Health Organization
6
Appendix 1: Terms of Reference
Joint Evaluation of the Paris Declaration, Phase 2
Generic Terms of Reference (ToR) for the Cook Islands Country Evaluation
Guidance to Readers Under the umbrella of the overall Evaluation Framework for Phase 2, this paper sets out the key common features and issues to be covered in the Country evaluations, including agreed purposes and objectives, design, management and governance arrangements, support, staffing, quality assurance, and timelines. The document includes a draft Common Evaluation Matrix for country evaluations, and a proposed draft outline for the eventual evaluation reports, aligned with the matrix. The latter should encourage a clear understanding from the outset on the intended end products. The Evaluation synthesis in turn will be aligned with this matrix, integrating the findings of the country evaluations, donor/agency headquarters (HQ) studies, and other agreed sources.
It should be stressed that each participating country - while contributing fully to answering the minimum common evaluation questions that will be agreed upon for all - may also wish to supplement this coverage with particular evaluation issues or questions of special interest or relevance to the country, within the resources available for the evaluation.
The Cook Islands have not specifically added supplementary questions but have decided to add Infrastructure as a focal sector in addition to Health. The Draft Evaluation Matrix will also be fine tuned to take into account the local conditions and potential data gaps that may be encountered locally.
This Generic ToR will be used by the Country Coordinators and Reference Groups to guide them in their responsibilities for launching, managing and ensuring the success of the evaluations. They will also provide guidance for the professional Teams that will be recruited to carry out the work, backed up by the overall Framework and the continuing engagement and support of the Core Evaluation Team. It was strongly confirmed in each of the regional workshops from 27 October to 20 November, 2009 that, in order to meet both sets of needs, the Generic ToR for this challenging evaluation needs to be clear, straightforward, rigorous, manageable and comparable across the two dozen countries where the evaluations will be carried out.
The substantial revisions from a first draft version, circulated on 20 October, 2009, reflect a systematic consolidation of the inputs of the many participants in the regional workshops, all the advance comments on the first draft by other members of the International Reference Group (IRG), and final refinements agreed upon at the meeting of the Group on 1 December, 2009. The Generic ToR goes as far as possible at this stage to set out the main lines of the approach. This will enable National Coordinators and Reference Groups to launch the national evaluation exercises, recruit evaluation teams, and complete the evaluations in time to inform the 4th High Level Forum in Seoul. As confirmed at the IRG meeting (with a number of specific suggestions), revised evaluation matrices with final, detailed methodologies and standard methods will be finalized through regional workshops with Country Teams, National Coordinators and the Core Evaluation Team as soon as the individual country evaluation teams are in place (by March 2010). The preparatory proposals will also be circulated to the full IRG for its review, as will the April 2010 Inception Report for the Evaluation which will contain the final version.
7
1. Background and Rationale: the overall Phase 2 Evaluation
1. The Paris Declaration expresses a broad international consensus developed in the 15 years up to 2005, stipulating that new partnership relationships and ways of working between developed countries and partner countries are essential if development results are to be assured, aid well spent and aid volumes maintained.
2. The Paris Declaration1 was endorsed at the 2nd High Level Forum held in Paris in 2005 by 52 donors/agencies and partner countries and 30 other actors in the development cooperation field (United Nations and other multilateral agencies and non-governmental organizations). The Declaration consists of 56 “Partnership Commitments”, and aims to strengthen “partnerships” between donor countries and countries receiving aid in order to make aid more effective and to maximize development results.
3. The requirement for independent evaluation was built into the original Declaration and re-confirmed in the Accra Agenda for Action in 2008.2 The first phase of the Evaluation3 ran from March 2007 to September 2008 and aimed at providing information on the “HOWs and WHYs” of the early implementation process of the Paris Declaration, looking at inputs and early outputs. It was designed and used to deliver practical lessons and help take stock of implementation performance at the 3rd High Level Forum on Aid Effectiveness held in Accra, Ghana in September 2008. 4. The second phase of the Evaluation will run from the 3rd High Level Forum in 2008 up to the 4th High Level Forum in Korea in 2011. This phase will emphasize outcomes and results and offer answers to the critical policy question of whether the intended long-term effects of the Paris Declaration are being achieved or advanced. The evaluation is expected to analyze results in context, taking into account preconditions or enabling conditions that may lead to or inhibit positive development results supported by aid.
2. Country Evaluations: purpose, objectives, uses and approach 5. Purpose: The country evaluations that will be the Evaluation’s primary focus will be the main vehicles for answering the core evaluation questions on the effects of the Paris Declaration on aid effectiveness and development results, including poverty reduction. These country evaluations will assess the effectiveness in this regard of donors/agencies in the country, alongside that of the country stakeholders, and of the partnerships between them.
6. Objectives: The aim of the evaluation is to document, analyze and assess the relevance and effectiveness of the Paris Declaration in the country and its contribution to aid effectiveness and ultimately to development results, including poverty reduction. 1 The full Declaration can be found at: http://www.oecd.org/dataoecd/11/41/34428351.pdf and the Accra Agenda for Action at http://www.oecd.org/dataoecd/58/16/41202012.pdf 2 The Evaluations complement the monitoring of the implementation of the Paris Declaration, undertaken through the Cluster D of the Organisation for Economic Co-operation and Development (OECD), Development Assistance Committee (DAC), Working Party on Aid Effectiveness “Assessing Progress on Implementing the Paris Declaration and the Accra Agenda for Action.” 3 Wood, B; D. Kabell; F. Sagasti; N. Muwanga; Synthesis Report on the First Phase of the Evaluation of the Implementation of the Paris Declaration, Copenhagen, July 2008. The report can be found at: http://www.diis.dk/graphics/Subweb/paris_evaluation_web/index.htm
8
7. Specific objectives include:
• To document the results achieved in the country through implementing the Paris
Declaration.
• To enable the partner countries and donors/agencies active in the country to clarify, improve and strengthen policies and practice consistent with the Paris Declaration in pursuit of aid effectiveness and development effectiveness.4
• To highlight barriers and difficulties that may have limited the effectiveness of the Paris Declaration and its effects and impacts – and ways that these barriers and difficulties may be overcome.
• To enable sharing and exchange of experience among stakeholders, countries and partnerships so as to facilitate reflection, lesson-learning and policy improvement.
8. The Accra Agenda for Action further specified some of the Paris Declaration’s commitments with the aim in particular of strengthening country ownership; building more inclusive partnerships; and sharpening the focus on development results. The Phase 2 evaluation will therefore pay particular attention to assessing implementation of these Accra commitments, which address the current concerns of many stakeholders. These Accra commitments are reflected in these ToR.
9. Audiences, Stakeholders and Usefulness of the Evaluation: The focus of Phase 2 is on a results oriented evaluation, with the synthesis and component evaluation reports to be presented to the 4th High Level Forum in 2011. It is equally intended that the evaluation process will spur interest and improvement efforts in the participating countries and agencies.
10. Key constituencies include the executive and legislative branches of government in the country, those of its bilateral development partners, and governing authorities and senior managements of development agencies. Also crucial are those tasked with implementing the Paris Declaration: government, donor, civil society and private sector stakeholders in the partner countries as well as donor agencies. The findings are also expected to be of direct interest to many citizens of both the host countries and of countries providing international development assistance.
11. The goal of ensuring wide dissemination and use of the evaluation by its intended audiences should influence the process and products at every stage of the evaluation, by:
a. Keeping the central questions and key audiences constantly in sight;
b. Using straightforward language: minimizing acronyms, jargon and unnecessary technical language in all products;
c. Open internal communications – as in the planned knowledge-sharing system within and among teams;
d. Trilingual operation: specific work to ensure timely translation of key documents and balanced literature sources in English, French and Spanish;
e. Building in the time required for peer exchanges, edits, strong summaries; 4 In a number of participating countries, clear links are already being forged between this evaluation and other, related monitoring and evaluation activities in order to maximise the synergies, guard against duplicative work, and strengthen the usefulness of the evaluation in the country.
9
f. Critically, meeting the required deadlines for progress steps and the submission of draft and final reports and dissemination summaries.
12. National communications plans should be directly linked to key points in the national and international dialogue on aid effectiveness and Millennium Development Goal (MDG) trends over the coming two years to build policy engagement with the study and ensure its timely contribution to the debates.
13. Approach for Country Evaluations: An approach for the overall Evaluation has been set out in the “Evaluation Framework and Work-plan” for Phase 2. It takes account of the distinctive methodological challenges of evaluating the Paris Declaration. The Phase 2 evaluation will focus on effects at the level of partner countries and their partnerships, i.e. the joint arrangements between donors and the recipients of aid that have been put in place to support the implementation of the Declaration.
14. As the main foundation for the overall evaluation, well-grounded comparisons between experiences (within and across countries) will be important to test claims for the effects of the Paris Declaration.
15. There will be country evaluation teams in each participating partner country, responsible for undertaking independent evaluations of aid effectiveness and development results. These teams will address both:
• Implementation or “process” – assessing changes of behaviour of countries and donors
around aid and development and within the aid partnership itself. A strong focus on the context for implementation in each country (including one major block of evaluation questions) is designed to ensure that the evaluation remains realistic and relevant in individual country situations; and
• Results or outcomes in terms of aid effectiveness and development results, with rather
precise minimum common “core” questions, scope and methodologies for all country evaluations, to allow meaningful aggregation and synthesis. This will not limit the ability of country evaluations to supplement the Common Evaluation Template/Matrix with questions of special relevance or interest to their particular situations.
16. Whilst most evaluative activity for the overall Evaluation will be undertaken by country teams, their evidence will be complemented by a number of headquarters-level donor/agency studies, together with the eleven conducted in Phase 1; and a small number of “supplementary studies” where essential to provide adequate coverage of important issues. Specific opportunities for complementary coverage will be sought out and together these elements are intended to ensure adequate depth and breadth of the evaluation. The building blocks for the Phase 2 Evaluation (and the central role of the country evaluations) are illustrated in the Figure below.
10
Building blocks of the Paris Declaration Evaluation Synthesis
3. Evaluation Methodology: evaluation questions and methods 17. Evaluation Questions: The evaluation draws on a good deal of preparatory work which took into account the many complex factors and relationships at work in the implementation of the Paris Declaration and the special challenges involved for evaluation methodology5.
18. The Evaluation Matrix for Country Evaluations set out in Appendix B will be the principal instrument for guiding and conducting these evaluations and the preparation of their products. It is constructed around a set of core evaluation questions and sub-questions which will serve as the minimum common structure for all individual country evaluations and for the final comparative synthesis report (which will also integrate the results of Donor HQ studies, the Phase 1 evaluation, and other inputs).
19. The evaluation will: a) evaluate to what extent the Paris Declaration has been implemented, and b) insofar as it has been implemented, evaluate what the results have been in terms of aid effectiveness and development. The core questions (as refined through the regional workshops and inputs from the International Reference Group members) are set out
5 This work, summarized in the “Approach Paper for the Phase 2 Evaluation” (May 2009) included a major workshop of the International Reference Group in Auckland, New Zealand in February 2009 and a commissioned study on “The Paris Declaration, Aid Effectiveness and Development Effectiveness” in November, 2008 (the “Linkages Study”).
11
below and then in the Matrix in Appendix B, where they are backed with the sub-questions, together with indications of the common types, indicators, and sources of evidence, to be used, as well as initial directions on common techniques and methods. Once the core questions and sub-questions are confirmed (through approval of the Generic ToR, December 2009), additional guidance will be developed to flesh out the Matrix, particularly the methods and tools in Column 4, with a more precise identification of the analytical methods for each study element. This will ensure a clear understanding of all the steps involved to support standard approaches, e.g. on data handling and analytical steps for each stage.
20. The “logic chain” of the questions is illustrated in three different diagrams in the Evaluation Framework, and it should be noted that the order and content of the three main evaluation questions, and the framework for conclusions, successively emphasize the accepted guiding evaluation criteria of relevance, efficiency, effectiveness and sustainability.
12
21. Special Challenges: In addressing these core evaluation questions it is clear that the challenges of attributing results to a set of commitments like the Paris Declaration are especially complex. One vital starting point is to recognize that the 2005 Declaration itself brought together a variety of reform efforts and initiatives that had been underway in different settings for some years before. Thus each evaluation should explicitly include the assessment of these “upstream” or precursor steps as an integral part of its scope.
22. Paris Declaration implementation is a multidimensional, multi-level process, affected by many factors, which can change its direction, emphasis, and pace at different times and in response to different influences. One way of making these factors more explicit and prominent throughout the evaluation is the emphasis placed through the first question on a far more in-
The Core Questions 1. “What are the important factors that have affected the relevance and implementation of
the Paris Declaration and its potential effects on aid effectiveness and development results?” (The Paris Declaration in context)
2. “To what extent and how has the implementation of the Paris Declaration led to an improvement in the efficiency of aid delivery, the management and use of aid and better partnerships?” (Process and intermediate outcomes)
3. “Has the implementation of the Paris Declaration strengthened the contribution of aid to sustainable development results? How?” (Development outcomes)
The Framework for Conclusions i. What has been the relevance of the Paris Declaration and the ways it has been
implemented to the challenges of aid effectiveness?
ii. To what extent has each of the five principles of the Paris Declaration been observed and implemented, and the Accra Agenda priorities reflected? Why? Have there been conflicts or trade-offs between them?
iii. What has the Paris Declaration achieved for aid effectiveness and development results? How significant are these contributions? How sustainable? Is there evidence of better ways to make aid more effective and contribute more to development results?
iv. What effects has the implementation of the Declaration had on the respective burdens of aid management falling on partner countries and donors, relative to the changing volume and quality of aid and of the aid relationship itself? Are these effects likely to be transitional or long term?
v. What has been the added value of the Paris Declaration-style development cooperation compared with the pre-Paris Declaration situation, and seen alongside other drivers of development in the country, other sources of development finance and development cooperation partners beyond those so far endorsing the Declaration?
vi. What are the key messages for a) national stakeholders, and b) donor countries and agencies?
vii. What are the key implications for aid effectiveness in the future taking account of new challenges and opportunities (e.g. climate change) and new actors and relationships?
13
depth and dynamic analysis than would be usual of the context for the implementation of the Paris Declaration and Accra Agenda in each country where an evaluation is undertaken.
23. The main evaluation questions will be operationalised through a set of sub-questions including descriptive, analytical, normative and evaluative questions. These will be supported wherever possible by common specifications and suggestions of:
i. the types of evidence and, where applicable, indicators to be used; ii. the anticipated availability and (probable) reliability of data sources; and iii. proposed sources, methods and techniques for data collection, analysis, triangulation
and validation.
24. Key elements: A ready guidance for the Country Level Evaluations, the key elements of the overall evaluation methodology set out in the Evaluation Framework can be summarized as follows:
a. A “theory based” approach – which recognizes that outcomes/results from the Paris
Declaration implementation may not be fully visible by the time of the Evaluation – so it focuses instead on identifying the chains, directions, causes and trends of causality and the linkages involved (see points below);
b. A “theory of change” which anticipates and explores complexity rather than expecting
to apply simple or one-dimensional models of attribution;
c. Seeking out and exploring the causal mechanisms and key actors driving or inhibiting change, their roles, inter-relations, and relative weightings in influencing outcomes (especially through Core question 1);
d. Focused on causality in context: searching for common trends rather than (necessarily)
generalized truths, but recognizing that the shape, nature and pace of change is heavily determined by locally specific factors and influences;
e. Focused on comparability, ensuring robust analysis at aggregate level (through e.g. the development of common standards for analytical frameworks and data collection) while giving full weight to contextual factors;
f. A summative and formative model – allowing judgments around outcomes and results
whilst supporting forward-looking policy development and improvement.
25. Specific methods for pursuing the evaluations include:
a. Literature and documentation review
b. The analysis of the most relevant existing statistical data such as human development and poverty indicators, Poverty Reduction Strategy Paper (PRSP) reports, sector reports, MDG reports etc.;
c. Syntheses and meta-analyses of existing evidence (i.e. secondary sources such as policy, evaluations and research). Common specified parameters will be proposed and agreed for data identification, inclusion and structured assessment;
14
d. Structured surveys and questionnaires (key informant groups) deepened by semi-structured interviews and focus groups (key stakeholders including government (different branches and levels) donor agencies, civil society and the private sector). Any possibilities for drawing on participative approaches will be pursued;
e. To help ground the evaluations, a common template for analysis by all or almost all country evaluations of one important “tracer sector” (health) and for comparable analysis of the other sectors of priority chosen within each country. Following broad agreement in the regional workshop process to a special focus on two sectors per evaluation, an agreed template and guidance will be developed for the identification, design and implementation of these analyses;
f. Backward tracking, retrospective or inductive studies of sector, site or theme; using methodologies such as the analysis of time-series data; statistical trends; synthesis studies to assess “distance travelled” etc.;
g. Forward looking analysis; which anticipates development results that are in formation but have not become fully evident, and backward-tracking studies as a basis for seeking plausible links in the causal chain - from Paris Declaration-style aid inputs to development results - to assess and predict the likely direction of further travel.
26. Rigour and Comparability: In addition to the use of the agreed minimum common questions, sub-questions and methods, the robustness of the approach and methodology for the evaluation and its results will be further ensured by:
a. A consistent stance in the evaluation that does not assume attribution of results to the
Paris Declaration, but rather takes a critical approach and examines alternative explanations;
b. A set of support mechanisms available to individual evaluation coordinators, reference groups and teams, particularly from the Core Evaluation Team, both directly and through research resources and interactive internet facilities [see Section “Support Arrangements for Country Evaluations” for detail];
c. Verification of evidence emerging through ongoing triangulation between the multiple data sources and methods employed;
d. Step-by-step validation of evaluation results by national core teams (with peer review among them encouraged) by the core team, country reference groups, the Evaluation Secretariat and Management Group, possibly high level external reviewers, and the International Reference Group;
e. Quality assurance processes that are built in to each component evaluation (as well as the preparation of the final synthesis report) – all are required to meet the Development Assistance Committee (DAC) Evaluation Quality Standards, United Nations Evaluation Group (UNEG) Standards, or the comparable national or regional standards where these have been adopted;
f. Selection and contracting of appropriately-skilled evaluation teams by established procedures, with protection for the independence and professional integrity of their work;
g. Forming country teams using national expertise to the maximum extent possible but also including regional and international experts where appropriate, assuring that all are free of potential conflicts of interest;
h. Prioritizing the use of country systems to capitalize on existing data/literature including academia, universities, and civil society;
15
i. Wherever possible, seeking the engagement and coverage of providers of development resources not yet formally endorsing the Paris Declaration in the capacity of donors; and
j. Using a set of agreed working definitions for key terms6 and a common style guide to avoid confusion and inconsistent treatment.
4. Management of the Evaluation: responsibilities and accountabilities 27. The points below draw and build on the ‘Guidance for Management of Country level Evaluations’ Note issued by the Secretariat in September 2009. More detail on the international structure, relationships and governance in the overall Evaluation is provided in the “Evaluation Framework and Work-plan” for Phase 2.
28. Management considerations: The key management considerations for a Country Evaluation are:
• In-country management arrangements that are operational • Clarity on roles, responsibilities, quality assurance and accountabilities • Communication with stakeholders • Progress reporting
29. In-country management arrangements: The National Evaluation Coordinator, appointed by the Government, is responsible for managing all aspects of the Country Evaluation process including, most importantly:
a. Selecting, setting up and then scheduling and convening meetings of the in-country
National Reference/ Advisory Group, expected to include major stakeholders from governments, donors, civil society and possibly academia;
b. Developing the final ToR for the Country Evaluation in consultation with the National
Reference/ Advisory Group; incorporating the common evaluation matrix for Country Evaluations and (if required) a module with country-specific evaluation questions;
c. The recruitment and contracting of the consultants for the Country Evaluation (with
selection where possible by the National Reference/ Advisory Group);
d. At least bi-monthly reporting on the progress of the evaluation in line with a
manageable agreed common format;
e. Quality control; assuring that the evaluation is of acceptable quality in reference to
identified relevant national, regional and/or international (DAC) standards and drawing on the pro-active and responsive services of the Core Evaluation Team and the Evaluation Secretariat.
6 A Glossary has been prepared as part of the guidance to the Phase 2 Evaluation.
16
30. This management role will require significant inputs of ‘dedicated management time’ over the whole evaluation process, with concentrated effort anticipated during the start-up period, inception, first draft report and final reporting milestones.
31. The National Reference/ Advisory Group will normally be responsible for the following important functions:
a) Approving the design of the pertinent evaluation that comprises a common set of
evaluation questions applicable to all country evaluations and where desired a module with supplementary, country-specific evaluation questions;
b) Deciding on selection criteria for the country teams;
c) Selecting the members of country evaluation teams, consistent with the selection criteria and national competitive procurement or tender rules;
d) Serving as a resource and to provide advice and feedback to the National Coordinator and Team;
e) Helping to ensure the independence, integrity and quality of the evaluation;
f) Reviewing and commenting on (but not approving) the draft products of the respective country evaluation.
32. National Reference/ Advisory Groups should also have important roles to play in accessing information, exerting quality control, linking to government and engaging civil society, facilitating the necessary wider consultation, and encouraging the use and usefulness of the evaluations findings.
33. These roles will require a Group with sufficient representation from key stakeholders, good credibility and access, together with the necessary measure of independence. The tasks will imply the need for a series of dedicated inputs of time from the individual members of the National Reference/ Advisory Group.
34. Management in-country will be supported by self-monitoring of progress with the evaluation, and reflection at periodic National Reference/ Advisory Group meetings on the extent to which the Country evaluation remains ‘on track’ and actions to be taken if and when ‘gaps’ appear.
35. Clarity on roles, responsibilities and accountabilities. The success of this collaborative exercise in-country will be heavily influenced by initial clarity and ongoing discipline on who is expected to deliver on what and by when, and who is accountable. Key accountabilities in the process are:
i. Competent independent Country Evaluation Team selected, contracted and
resourced by latest 31st March 2010: The National Evaluation Coordinator is accountable for this milestone being reached with the support of the National Reference/ Advisory Group, and for the independence of the evaluation being maintained throughout the process.
17
ii. Country Evaluation Report delivered in-country on time: The Team Leader [and/or the contracted firm or institution] of the Country Evaluation Team is accountable for the organization and co-ordination of the work of the evaluation team (and through this ensuring the quality and relevance of team member contributions) and assuring the delivery of emerging findings and a comprehensive final report which meets evaluation standards, within the contracted timeframe/ specifications.
iii. Country Evaluation Report of an acceptable quality submitted to the Core
Evaluation Team for use in preparing the synthesis report and publishing: The National Evaluation Coordinator, through successive processes of quality control, is accountable for the delivery of a report of acceptable quality for the Synthesis stage.
36. Communication with stakeholders: Each Country Evaluation is expected to develop and implement a ‘Communication Plan’ through which stakeholders for the evaluation within the country will be kept informed and engaged. A variety of channels and activities should be used and opportunities maximized to link to key points in national strategic and decision-making cycles (already planned in several countries). Links should also be forged with key milestones in the international dialogue on aid effectiveness and MDG trends over the coming two years to build policy engagement with the study and ensure its timely contribution to the debates.
37. Ensuring this communication and engagement takes place and in a form that fosters stakeholder interest, civil society involvement, and ‘buy-in’ to the evaluation process would be a responsibility of the National Reference/ Advisory Group.
38. Progress reporting: The National Evaluation Coordinator, in his/her role as in-country focal point for the Phase 2 Evaluation will provide the Secretariat with bi-monthly updates (copied to the Core Evaluation Team) – starting end of December 2009 – on the status of the Country Evaluation process. This will use a simple proforma to be developed by the Evaluation Secretariat in consultation with the Core Evaluation Team which will facilitate the updating by the Core Team of the ‘master sheet’ on progress across the 20+ country Evaluations.
39. Country Evaluation Timeline: The overall Evaluation Work-plan and Schedule below incorporates the sequence of key milestones for the Country evaluations, alongside other elements and processes.
18
Evaluation of the Paris Declaration Phase 2
Critical Milestones
Period/Date In country and donor/agency HQ Core Evaluation Team
1 Dec 2009 IRG endorsement of Evaluation Framework and
Generic Terms of Reference
2010
Dec 2009/ Jan 2010
Establish National Reference Groups and approve TORs for Country Evaluations/Donor HQ Studies Core Evaluation Team
support to National Evaluation Coordinators as required Dec 2009/
Mar2010 Select and contract evaluation Teams
Feb - Mar Core Team mission to coordinate with advanced Cambodian evaluation and early South Pacific workshop for Team Leaders and National Coordinators with Core Team/EMG
26 March
Core Evaluation Team circulates draft operational matrix including methodology paper/package to workshop participants and to IRG/EMG for comments by 9 April. (Participants in the sub-regional workshops will, of course have the opportunity to comment and discuss this at the workshops).
April Sub-regional/national workshops for Team Leaders and National Coordinators with Core Team/EMG
30 April
Core Team submits Inception Report including revised operational Matrix including methodology paper/package to IRG for comments by 10 May (noting that input from the delayed SEA workshop will still need to be taken into account.)
11 - 13 May Core Team and EMG meet to finalise Inception Report and operational Matrix including methodology paper/package
31 May Country and Donor/Agency HQ Teams submit inception reports Core Evaluation Team
support to National Evaluation Coordinators as required
April – Oct
Conducting Country and donor/agency headquarters-level evaluations
15 Oct Submission of first draft report including summary of findings by each country team and donor/agency headquarters-level team to EMG and CET
Core team prepares consolidated emerging findings by 15 Nov
Oct – Dec Consultation, validation and finalisation of reports in countries and donor/agency HQ
7 – 9 Dec
Meeting/workshop of country and donor/agency headquarters study team leaders, Core Team and International Reference Group to discuss emerging
19
5. Support Arrangements for Country Evaluations
40. The Core Evaluation Team: The Core Team contributes to the Phase 2 evaluation across all components at all stages: at planning and set-up; on an ongoing basis to ensure consistency and solve problems that may arise; and in the final stages when it will be expected to bring together all evaluation findings in a free-standing Synthesis Report. The Core Team reports and is responsible to the Evaluation Management Group through the Evaluation Secretariat.
41. Services to Country Evaluations. The Core Team has been in place since September 2009. With a view to ensuring the quality and integrity of the Country Evaluations within the overall Phase 2 Evaluation it is charged with providing the following set of support services to Country Evaluation processes:
• After intensive regional consultations, design for the approval of the Evaluation
Management Group and the International Reference Group a “Generic Terms of Reference” for Country Evaluations that will guide data gathering and fieldwork in a way that will ensure quality and enable comparison and the synthesis of findings.
• Provide professional advice on request to the National Evaluation Coordinator and
members of the National Reference/ Advisory Group on the basis for selection, contracting and briefing of Country Evaluation Teams
• Review and collate relevant existing research and evaluations, including through a
series of initial ‘Country Dossiers’, providing Country Evaluation Teams with some key
findings and the plan for the synthesis
31 Dec Submission of country and donor/agency-level reports
2011
Jan – Apr Dissemination of evaluation results in countries Drafting Synthesis Report
April Meeting of the International Reference Group to comment on the draft Synthesis Report
April - May Dissemination of evaluation results in countries Finalisation of Synthesis Report
May – Sep Dissemination activities/inputs to preparations for High Level Forum
29 Nov -1 Dec 4th High Level Forum in Seoul
20
references relevant to the common methodology and core questions. The Country Evaluation Teams themselves will then add further secondary information to the Dossier and to the wider literature review being conducted for the Phase 2 Evaluation.
• Provide ongoing advice and support to Country Evaluation Teams to ensure the
coherence of the evaluation and the comparability of its different elements.
42. To make best use of the support resources of the Core Evaluation Team, it will work both proactively and responsively to engage with and support the Country evaluations. In addition to important arrangements for indirect support, planned face-to-face opportunities have been identified (see Table below) to help lay solid foundations and clear directions for Country Evaluations to follow, support continuing adherence to evaluation standards, provide guidance if/ where evaluation teams run into problems, and facilitate sharing and learning among country teams.
Table: Planned face-to-face meetings by members of the Core Evaluation Team with Country Evaluation Processes
Activity/ Event Persons involved (from countries undertaking evaluation)
Face to face with country processes
Regional Workshops I (Oct/ Nov 2009)
National Evaluation Coordinators, or representatives, and major stakeholders or National Reference Group members if named.
All countries
Regional Workshop II (March/April 2010)
Team Leaders of Country Evaluation Teams, National Evaluation Coordinators – and possibly other team members
All countries
Evaluation Work-plan presentation (event)
Country Evaluation Team presentation to National Reference Group
Mission option for a limited number of countries
Inception Report Presentation (event) – 1 month
Country Evaluation Team presentation to NEC and the National Reference Group
Mission option for a limited number of countries
Team analysis ‘stage’ – August Country Evaluation Team
Mission option for the majority of countries
Draft report to National Reference/ Advisory Group (event)
Country Evaluation Team presentation to National Evaluation Coordinator and the National Reference/ Advisory Group
Mission option for a limited number of countries
International “Emerging Findings” workshop
Country Evaluation Team and National Evaluation Coordinators All countries
Final Report to CRG
Country Evaluation Team presentation to National Evaluation Coordinator and the National Reference/ Advisory Group
Remote – all countries
21
43. The Core Evaluation Team is developing a web-based knowledge management system – an “Extranet” – for the Phase 2 Evaluation. The National Evaluation Coordinator, Country Evaluation Teams and National Reference/ Advisory Group members will have access to this facility and it will provide the channel for the sharing of guidance and progress updates between the Core Team and the country processes. The structure of the extranet provides a shared space and also a part of the site that can be dedicated to a particular country process.
Draft Outline for Country Evaluation Reports December 2009
(Note: The Synthesis report for the whole comparative evaluation will closely follow a similar outline, with some additional elements to capture relevant Phase I results and the results of donor headquarters studies, and selective reference to evaluation results on country-specific questions, outside the common template.)
Preface Acknowledgement Acronyms Executive Summary (Max. 5 pp.)
• Purpose and background • Overall conclusions (on common and country-specific questions) • Key lessons (on common and country-specific questions) • Key recommendations if applicable (on common and country-specific questions)
A. Introduction (Max. 4 pp.)
• The Paris Declaration and Accra Agenda for Action: Engagement of country X • Purpose and scope of the Phase Two Evaluation • Approach, methodology and limitations
B. Country Findings on the Common Evaluation Questions
1. “What are the important factors that have affected the relevance and implementation of the Paris Declaration and its potential effects on aid effectiveness and development results?” (The Paris Declaration in context) (Max. 10 pp.)
2. “To what extent and how has the implementation of the Paris Declaration led to an improvement in the efficiency of aid delivery, the management and use of aid and better partnerships?” (Process and intermediate outcomes) (Max. 10 pp.)
3. “Has the implementation of the Paris Declaration strengthened the contribution of aid to sustainable development results? How?” (Development outcomes) (Max. 10 pp.) 4. Framework for Conclusions (Max. 10 pp.)
i. What has been the relevance of the Paris Declaration and the ways it has been implemented to the challenges of aid effectiveness?
22
ii. To what extent has each of the five principles of the Paris Declaration been observed and implemented, and the Accra Agenda priorities reflected? Why? Have there been conflicts or trade-offs between them? iii. What has the Paris Declaration achieved for aid effectiveness and development results? How significant are these contributions? How sustainable? Is there evidence of better ways to make aid more effective and contribute more to development results? iv. What effects has the implementation of the Declaration had on the respective burdens of aid management falling on partner countries and donors, relative to the changing volumes and quality of aid and of the aid partnership itself? Are these effects likely to be transitional or long term? v. What has been the added value of the Paris Declaration-style development cooperation compared with the pre-Paris Declaration situation, and seen alongside other drivers of development in the country, other sources of development finance and development cooperation partners beyond those so far endorsing the Declaration? vi. What are the key messages for a) national stakeholders, and b) donor countries and agencies? vii. What are the key implications for aid effectiveness in the future taking into account of new challenges and opportunities (e.g. climate change) and new actors and relationships?
C. Key Lessons and Recommendations (if applicable) around the Common Evaluation
Questions (Max. 5 pp.)
D. Findings on the Country-Specific Evaluation Questions (if adopted) (Max. 15 pp.) [Possible sub-headings]
E. Key Conclusions, Lessons and Recommendations (if applicable) around the Country-
Specific Evaluation Questions (if adopted) (Breaking out conclusions, lessons and recommendations) (Max. 5 pp.)
F. Possible Key Implications beyond the Planned Term of the Paris Declaration. (Max. 3
pp.)
Annex 1: Paris Declaration on Aid Effectiveness Annex 2: The Accra Agenda for Action Annex 3: Generic Terms of Reference for Country Level Evaluations Annex 4: Specific Terms of Reference for the XXX Evaluation Annex 5: Selected Additional References
23
Appendix 2: Pacific Aid Effectiveness Principles
PACIFIC ISLANDS FORUM SECRETARIAT
“PACIFIC AID EFFECTIVENESS PRINCIPLES”7 Preamble These principles derive from the Paris Declaration on Aid Effectiveness (2005). They have been work‐shopped and consulted upon widely across the Pacific region, and are designed to fit the Pacific context. The Pacific Aid Effectiveness Principles, like the original Paris declaration, include actions and approaches for both countries and development partners (donors). References to the relevant section in the Paris Declaration, and related monitoring indicators, have been included.
Principle 1: Country leadership and ownership of development through an accountable and
transparent national development planning and financial management system/mechanism which is adequately resourced from the national budget ‐ including longer term operation and maintenance of donor sponsored development (Paris Declaration Section 14, 19; Indicator 1, 2)
Principle 2: Multi‐year commitments by development partners and countries aligned nationally
identified priorities as articulated in national sustainable development strategies, or the like, with agreement on performance indicators and monitoring and evaluation mechanisms (Paris Declaration Section 16, 26; Indicators 3, 5, 7)
Principle 3: Greater Pacific ownership of regional development, Development Partners’ Pacific
Regional Strategies designed and formulated with the Pacific Plan and other Regional Policies as their corner stone (Paris Declaration 14, 15; Indicator 1)
Principle 4: Pacific Development Partners and Countries pursue a coordinated approach in the delivery of assistance. Encouraging harmonization will be a priority for both. (Paris Declaration 32 – 42; Indicators 9, 10)
Principle 5: Strengthened institutional mechanisms and capacity in countries to enable increased
use of local systems by development partners. (Paris Declaration 17, 21, 22‐24, 31; Indicator 4, 6, 8)
7 These principles are not legally binding. The Government of Japan and the European Union have expressed reservations to the principles.
24
Principle 6: (i) Provision of technical assistance (TA), including in aid coordination/management, in such a way that ensures that capacity is built with tangible benefits to the country to support national ownership. Provision of an appropriate level of counterpart resources through established procedures and mechanisms.
(ii) Short term TA, that address local skills gaps to conduct studies, are culturally sensitive. (Paris Declaration 22‐24; Indicator 4)
Principle 7: Use of an agreed monitoring and evaluation framework that will ensure joint
assessments of the implementation of agreed commitments on aid effectiveness. (Paris Declaration 43‐46; Indicator 11)
Supplementary Notes to Pacific Principles on Aid Effectiveness
Principle 1:
o Member Governments take the lead role in the establishment of priorities and the coordination of development assistance;
o Comprehensive processes for the management of development assistance with clear disbursement, acquittal and reporting procedures developed and used;
o Development projects incorporated in approved national budgets including commitment to ongoing operations and maintenance;
o Project appraisal and selection criteria developed and established, consistent with priorities outlined in National Sustainable Development Plan/Strategies;
o Members must build their capacity to manage and plan development assistance; o Demonstrated commitment at the highest decisions making level to established
processes and resolve to ensure that set processes/procedures are adhered to; o Aid Management Agencies are adequately resourced to enable national capacity
building on development assistance processes for recipients of development assistance in the wider community and undertake regular dialogue with development partners; and
o Transparent contracting and reporting arrangements with clear tender procedures developed and implemented
Principle 2:
o Partners need to support country strategies and have confidence in country mechanisms especially in terms of the prioritisation of development policies, programmes and projects. The lack of confidence weakens countries ownership of development assistance;
o Partner country programmes need to be formulated in close consultation with recipient countries to ensure that country programmes reflect Members priorities;
o Partners should programme their assistance over a multi‐year timeframe that is consistent with FICs National Sustainable Development Strategies;
o FICs endeavour to implement the FEMM Principles of Accountability where multi‐year budget frameworks are critical to national accountability and transparency ;
25
o Partners should as far as possible adopt the development performance indicators incorporated in the Members National Sustainable Development Strategies;
o Partners to be more open and provide full information on the level of development assistance regularly and in a timely manner to enable these to be incorporated into national budgets;
o Partners to use FICs structures, systems and mechanisms [budget, management, monitoring and evaluation, reporting and organizational structures];
o Consultation meetings established with recipient countries annually/biannually; and
o Maximum use of existing systems to deliver aid with an acceptance of and confidence in established mechanisms at country level
Principle 3
o Partners consult with regional organizations and use them as a conduit in the delivery of regional programmes to ensure consistency with regional priorities, especially those identified under the Pacific Plan and other regional frameworks;
o The Pacific Plan provides a framework for partners to support demand driven priorities, locally owned policies and institutions for coordination with and complementary for all stakeholders;
o Partners to collaborate with CROP and the Pacific Plan Office on the implementation, monitoring and evaluation of Pacific Plan initiatives with the agreed framework
Principle 4
o Countries to take the lead in encouraging and supporting harmonized approaches to development assistance;
o Partners to be open and share proposed respective programmes and objectives with other regional donors regularly to ensure that they are not duplicating efforts;
o Partners to develop a common framework for development cooperation and establish multi‐year programming of development assistance with common performance indicators;
o Partners to hold regular dialogue amongst themselves to determine how best they can deliver programs jointly both at the national and regional level;
o Joint country missions should be pursued in the first instance. Use and sharing of country analytic work is encouraged;
o Countries establish Multi‐Donor round table ‐ consultation meetings annually/biannually.
Principle 5 o Untying aid and greater use of local and regional firms and consultants with an
increased confidence in local and regional capacity; o Reduce reliance on third party contractors; increase use of external technical
advisers that encourage use of local consultants leading to a demonstrated development partnership between local and regional consultants;
o Partners to decentralize decision making to country based staff to avoid unnecessary delay in the implementation of project activities;
26
o The choice of aid delivery mechanism should be based on maximum cost effectiveness and efficiency; and
o Partners and countries work together to strengthen country mechanisms and capacity to allow for increased use of local systems as delivery mechanisms
Principle 6
o Countries to develop capacity and HRD policies and plans to address capacity constraints;
o Existing systems and procedures reviewed and updated at regular intervals to ensure that they are not out dated;
o Commitment to the utilisation of established mechanisms at all levels especially leadership;
o Planning and sequencing of initiatives must be in line with local needs and capabilities;
o Extensive needs analysis undertaken prior to commencement of capacity building exercises;
o Adequate resources made available to facilitate transfer of skills; o Use a range of capacity building options through flexibility and responsiveness; o Policies on the use of TA in aid programmes should be developed and largely
accepted with capacity building as the first priority of TA; TA responsible to the counterpart organization for delivery and performance; TAs to have a counterpart(s) assigned to them; TAs together with counterparts develop training manuals for counterparts and identify milestones to ensure that skills are adequately transferred and gaps addressed;
o To ensure TA occurs in an environment which is receptive to the building of capacity, the objective of public sector personnel management should be quality public service delivery rather than job security;
o Advisers are briefed on appropriate conduct and behaviour in the workplace and undertake orientation/briefing for TAs especially those in long term assignments; and
o Where possible, South/South technical cooperation and work exchanges among countries in the Pacific should be encouraged.
Principle 7
o Countries and partners work together to develop joint monitoring and evaluation frameworks for development activities; and
o Joint reviews and monitoring reports are used to assess development progress, and development effectiveness, regularly, against national development strategy indicators
Adopted on 13 July 2007, Koror, Palau
27
Evaluation Tools
Appendix 3: Matrix Evaluation Questions
Appendix 4: Interview Template
Appendix 5: Survey Questionnaire Template
28
Appendix 3: Matrix Evaluation Questions Template
Core Q1. “What are the important factors that have affected the relevance and implementation of the Paris Declaration and its potential effects on aid effectiveness and development results?” (The Paris Declaration in context)
Key Characteristics of the Country: Poverty/Development Status Sources of Evidence / Indicators
Methods / Forms of Analysis
(see Annex A, section 2)
Categories for Analysis
& Judgements
1a) What are the key characteristics of the country that have been most relevant to the implementation of the Paris Declaration?
Baseline (2000) plus changes identified between 2005 and 2010 (milestones of 2005 & 2009)
i. Human development, social and poverty conditions including gender equity.
At a minimum refer to country specific data within the following global sources (see Country Profile, Section 3 Country Dossier), complemented by standardised/ agreed data sets in the country: • Human Development Index
(UNDP) • World Bank Poverty
Assessments: Proportion of population below national poverty line (World Bank/ National)
• MDG Progress Reports (on track/ off track) (UNDP)
• Gender and Empowerment (GEM) status (UNDP/ UN Genderstats)
• Gender and Development Index (GDI) (UNDP/ UN Genderstats)
• Gini co-efficient (UNDP)
A, B Description of broad contextual/ structural conditions of the country noting and analysing trends including the pace of change in respect to:
Economic & social conditions
Governance
Development strategy
Aid management policy
Level and source of ODA
29
8 The Term National Development Strategy (NDS) is used here as it is applied in the Paris Declaration; namely, to include poverty reduction and similar overarching strategies, as well as sector and thematic strategies.
ii. Key economic features, issues and trends
• GDP / GNI per capita (World Bank/ National)
• Doing Business indicators (World Bank)
• Creditor Reporting System reports (OECD-DAC)
• Global Competitiveness Reports (2005/ 2010) World Economic Forum
• International Trade and tariff statistics (WTO/ UNCTAD)
A, B
iii. Governance and fragility (the rule of law and a functioning legislature, and respect of human rights are likely to be key conditions)
• Africa: Ibrahim Index • CPIA (World Bank) • List of Fragile States (World
Bank)
A
iv. National development strategies8 (NDS) • Presence/ absence and date of National Development Strategies (or equivalent) and revisions
• Key priorities in the NDS
A
v. Organisation of Government; aid management, decentralisation.
• Role of key actors in the development process (Government, Civil society & Private Sector).
• Presence/ absence and date of national Aid Management Policy statements/ revisions
• Constitutional arrangements; nature and extent of decentralisation.
A, F
30
vi. External and domestic resource mobilisation patterns
• National sources • Levels of Official Development
Assistance; International Development Statistics (OECD)
• Major development actors: International Development Statistics (OECD)
A, B
1b) What are the most important national and international events that have affected [in the country] the implementation of the Paris Declaration and the Accra Agenda for Action priorities, and how?
Changes identified between 2005 - 2010
i. Identification of key issues that have / are influencing the aid arena in country: For example;
• Political priorities and policy reforms, governance reforms, decentralisation.
• Economic conditions; both domestic and international macro level changes e.g. financial crisis, including changes in donor countries influencing future aid flows.
• Civil unrest, natural & man-made disasters.
• New resources (internal or external).
• Radical changes in relations with key donors, new entrants
• National / international drivers e.g. upcoming High Level Forum on Aid Effectiveness.
A, C, D Among possible list of influences, identify those that are important and rank the significance of each in terms of its implications for the implementation of the PD
• Very significant • Quite significant • Limited significance • Insignificant • Did not occur
The Degree of change in the wider environment affecting the aid arena since 2005
• Very significant change • Quite significant change • Limited change • Very limited change • No change
31
9 The Core Evaluation Team will provide the compilation of best available international sources and statistics to all Country Teams in relation to these questions, for them to cross-check, widen and deepen from national sources (e.g. National ODA database) wherever possible.
1c) What is the place of aid subject to PD principles among all sources of development finance and resources? What have been the trends from early roots to 2005 and since?”
Baseline (2000) plus changes identified between 2005 – 2010 (milestones of 2005 & 2009)
i. Pre and post PD trends in Official Development Assistance shares and components of external and overall development finance and national resource mobilisation (inc. private investment, trade receipts, remittances etc.).
OECD-DAC statistics on Official Development Assistance and other development finance9 (alongside other international sources – World Bank, UNDP, etc.)
Financial flows through South-South programmes
National budget / forecasts on revenue
Trade receipts
Remittance inflows as % of GDP / ODA
Levels / sources of new resources (internal or external)
Private sector investment (domestic / external – FDI etc)
A, B Aid / ODA as a proportion of GDP • Significant increase • Slight increase • Stable • Slight decline • Significant decline
Aid / ODA as a proportion of total national budget (and as a proportion of capital/ development budget)
• Significant increase • Slight increase • Stable • Slight decline • Significant decline
Breakdown of aid modalities:
1st level; distinction between ‘on budget’ and ‘off budget’
2nd level; distinction using the categorisation adopted by the country. This is likely to cover a number of
ii. How the government – donor/agency landscape has evolved; how close and how important is the relationship with different donors? New entrants/ new sources of development finance?
Reports Government/ Donor Forums.
D
iii. How is the delivery of aid organised, e.g. is there a strong coordinating Ministry, is there a joint assistance strategy? What are the key modalities used?
Major shifts in Partner Government (sectoral) spending and in donor/ agency aid (sectoral) commitments
A
32
iv. What shares and types of ODA flows in turn are in practice subject to PD principles?
Note: Ensure appropriate coverage of technical cooperation, South-South and triangular cooperation, NGOs/CSOs and faith-based groups, and other sources of development cooperation not covered by the PD.
See Guidance Note on “What is Aid”
Proportion of total ODA from PD signatories
Modalities (proportion of PD-type aid using programme-based approaches (GBS, SBS etc)
Number/ volume of Global Programs.
A, B categories (that may overlap); • Project aid • Technical co-operation. • Sector support (e.g. sector wide
approaches and sector ‘narrow’ approaches such as single issue support)
• General Budget Support • Other programme aid • Fellowships, scholarships etc. • Other (specify e.g. humanitarian
assistance)
Estimated share of total ‘aid’ provided from sources not yet endorsing PD.
Development and use of joint co-operation strategies / joint institutional structures
• Significant increase [volume terms] • Slight increase • Stable • Slight decline • Significant decline
1d) Which are the key actors, in the country and among its development partners, who take major decisions on aid? What influence do the Paris Declaration and AAA commitments have on them, in relation to their other priorities and incentives?
Changes identified between 2005 - 2010
i. Chronology of major decisions taken by Partner Government and donors/ agencies affecting the level and nature of ODA
Key points in budget and spending estimates cycles, main aid consultations and pledging sessions, joint performance reviews, if applicable
A, D
Proportion of total resource flows (including ODA) to decentralised structures
33
ii. Identification of the relevant key decision-makers. Maps of the key objectives, interests, capacities, priorities and motivations of key actors on both sides of the aid relationships in the country, relative to the commitments of the PD and AAA. Taking into account of changing relations with key donors, parliament, local government and decentralised funding, civil society, organisations representing women and excluded groups, private sector and media actors.
This evidence is related to, but goes beyond, the “commitment, capacities and incentives” surveyed in Phase 1.
C, F • Significant increase • Slight increase • Stable • Slight decline • Significant decline
Level of delegated authority over donor / agency decisions at country level (e.g. decisions on financial approvals and decisions on new areas of/ additional support)
• Significant increase • Slight increase • Stable • Slight decline • Significant decline
iii. Consistency / degree of decentralised decision-making between donor/agency HQs and country Field Offices.
A
iv. Who takes direct responsibility for PD implementation within national government?
A
v. How many different donors are active and what shares of aid are provided by each? How concentrated is the donor community (i.e. shares of total aid provided by the top third, middle third and bottom third of contributors by volume).
A, B
vi. Who are perceived as the five most important donors? Why? What institutional and financial resources do they direct to the implementation of the PD agenda in-country?
A, B
vii. What are the mechanisms for parliamentary, public and civil society oversight of the budget and aid allocations?
A
1e) To what extent and where have the PD principles been implemented? Baseline (2000) plus changes identified between 2005 – 2010 (milestones of 2005 & 2009)
34
i. How have the different PD principles been interpreted, weighted and implemented in the country? Since when have they been implemented in the country? (e.g., pre-2005, later)
Shifts pre-2005 towards aid effectiveness discussion / processes / mechanisms, if relevant
Date of adoption / endorsement of PD
National level PD implementation strategy / targets
Country progress reports re: PD implementation plan
Brief updating of key Phase 1 evaluation results (where applicable.)
A, C, D Length and duration of engagement with PD (from 2005) in aid effectiveness agenda
• Very significant engagement • Quite significant engagement • Limited engagement • Very limited engagement • No engagement
Length and duration of engagement prior to 2005 with ‘PD like’ aid effectiveness agenda
• Very significant engagement • Quite significant engagement • Limited engagement • Very limited engagement • No engagement
ii. What have been formal statements and changes around PD implementation (implementation plan, Aid Management Policy, PD progress reports, consultations on e.g. AAA etc)
A
35
Conclusions on the Paris Declaration in context:
A. Status and relevance of the aid effectiveness agenda: • Political engagement / take-up • Evidence of a level of continuing interest and engagement in the aid effectiveness agenda among key
stakeholders
Take-up and application of the aid effectiveness agenda
• Very significant take-up and application
• Quite significant take-up and application
• Limited take-up and application • Very limited take-up and
application • No take-up and application
Evidence of continued interest / engagement in the aid effectiveness agenda
• Substantial interest / engagement • Some interest / engagement • Little interest / engagement • No interest / engagement • Reduced interest / engagement
B. What have been the main influences 2005-2010 that have affected the ways aid has worked? • Accra Agenda for Action • Changes in the aid environment (donors, government actors, laws and regulations around aid) • Changes in national context (political change / unrest, political economy factors, change of government,
natural disasters, changing population profile etc) • Changes in international context (economic volatility, impacts of climate change etc)
C. Extent of adoption / implementation of the PD principles, and explanation • What have been the key factors influencing the extent of adoption/ implementation • Any evidence of perceived tensions / tradeoffs between principles? • Interest in and initiatives with regards to alternative approaches (not directly associated with the Paris
Declaration) to aid effectiveness at country level
36
Core Q2. “To what extent and how has the implementation of the Paris Declaration led to an improvement in the efficiency of aid delivery, the management and use of aid and better partnerships?” (Process and intermediate outcomes)
10 Core Question 2 is structured according to the main headings of the Accra Agenda for Action; (A) Ownership, (B) Partnerships and (C) Development Results.
PD expected outcomes Progress markers Potential indicators of change / milestones
Methods / Forms of Analysis
Judgement on progress, especially
since 2005 Key reasons
& explanation
Intended state of the aid relationship at country level in 2010 (PD)
What evidence of progress to date towards outcomes? 4 primary sources for the progress markers indicated:
• Paris Declaration on Aid Effectiveness (2005)
• Accra Agenda for Action (2008) (shaded)
• Monitoring Survey (MS) • Regional workshop and
IRG member suggestions
MS refers to Monitoring Survey (2005 & 2008) and the results under its numbered indicators (see Annex B for reference)
See Annex A, section 2
Rating: • Substantial • Some • Little • None • Regression
To justify the judgement on progress
A Country ownership over development10
i. Stronger national strategies and operational frameworks.
• Government lead in aid co-ordination at all levels with donors
Aid co-ordination groups led by government representatives
A, C, E • Substantial • Some • Little • None • Regression
37
11 Medium Term Expenditure Framework.
• NDS with clear, results-oriented strategic priorities and ‘bankable programmes’ linked to MTEF11 or similar
MS Ind 1
Linkages between NDS & annual and multi-annual budget processes
A • Substantial • Some • Little • None • Regression
• Increased monitoring and scrutiny through parliamentary processes of progress with the national development strategy.
Dedicated parliamentary processes and records for NDS progress review debate.
A, D • Substantial • Some • Little • None • Regression
• Fully consultative process (including civil society organisations and those representing women and excluded groups, local authorities and the private sector) in NDS development
Clear frameworks for consultation
Range of consultations undertaken and range of actors involved in dialogue
Donor funded programmes / activity around engagement with non-state actors
A, C, D
• Substantial • Some • Little • None • Regression
• NDS and sector strategies respond to international commitments on gender equality, human rights, disability and environmental sustainability
NDS has a clear analytical basis which addresses gender, rights, disability and environmental sustainability
NDS based on improved information systems, including e.g. disaggregated data around e.g. gender and disability
A • Substantial • Some • Little • None • Regression
38
12 See Glossary (Extranet) for a listing of vertical funds.
ii. Increased alignment of aid with partner countries’ priorities, systems and procedures, help to strengthen capacities
• Overall support based on partner NDS, specific related priorities of sectoral ministries and NDS/ sector progress reviews
MS Ind 3
Shifts in key donor country strategies / programmes / expenditure reflect changes in government priorities (e.g. new NDS and in progress reviews) and/or related financing gaps identified by the government
A, C, • Substantial • Some • Little • None • Regression
• Increased use by donors/ agencies (and all vertical funds12) of country systems and procedures
Diagnostic reviews on country systems / reforms undertaken
Rationale provided for non or limited use
MS Ind 6
MS Ind 5a
MS Ind 5b
A, C
• Substantial • Some • Little • None • Regression
• A single framework and/or manageable set of indicators drawn from NDS and progress reviews from which donors derive their conditions
Single framework for conditions(NDS linkages) exists
Conditions developed transparently and in consultation with other donors / government
Joint indicator / conditions frameworks including disaggregated data
A, C
• Substantial • Some • Little • None • Regression
39
• Increased support to capacity-building of country systems
MS Ind 4
Explicit objectives / strategies for capacity strengthening of partner systems within donor strategies and programmes
Volume / proportion of support to capacity-building of partner systems
A, C
• Substantial • Some • Little • None • Regression
iii. Defined measures and standards of performance and accountability of partner country systems in public financial management, procurement, fiduciary standards and environmental assessments, in line with broadly accepted good practices and their quick and widespread application
• Strengthened financial management capacity
MS Ind 2
Number of harmonised diagnostic reviews / performance assessment frameworks for PFM
Proportion of diagnostic reviews followed up.
Number and value of (joint) programmes addressing PFM / fiduciary reforms
Use of sex-disaggregated data and analysis in public financial management systems, e.g. gender audits, gender budget analysis
Use of tools such as gender audits, gender budget analysis to improve knowledge around PFM
Thematic group on PFM, procurement, and fiduciary standards set up
Number of audits of major development programmes accepted (without question) by Auditor General.
A, C
• Substantial • Some • Little • None • Regression
40
• Strengthened national procurement systems
Number of harmonised diagnostic reviews / performance assessment frameworks for procurement
Number and value of (joint) programmes & value addressing procurement reforms
Number of procurement processes where local / regional firms allowed to compete
Value of [% of total ODA] procurement under International Competitive Bidding / value of direct procurement
Thematic group on procurement, standards set up?
A, C
• Substantial • Some • Little • None • Regression
• Increased use of mutually agreed fiduciary standards
Number of processes using internationally agreed standards or accepted best practices such as OECD-World Bank diagnostic tools and others
Instances of International community agreeing on a common standard
Number of audits of Government programmes approved by Auditor General
A, C
• Substantial • Some • Little • None • Regression
41
13 See Glossary (Extranet) for definition. 14 See Glossary (Extranet) for definition.
• Increased use of mutually agreed processes to carry out diagnostics, develop fiduciary reforms and monitor implementation
Thematic group on fiduciary standards set up?
C • Substantial • Some • Little • None • Regression
B Building more inclusive and effective partnerships for development
iv. Less duplication of efforts and rationalised, more cost-effective donor activities
• Increased use of donor comparative advantage (relative strengths / complementarity) led by government
Clear views/ strategy by Government on donors comparative advantage and how to achieve increased donor complementarity
Evidence of reprogrammed aid according to statement of relative strengths
A, C, D • Substantial • Some • Little • None • Regression
• Increased ‘division of labour’13 at country / sector level
Mapping process conducted / maintained
Number and type/theme of formal Division of Labour arrangements
Reprogrammed aid according to Division of Labour agreements / arrangements
Co-operative / joint work between agencies within e.g. project modalities
A, C
• Substantial • Some • Little • None • Regression
• Increased delegation to lead donors for the execution of programmes, activities and tasks
Increased use of donor lead arrangements [e.g. ‘silent partnerships’14]
C • Substantial • Some • Little • None • Regression
42
15 See Glossary (Extranet) for definition of untying.
• Reduced fragmentation, overall
% (Proportion) of donors to total volume of aid (i.e. 80% of aid provided by 20% of the number of active donors/ agencies)
A, B • Substantial • Some • Little • None • Regression
• Reduced fragmentation within sectors
Number of programmes/ projects, transactions, contracts and funding arrangements compared to the total aid volume within a sector
A, B • Substantial • Some • Little • None • Regression
• Increased untying of aid15 MS Ind 8
See also issues raised in Untying of Aid Report (2009)
A • Substantial • Some • Little • None • Regression
v. Reformed and simplified donor policies and procedures, more collaborative behaviour
• Evidence of reforms and simplifications by individual donors in their own operations
MS Ind 10 A, C • Substantial • Some • Little • None • Regression
• Common or harmonised arrangements amongst donors at country level [for planning, funding (e.g. joint financial arrangements), disbursement, monitoring, evaluating & reporting to govt on donor activities & aid flows]
Joint Assistance Strategies / plans (national and sector level)
Joint thematic strategies on cross-cutting issues, e.g. gender, exclusion, climate and environment
Use of common procedures for pledged funds
Shared conditions for tranche funding
Joint monitoring / evaluation / reporting processes
MS Ind 9
A, C, D
• Substantial • Some • Little • None • Regression
43
• Strengthened incentives for harmonisation, alignment and results orientation
Extent to which action plans (Partner Governments, Donors/ Agencies) developed and implemented e.g. as part of the 2003 Rome High Level Forum
Joint accountability frameworks featuring changed incentives, e.g. (joint) annual programme performance reviews
Supportive incentives in donor agency performance management frameworks
A, D
• Substantial • Some • Little • None • Regression
vi. More predictable and multi-year commitments on aid flows to committed partner countries. [Has the nature of conditionalities been changed to support ownership in line with the AAA commitment (para. 25)]
• Increase in proportion of aid being committed through multi-year frameworks
Number of donors setting out indicative commitments within multi-year frameworks and delivering these
Proportion in terms of (total volume and the number of) donors/ agencies providing indicative aid commitments 2010-2015; based on 3 year commitment, on 5 year commitment.
A, B, C, E
• Substantial • Some • Little • None • Regression
• More timely and predictable aid disbursements
Number / proportion of donors with clearly set out agreed disbursement schedules with government
MS Ind 7
Share / type of aid disbursed according to schedule
Proportion of Government expenditure in line with budget.
A, B, C, E
• Substantial • Some • Little • None • Regression
44
16 See Glossary (Extranet) for typology/ listing.
• Limited set of mutually agreed conditions jointly agreed, made public and jointly assessed
Number of mutually agreed conditions made public
Number of joint assessments
A • Substantial • Some • Little • None • Regression
vii. Sufficient delegation of authority to donors’ field staff, and adequate attention to incentives for effective partnerships between donors and partner countries
• Increased levels of delegation to country offices
Levels of decision-making authority (financial approvals, reallocating resources) of donor country offices Number of technical staff within country offices proportionate to donor/agency commitment. Number of technical staff with gender expertise working within country Field Offices
A, C, D, E
• Substantial • Some • Little • None • Regression
• Increased capacity of staff in country offices to manage increased levels of delegation
Frequency of staff rotation C, E • Substantial • Some • Little • None • Regression
• Incentives for changed behaviour in line with aid effectiveness principles
Donor performance frameworks (institutional and staff) for development effectiveness including references to aid effectiveness principles
A, C, D • Substantial • Some • Little • None • Regression
viii. Sufficient integration of global programmes and initiatives into partner countries’ broader development agendas
• Global programmes work to strengthen country policy environment / institutions
Global programmes16 country implementation strategies based on NDS
National planning / monitoring frameworks incorporating global programmes
A, C • Substantial • Some • Little • None • Regression
45
ix. Stronger partner countries’ capacities to plan, manage and implement results-driven national strategies
• Results-oriented reporting and assessment frameworks for assessing the impact of development policies/ strategies.
MS Ind 11
MS Ind 4
Number of government plans /programmes / policies which set out clear linkages between expenditure and results over the medium term
Frameworks including manageable number of disaggregated indicators / for which data sources are available
Availability within Government of regularised socio-economic data sets
Use of disaggregated data (gender, excluded group) within results and assessment frameworks
A, B, C, D
• Substantial • Some • Little • None • Regression
• Donor programming and resources increasingly linked to national level/ sector level development results
Proportion of donor country plans which specify links between expenditure and results
Proportion of donor results frameworks which reflect national results areas (including cross-cutting issues e.g. gender, exclusion, climate change, environment)
A, C, D, E
• Substantial • Some • Little • None • Regression
46
• Increased joint support (analytic and financial) to capacity development for results
Explicit objectives / strategies for capacity strengthening within the NDS
Explicit objectives / strategies for capacity strengthening within donor support programmes
Joint initiatives for capacity development
Number of country capacity analyses undertaken / strategies developed
Volume / proportion of support to capacity-building objectives
A, B, C, D
• Substantial • Some • Little • None • Regression
C Delivering and accounting for development results
x. Enhanced respective accountability of countries and donors to citizens and parliaments
• Strengthened parliamentary role in NDS / budgets
Regular reviews by parliament of development policies, strategies, budgets and performance
National Audit reports on use of aid
A, C, D • Substantial • Some • Little • None • Regression
• Timely, transparent and comprehensive information on aid flows publicly available (donors)
Publicly available donor annual reports on aid flows
C, A • Substantial • Some • Little • None • Regression
47
10b) Progress towards commitments in the Accra Agenda for Action (para. 24) on transparency and accountability for development results
• Increasing accountability and transparency to the public for development results
MS Ind 12
Number of mutual assessments of ((i) General Budget Support, (ii) Sector support (iii) other programs) conducted based on country results reporting and information systems
Proportion of large Government (donor supported) programmes for which mutual assessments of an accepted quality have been completed
Number of mutual assessments which address gender and exclusion issues
A, C, D
• Substantial • Some • Little • None • Regression
• Supporting the strengthening of the existing international accountability mechanisms
Number of joint reviews of existing international accountability mechanisms e.g. number of peer reviews conducted and published
A • Substantial • Some • Little • None • Regression
xi. Less corruption and more transparency, strengthening public support and supporting effective resource mobilisation and allocation
• Greater transparency in public financial management
Records of disclosure on both sides of aid disbursements, revenues, budgets, expenditures, procurement and audits
Internal and external audits reporting progress on financial management
A, C • Substantial • Some • Little • None • Regression
48
• Increased measures to address corruption
Strategies and institutional mechanisms to tackle corruption on both sides
Number of investigations undertaken / concluded on both sides
Improved systems of investigation, legal redress, accountability and transparency in the use of public funds in partner countries.
Increased steps by donors/ agencies to combat corruption by individuals or corporations and to track, freeze and recover illegally acquired assets from donor/ agency funded programmes/ projects.
A, C
• Substantial • Some • Little • None • Regression
Conclusions on the effects of the Paris Declaration implementation:
A. Report against 3 dimensions of aid effectiveness covered by Core Question 2:
• Improvements in the efficiency of aid delivery
iv. Less duplication of efforts and rationalised, more cost-effective donor activities
v. Reformed and simplified donor policies and procedures, more collaborative behaviour [extent to which donor/ agency operations in-country are ‘fit for purpose’ in line with honouring the commitments of the Paris Declaration and Accra Agenda for Action]
• Substantial • Some • Little • None • Regression
• Improvements in the management and use of aid; extent of change, contributing factors (why?) and validity
ii. Increased alignment of aid with partner countries’ priorities, systems and procedures, help to strengthen capacities
viii. Sufficient integration of global programmes and initiatives into partner countries’ broader development agendas
i. Stronger national strategies and operational frameworks
ix. Stronger partner countries’ capacities to plan, manage and implement results-driven national strategies
• Substantial • Some • Little • None • Regression
49
xi. Less corruption and more transparency, strengthening public support and supporting effective resource mobilisation and allocation
iii. Defined measures and standards of performance and accountability of partner country systems in public financial management, procurement, fiduciary standards and environmental assessments, in line with broadly accepted good practices and their quick and widespread application
• Better (more inclusive and effective) partnerships
x. Enhanced respective accountability of countries and donors to citizens and parliaments
vi. More predictable and multi-year commitments on aid flows to committed partner countries. [Has the nature of conditionalities been changed to support ownership in line with the AAA commitment (para. 25)]
vii. Sufficient delegation of authority to donors’ field staff, and adequate attention to incentives for effective partnerships between donors and partner countries
• Substantial • Some • Little • None • Regression
B. Is there evidence of the Accra Agenda for Action triggering an acceleration of the aid effectiveness agenda in respect to progress on; (A) Country ownership over development, (B) Building of more inclusive and effective partnerships for development, and (C) delivering and accounting for development results.
C. Have there been unintended consequences of the Paris Declaration for aid effectiveness (e.g. unintended impacts on particular groups including women and girls, new transaction costs due to additional meetings, reports etc)?
D. Are there possible alternative ways of achieving more effective aid, e.g. in the experience with non-PD donors?
50
Core Q3. “Has the implementation of the Paris Declaration strengthened the contribution of aid to sustainable development results? How?” (Development outcomes) Note: Not a linear results chain – mapping [in 3a, 3b, 3c] the plausible contribution of the PD towards development results. Basis for establishing the extent to which there is evidence of PD implementation having accelerated progress towards development outcomes?
SECTOR LEVEL VIEW
3a) Were results in specific sectors enhanced through the application of the PD principles?” [minimum of the ‘Health sector’ as a case study and option of case study coverage of an additional 1-2 specified other sectors] [Further Guidance to follow]
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Effective, efficient and sustainable progress towards sector long term development goals
Good progress against sector programmes, strategies, policies and related targets
• Evidence of progress towards policy goals?
• Sector plans implemented as intended?
• Sector strategies on track?
• Progress reviews indicate that delivery on-track to meet targets?
Sector governance improved / Institutional mechanisms operating effectively
Overall scale of committed aid within the sector, predictability and disbursement.
Active and productive policy dialogue in the sector?
Alignment of aid with sector programmes, strategies, policies and related targets
Sector strategies and plans jointly financed (government and donor) to meet agreed national targets
Barriers to achievement jointly recognised and strategies in place to address them. e.g. joint sector platforms, joint
[See note on “Guidance to sector study” – Annex C for detailed set of progress markers/ indicators to work through in this column, covering;
• Efficiency in aid delivery
• Management and use of aid in the sector
• Partnerships
51
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
assessment frameworks, strategies and reviews
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Has there been an acceleration (in the period 2005-2010 compared to 2000-2004) in progress towards development outcomes in the sector
• Substantial • Some • Little • None • Regression
Has the contribution of PD implementation to any acceleration been • Substantial • Some • Little • None • Not relevant
52
MACRO LEVEL VIEW
3b) Did the implementation of the PD help countries to improve the prioritisation of the needs [beyond income poverty] of the poorest people, including women and girls?”
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Greater prioritisation of the needs of the poorest including women and girls
Increased generation and use of disaggregated data and analysis around poor and excluded groups, including women and girls?
• Extreme poverty, gender and exclusion-focused analyses conducted?
Increased recognition of extreme poverty, exclusion and gender issues within development policy and planning?
• Sector and national strategies reflecting data disaggregation by region, sex, excluded group etc?
• Policy / strategy / programmatic and sector responses?
More effective institutional machinery
For each of these interim results:
What has been the scale of aid?
The intensity and productivity of policy dialogue?
Joint recognition of barriers to achievement jointly recognised and strategies in place to address them, e.g. joint thematic platforms, strategies and reviews?
E.g.
Joint statements / dialogue / analysis
Joint programmes / activity
Joint groups / structures
Are there more / wider entry points for productive dialogue and engagement between partner countries and donors?
Unintended impacts or effects?
53
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
to comprehensively address extreme poverty, gender and exclusion issues?
• Required level of institutional mechanisms in place, staffed and functional?
Increased resource allocations to tackle extreme poverty, gender and exclusion issues?
• Levels of gender and exclusion-related budgetary allocations and expenditure flows?
• Level of pro-poor, gender responsive priorities in national strategies, budgets?
54
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Has there been any improvement in the prioritisation by partner governments of the needs of the poorest people, including women and girls (in the period 2005-2010 compared to 2000-2004).
• Substantial • Some • Little • None • Regression
Has the contribution of PD implementation to any improvement been • Substantial • Some • Little • None • Not relevant
55
MACRO LEVEL VIEW
3c) Has PD implementation led to sustainable increases in institutional capacities and social capital17 at all levels to respond to development challenges? Why, how and where, and what are the effects? [Guidance Note to follow]
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Increased institutional capacity at all levels
Improved administrative capacities among all development actors, including CSOs?
Improved ability to consult with and account to stakeholders?
Improved working through partnership and network formation?
Evidence of ‘learning by doing’?
Improved capacity to develop, implement and report upon comprehensive decentralisation plans?
Improved capacity to design and implement effective regulation?
Improved capacity for policy and strategic monitoring?
Improved capacity for evaluation and
For each of these interim results:
What has been the scale of aid?
The intensity and productivity of policy dialogue?
Barriers to achievement jointly recognised and strategies in place to address them e.g. joint strategies platforms, and reviews?
Country-led capacity development strategies?
Alignment of support to national objectives and strategies for capacity strengthening?
Effective use by donors/ agencies of existing capacities within partner countries?
Harmonised support for capacity development?
Joint identification of need for capacity strengthening to deliver services?
Donors strengthen own capacities and skills to be more responsive to country needs?
Jointly selected and managed technical co-operation?
Use of local and regional resources
17 See Glossary (Extranet) for definition.
56
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
reporting?
including South-South co-operation?
Promotion of operatonal changes to make capacity development more effective?
Unintended impacts or effects?
Increased social capital (i.e. problem-solving networks in society) at all levels
[Guidance to follow] [Guidance to follow] [Guidance to follow]
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Has there been a sustainable increase in institutional capacities and social capital at all levels to respond to development challenges (in the period 2005-2010 compared to 2000-2004). Has this been
• Substantial • Some • Little • None • Regression
Has the contribution of PD implementation to any sustainable increase been • Substantial • Some • Little • None • Not relevant
57
MACRO LEVEL VIEW
3d) How and why has the mix of aid modalities (including general or sector-specific budget support) evolved and what has been learnt on the development results?
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Country has a mix of aid modalities – defined as optimum by the partner country and respected by the donors/ agencies - to progress the NDS
Assessments of relative relevance and effectiveness of different modalities for different uses?
Assessment of management and absorptive capacity for the defined mix of modalities?
Government defines required mix of modalities?
Donors commit 66% of aid to programme based approaches where feasible?
Donors channel 50% or more of government-to-government aid through country fiduciary systems?
Agreement on optimal mix of modalities?
Combined response to desired mix of modalities including programme-based modalities to support common approaches?
Relative shares of allocations using agreed modalities?
Joint dialogue in and support for analysis of modalities?
Alignment to preferred national mix of modalities?
Harmonisation among donors re: mix of modalities?
Mutual accountability and progress reviews?
58
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
To what extent has the mix of aid modalities (in line with PD principles) changed in the period 2005-2010
• Substantial • Some • Little • None • Regression
What has been the contribution of PD implementation to any change?
• Substantial • Some • Little • None • Not relevant
59
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Conclusions on the influence of improved aid effectiveness on development outcomes:
A. Has the PD enhanced ODA’s impact on achieving the goals of the national development strategy [and the MDGs]? What factors have facilitated this change?
B. What (plausible) contribution has the Declaration made in terms of its own statement of intended effects, to:
“Increase the impact of aid in:
1. Reducing poverty
2. Reducing inequality
3. Increasing growth
4. Building capacity
5. Accelerating achievement of MDGs” (Paragraph. 2)
C. Has the implementation of the PD had unintended consequences for development results, negative or positive?
D. Is there evidence of different ways to make aid contribute more towards development results?
60
Core Q 4. Framework for Overall Conclusions:
i. What has been the relevance of the Paris Declaration and the ways it has been implemented to the challenges of aid effectiveness? ii. To what extent has each of the five principles of the Paris Declaration been observed and implemented, and the Accra Agenda priorities
reflected? Why? Have there been conflicts or trade-offs between them? iii. What has the Paris Declaration achieved for aid effectiveness and development results? How significant are these contributions? How
sustainable is it? Is there evidence of better ways to make aid more effective and contribute more to development results, for women and men and for those who are excluded?
iv. What effects has the implementation of the Declaration had on the respective burdens of aid management falling on the partner country and its respective donors, relative to the changing volumes and quality of aid and of the aid partnership itself? Are these effects likely to be transitional or long term?
v. What has been the added value of the Paris Declaration-style development cooperation compared with the pre-PD situation, and seen alongside other drivers of development in the country, other sources of development finance and development cooperation partners beyond those so far endorsing the Declaration?
vi. What are the key messages for a) national stakeholders, and b) donor countries and agencies? vii. What are the key implications for aid effectiveness in the future taking into account of the new challenges and opportunities (e.g. climate
change) and new actors and relationships?
61
Appendix 4: Interview template
Joint Evaluation of the Paris Declaration on Aid Effectiveness in the Cook Islands
Summary of Questions for Key Informant Interviews Dear Participant, We will only ask questions that you consider are appropriate and relevant to your experience of aid effectiveness. The Interview is expected to take 45 minutes. Some questions are based on a pre‐determined ranking. Others are open ended and seek your comments and examples. The interviewer will record your responses on a separate interview record. The range of questions include:
• What kind of involvement have you had with the Aid effectiveness Agenda (Extent of engagement
with aid effectiveness agenda i.e.: aid related activities, policy, planning, implementation) • How long have you been involved with Aid effectiveness agenda? (Duration of engagement with aid
effectiveness agenda i.e.: aid related activities, policy planning, implementation) • How familiar are you with the Paris Declaration? • Gender: Male /Female • Your current position/organisation
1 Country Context and Aid Effectiveness 1.1 Who makes the major decisions on behalf of the country about accepting and allocating aid?
(Offices / individual leaders/others?)
1.2 How much influence do the Paris Declaration / Accra Accord of Action priorities have on ‐ major decision makers, in relation to their other priorities and incentives?
Substantial influence/ Some influence/ A little influence/ No influence/ Not relevant
1.3 How does this influence work?
1.4 Which are the most influential donors in shaping the way aid operates in the country? List top 3‐5 1.5 Why? 1.6 Do the major donors make more decisions at HQ or field office level?
Donor: HQ / field level (circle) Donor: HQ / field level (circle) Donor: HQ / field level (circle) Donor: HQ/ field level (circle)
1.7 How does this affect aid management / co‐ordination in‐country? 1.8 When did the Paris Declaration emerge as a focus of discussion in the country? Year: _____ 1.9 How? or Why?
1.10 How would you assess Paris Declarations influence on aid effectiveness discussions in the country?
Substantial influence/ some influence/ A little influence/ No influence
62
1.11 Why? 1.12 Which of the 5 principles of the Declaration have been most prominent in discussion and
implementation? Why? 1.13 Which of the 5 principles (show list) have been least prominent in discussion and implementation?
Why? 1.14 Is there evidence of “aid effectiveness fatigue” (i.e. finding the processes of reform heavy and slow
and questioning the value of results achieved) on the part of:
Key donors? Substantial evidence/ Some evidence/ A little evidence/ No evidence
Key actors in government? Substantial evidence/ Some evidence/ A little evidence/ No evidence
Other stakeholders? Substantial evidence/ Some evidence/ A little evidence/ No evidence
1.15 If so, can you give examples? How is this fatigue affecting aid and development activity?
Describe 2. Process and Management of PD Principles 2A. Ownership and Alignment 2a.1 In general, would you say that the Government in fact leads in aid co‐ordination with donors?
Fully leading / Mainly leading / Partly leading / Not leading / Moving backwards 2a.2 How well do you think that donors as a group are accepting of / responsive to government
leadership?
Fully accepting responsive / Mainly accepting responsive / Partly accepting responsive / Not at all accepting responsive / Moving backwards
2a.3 How would you rate the development of the national strategy (NSDP) in terms of being
consultative with stakeholders in the country?
Fully consultative / Mainly consultative / Partly consultative / Not at all consultative / Moving backwards 2a.4 What explains a possible change from past practice? 2a.5 Can you think of / name actual cases where particular donors have clearly shifted their support /
priorities to reflect changes within national priorities? 2a.6 To what extent would you say that conditions (or conditionality’s”) on aid have been developed
transparently and in consultation with government and other donors?
Fully transparent consultative / Mainly transparent consultative / Partly transparent consultative / Not at all transparent consultative / Moving backwards
63
2a.7 Has there been any improvement in transparency and consultation on aid conditions in the last two years?
Substantial improvement / Some improvement / A little improvement / No change / Moving backwards
2a.8 Have you seen strengthening in the last two years of capacity building support to country systems
by donors?
Substantial strengthening / Some strengthening / A little strengthening / No strengthening / Moving backwards
2a.8i Can you think of any good examples? 2a.9 If donors are not using country systems – what are the main reasons? 2a.10 In general, have you seen any improvements in the country’s Public Financial Management
capacities in recent years?
Substantial improvement / Some improvement / A little improvement / No improvements / Moving backwards
[If no, why?] If yes, in what specific area/s?
2a.11 If yes, have you seen donors accordingly place more trust in those systems?
Substantially more / Some more / A little more / No more / Moving backwards 2a.12 In general, have you seen improvements in national procurement capacities in recent years?
Substantial improvement / Some improvement / A little improvement / No improvement / Moving backwards 2a.13 If no, why? 2a.14 If yes, have you seen donors accordingly place more trust in those systems?
Substantial improvement / Some improvement / A little improvement / No improvement / Moving backwards In what specific area/s?
2a.15 If internationally recognised standards of procurement are not being applied in the country or are
proving difficult to apply – what are the reasons? List 2B Harmonisation 2b.1 To what extent would you say that the government is steering/steers donors more to use their
particular strengths in specific areas?
Substantially more steering / Some more steering / A little more steering / No change / Less steering 2b.2 Is this done formally / informally?
Formally / Informally / Both / Not done 2b.3 In general, would you say that the government in fact leads in allocating responsibilities and setting
up the Division of Labour among donors?
64
Fully leading / Mainly leading / Partly leading / Not leading at all / Moving backwards
2b.4 Why (not)? 2b.5 Can you cite any examples of particular donors reprogramming their aid according to Division of
Labour agreements? 2b.6 Can you cite examples of donors channelling resources directly via other donors’ programmes [e.g.
“silent partnerships”?] 2b.7 Joint analytical work and shared missions among donors. How would you explain progress or lack
of progress in these areas to harmonize aid? 2b.8 Can you give examples of major reforms / changes or simplifications by major individual donors of
their policies / procedures? 2b.9 Making aid more predictable year to year and disbursing it according to schedule within the agreed
period How would you explain progress or lack of progress in these areas? 2b.10 How have donors been adapting the resources and skills of their field staff to the new ways of
managing aid?
Sufficiently adapting / Partly adapting / Insufficiently adapting / Moving backwards 2b.11 Can you cite examples of good or bad practice? 2b.12 Is there sufficient continuity among donor field office staff, or excessive turnover?
Sufficient continuity / Partly sufficient continuity / Insufficient continuity / Moving backwards 2b.13 Can you cite examples of good or bad practice? 2b.14 Can you cite examples of technical staff working for more than one donor? 2b.15 To what extent do you consider that global programmes e.g. global funds and major foundations
are working to strengthen the country’s own policies and institutions?
Substantial strengthening / Partial strengthening / A little strengthening / No strengthening / Moving backwards
2C Managing for Development Results 2c.1 Have you noticed any improvements of government monitoring frameworks at different levels?
(sector, programme, theme, national?)
Substantial improvements / Some improvements / A little improvement / No change / Moving backwards 2c.2 Are donors generally working to strengthen and use the country’s own monitoring frameworks?
Fully working / Mainly working / Partly working / Not working / Moving backwards 2c.3 Is the government leading a drive towards a greater focus on results?
Fully leading / Mainly leading / Partly leading / Not leading / Moving backwards
65
2c.4 Can you point to good examples of groups of donors working together to help the country
strengthen its capacity to manage development results? 2D Mutual Accountability 2d.1 Is there any evidence of changes in trust, mutual respect and open dialogue and flexibility between
donors and government?
Substantial increase / Partial increase / A little increase / No change / Moving backwards 2d.2 If so, can you give illustrations? 2d.3 Have there been changes in the accountability of the government towards citizens and parliament?
[e.g. increased parliamentary debate on the effectiveness of aid, increased dialogue and engagement with organised civil society?]
Substantial increase / Some increase / A little increase / No change / Moving backwards
2d.4 Can you give examples? 2d.5 Is information on aid flows and the use of aid in budgets publicly available / accessible:
From government? Fully available/ Mainly available/ Partly available/ Not available/ Moving backwards
From donors?
Fully available/ Mainly available/ Partly available/ Not available/ Moving backwards 2d.6 Are the government / donors taking increased measures to tackle corruption?
Substantially increased measures / Partially increased measures / Somewhat increased measures / No change / Moving backwards
How?
2d.7 Are you aware of any investigations undertaken / completed on both sides? 2d.8 Do you perceive rising or diminishing levels of corruption?
Substantial rise / Partial rise / No change Partial decrease / Substantial decrease 2d.9 What role do you think aid plays in this trend, if any? 2E Conclusions 2e.1 Overall, have there been improvements in the efficiency of aid delivery over the past 5 years or so?
Substantial improvement / Some improvement / Little improvement / No improvement / Moving backwards 2e.2 Overall, have there been improvements in the management and use of aid over the past 5 years or
so?
Substantial improvement / Some improvement / Little improvement / No improvement / Moving backwards
66
2e.3 Have partnerships between the country and donors become more inclusive and effective over the past 5 years?
Substantial improvement / Some improvement / Little improvement / No improvement / Moving backwards
2e.4 Have you seen any unintended effects of the PD on the effectiveness of aid? 2e.5 Have you seen different ways of achieving more effective aid (e.g. in the experience with non‐PD
donors) that seem promising? [Non‐PD donor meaning those who have not endorsed the Declaration]
3 Development Results 3a Health or Infrastructure Sectors only 3a.1 Have you seen evidence of donor support for innovative approaches to achieving development
objectives in particular sectors?
Substantial evidence / Some evidence / Little evidence / No evidence / Moving backwards
Can you cite examples? 3a.2 Have you seen evidence of aid helping leverage the country’s own policy and programs to achieve
higher level national objectives?
Substantial evidence / Some evidence / Little evidence / No evidence / Moving backwards 3b Development Results ‐ General 3b.1 Does the country have groups or sections of society that are alienated or disenfranchised, for
example because of social class, gender, minority status, disability, age or social class? 3b.2 If so, what groups and issues are primarily involved? 3b.3 Is there evidence of productive / intensive policy dialogue between the country and donors on
ways to include these groups in the mainstream of economic and social development?
Substantial dialogue / Some dialogue / Little dialogue / None / Moving backwards 3b.4 Has PD style aid (e.g. less projects, more support for national programs and budgets) improved the
contribution of aid to promoting social inclusion and gender equality?
Substantial improvement / Some improvement / Little improvement / No improvement / Moving backwards
3b.5 Has PD‐Style cooperation improved the contribution of aid to strengthening the country’s own institutions and problem‐solving networks?
Substantial progress/ Some progress / Little progress / No progress / Moving backwards
3b.6 How and why has the mix of aid modalities (including general or sector‐specific budget support,
projects and program support) changed? 3b.7 What effects do you think these changes have had on development results?
67
3c And finally: 3c.1 Looking back, (show list on separate page) which of the following factors would you say has had a
substantial influence on the implementation of aid effectiveness reforms in the country over the past five years?
• changing political priorities, • governance reforms, • economic conditions, • civil unrest, • natural & man‐made disasters, • new resources (internal or external), • decentralisation of government, • changing relations with key donors, • new sources of cooperation outside the Paris Declaration • other major factors not mentioned (please specify):
3c.5 How have these factors had an influence? Any final comments or questions you have? Thank you for participating! Cook Islands Team Health Specialists Ltd Joint Evaluation of Paris Declaration
68
Appendix 5: Survey template
Paris Declaration Evaluation – Cook Islands
Survey Questionnaire
Introduction
The Cook Islands is an international partner to the Paris Declaration on Aid Effectiveness. This survey is part of an Evaluation to examine how the Paris Declaration has been applied in the Cook Islands. The focus of the survey questionnaire is to gather in your views, ideas, knowledge, experience on aid effectiveness and development results, including poverty reduction in the Cook Islands.
The information you provide in this survey is confidential. It will be reported in a way which will not identify any organisation or person.
Instructions 1. This survey should be completed by you
2. Please answer honestly and accurately.
3. Please return your completed survey to the survey facilitator as soon as you have
completed it
4. It should take you 10 to 15 minutes
5. For assistance contact your Survey facilitator (Liz Ponga ph: 79786 , Nooroa Short Ph: 79782
or Tina Newport ph: 79785)
Paris Declarations has Five Principles of Aid Effectiveness: Ownership Developing countries to set their own strategies for development,
improve their institutions and tackle corruption. Alignment Donor Countries bring their support in line with these objectives and use
local systems. Harmonisation Donor countries co‐ordinate their action, simplify procedures and share
information to avoid duplication. Managing for results Developing countries and donors focus on producing – and measuring –
results. Mutual accountability Donor and developing country partners are accountable for
development results.
69
A: General Questions A1. Do you know what the Paris Declaration Agreement is about? (please tick one) Full knowledge Some knowledge A little knowledge No knowledge Comment: A2 Do you know about Aid effectiveness in the Cook Islands? (Please tick one) Full knowledge Some knowledge A little knowledge No knowledge Comment:
A3 What involvement do you have with the Aid effectiveness Agenda? (Please tick one or more of the following boxes)
National/sector/agency planning Aid Management & coordination Policy development Project Programme Implementation Monitoring & Evaluation Financial Management Other Please specify for “other”:________________________ B: Important factors that have affected the relevance and implementation of the Paris
Declaration and its potential effects on aid effectiveness and development results B1. Who makes the major decisions about accepting and allocating Aid on behalf of the
country? (Please tick one or more of the following boxes) Government Politicians Traditional leaders Donors NGO leaders Civil Society Other Please specify for “other”:________________________ B2 Who are the most influential donors in the Country? (Please rank in order of most influence 1 = most, 2 = second most etc) Asian Development Bank ____ Australia ____ China ____ Donor/NGO ____ European Union ____ New Zealand ____ UN Agencies (WHO, UNESCO, FAO, etc) ____ Other ____ Please specify for “other”:________________________
70
B3 What influence does the Paris Declaration have with aid effectiveness discussion in the Cook Islands?
Substantial Some a little No Moving Influence Influence Influence Change Backwards
B4 How important are Paris Declaration principles in aid effectiveness discussions and
implementation? Ownership: (The Cook Islands set their own strategies for development, improve their
institutions and tackle corruption). Very Some a little No Important Importance Importance Importance
Alignment: (Donor Countries bring their support in line with these objectives and use local
systems). Very Some a little No
Important Importance Importance Importance Harmonisation: (Donor countries co‐ordinate their action, simplify procedures and share
information to avoid duplication). Very Some a little No
Important Importance Importance Importance Managing for results: (Developing countries and donors focus on producing and measuring
results). Very Some a little No
Important Importance Importance Importance Mutual accountability: (Donor and developing country partners are accountable for
development) Very Some a little No
Important Importance Importance Importance B5 Is there evidence of ‘aid effectiveness fatigue’ (i.e. finding the processes of reform heavy
and slow and questioning the value of results achieved) on the part of:
a. Key donors Substantial Some a little No evidence evidence evidence Change comment: b. Key actors in government Substantial Some a little No evidence evidence evidence Change Comment:
71
c. Other stakeholders Substantial Some a little No
evidence evidence evidence Change Comment: C: To what extent and how has the implementation of the Paris Declaration led to an
improvement in the efficiency of aid delivery, the management and use of aid and better partnerships?
Ownership and Alignment C1 In general, would you say that the Government leads in Aid Co‐ordination with Donors? Fully Mainly Partly Not Moving
Leading Leading Leading Leading Backwards Comment:
C2 How well do you think Donors as a group are responsive to Government Leadership in Aid Co‐
ordination? Fully Mainly Partly Not Moving
Responsive Responsive Responsive at all Backwards Comment:
C3 How would you rate the development of the National Sustainable Development Plan (NSDP) in terms of being consultative with stakeholders in the Cook Islands? Fully Mainly Partly No at all Moving Consultative Consultative Consultative Consultative Backwards
Comment:
C4 Do you see an improvement in Donors co‐ordinating their aid programmes with Cook Islands
Priorities? Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards
C5 To what extent have conditions on aid been developed transparently and in consultation with
the CI government and donors? Fully transparent Mainly transparent Partly transparent No at all Moving Consultative Consultative Consultative Consultative Backwards
C6 In the last two years, have you seen strengthened capacity building support by donors of Cook Islands systems? Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards
72
C7 Have you seen improvements in the Cook Islands Local systems i.e. Public finance, local
purchasing/supplies of goods and services. Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards Comment:
C8 Are Donors using these local systems more to deliver aid?
Substantially Some a little No Moving more more more more backwards Comment:
Harmonisation C9 To what extent is the government steering donors more to use their particular strengths in
specific areas? Substantial more Some more a little more No Moving Steering Steering steering Change Backwards
C10 Have you seen donors adapting the resources and skills of their field staff to the new ways of
managing aid? Sufficiently Partly Insufficiently Moving Adapting Adapting Adapting Backwards
Managing for Results C11 Have you seen an improvement in the way the Cook Islands Government monitors
progress towards achieving its national priorities? Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards
C 12 Are donors working to strengthen and use the Cook Islands own monitoring frameworks? Fully Mainly Partly Not Moving
Working Working Working at all Backwards
C13 Have you seen Donors working together with the Cook Islands Government to improve systems to achieve better results with aid programmes?
Fully Mainly Partly Not Moving
Working Working Working at all Backwards
73
Mutual Accountability C14 Have you seen changes in trust, mutual respect, open dialogues and flexibility between
Donors and the Cook Islands Government? Substantial Partial a Little No Moving
Increase Increase Increase Change Backwards
C15 Have there been changes in the Government accountability towards citizens and parliament? (e.g. parliamentary debate on aid effectiveness, increase dialogue with Civil Society)
Substantial Partial a Little No Moving
Increase Increase Increase Change Backwards
C16 Are the Government and Donors taking increased measures to tackle corruption? Substantial Partial a Little No Moving
Increase Increase Increase Change Backwards measures measures measures
C17 Overall have there been improvements in the efficiency of aid delivery over the past five years?
Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards
C18 Overall have there been improvements in the management and use of aid over the past
five years? Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards
D: Has the implementation of the Paris Declaration strengthened the contribution of aid to sustainable development results? How?
D1 Do you think there are groups who are excluded from mainstream economic and social
development in the Cook Islands? Yes No Do not know
If so, List: D2 Are there productive policy discussions between Cook Islands Government and Donors to
include these groups in the mainstream economic and social development? Yes No Do not know
74
E: Finally, some questions about you (tick) E1: Gender Male Female
E2: Your current position: ____________________________
E3: Tick one of the following:
Government central agency
Government line ministry/department
Government owned enterprise
Non Government organisation
Donor Agency
Other:
Please specify for ‘other’: ______________________
THANK YOU FOR FINISHING THE SURVEY QUESTIONNAIRE
1. Please complete the survey questionnaire 2. Please return to Nooroa Short Ph: 79782. [email protected]
For office Use only Ref No:___ Data Coded: ____ Data Entered: ____
75
Appendix 6: List of participants
First Name Surname Position Title Organisation Julie Affleck NZAID Manager Ministry of Foreign Affairs & Trade John Akavi Senior Project Officer Ministry of Agriculture Paul Allsworth Director Cook Islands Audit Office Vaine Arioka President Punanga Tauturu Incorporated Jim Armistead Senior Research and Policy Adviser Aid Management Division Glenn Armstrong President Cook Islands Canoeing Association Debbie Ave Coordinator Cook Islands National Disability Council Steve Barrett Senior Project Officer Aid Management Division Carmel Beattie Chief Executive Officer Cook Islands Tourism Corporation Georgina Bonin Assistant Resident Representative United Nation Development Programme Anthony Brown Head of Ministry Ministry of Agriculture Mariana Bryson Finance Manager Aid Management Division Ken Buchanan Chief Executive Officer Office of the Minister of Health Tepaeru Cameron Senior Administration Officer Island Administration Office, Aitutaki Kevin Carr Acting Financial Secretary, NSDC Ministry of Finance & Economic
Management Pasha Carruthers Technical Adviser, Climate Change National Environment Service George Cowan Chairman Cook Islands Infrastructure Committee Rod Dixon Director University of the South Pacific, Cook Islands Brenda Drollet Head of Ministry Ministry of Internal Affairs Des Eggelton Committee Member Cook Islands Infrastructure Committee Hon Cassey Eggelton Cabinet Minister Minister (Environment & Culture) Tingika Elikana Solicitor General, NSDC Member Crown Law Office Boyd Ellison Manager Cook Islands Trades Training Centre Navy Epati Public Service Commissioner, NSDC Office of the Public Service Commissioner Jackie Evans Programme Manager Te Ipukarea Society Tupou Faireka Head of Ministry Ministry of Health Dr Rangi Fariu Director of Public Health Ministry of Health Eusenio Fatialofa Project Officer Cook Island Investment Corporation Rongo File Manager Cook Island Family Welfare Association Ann Fisher President Te Tupu o te Manava Outrigger Canoe Club Debbie Futter-Puati HIV & STT Coordinator Ministry of Health Dimitri Geidelberg Programme Manager NZAID George George Executive Member Avatiu Sports Club Jan Goldsworthy Manager Small Island States AusAID Hilary Gorman Research officer Pacific Island Aids Foundation Jim Gossellin Head of Ministry, NSDC member Ministry of Foreign Affairs & Immigration Peter Graham Legal Adviser Ministry of Marine Resources Robert Graham Sports Development Officer Cook Islands Sports & National Olympic
Committee Helen Greig Senior News Reporter Cook Island News Hon William Heather Cabinet Minister Minister (Infrastructure & Planning) Taamo Heather Executive Member Arorangi Girls Brigade Garth Henderson Manager Aid Management Division Alex Henry Human Resource Manager Office of the Public Service Commissioner Helen Henry Island Secretary Mangaia Island Administration Atatoa Herman Consultant Infrastructure Sector Tepaeru Herman Director, International Affairs Ministry of Foreign Affairs & Immigration John Hobbs Commissioner Financial Services Commission Elise Huffer Team Leader Human Development
Prog. South Pacific Commission
76
First Name Surname Position Title Organisation Karl Hunter Director, Pacific Affairs Ministry of Foreign Affairs & Immigration Tarita Hutchinson Chief Executive Officer Office of the Minister for Environment &
Culture Sholan Ivaiti Former Financial Secretary Ministry of Finance & Economic
Management Lewu Katoa Island Secretary Pukapuka Island Administration Romani Katoa Committee Member Cook Islands Infrastructure Committee Ngara Katuke National Secretary Cook Islands Assn. for Non Govt
Organisation Dr Stephenie Knight Development Program Manager NZAID Kora Kora Island Mayor Manihiki Cathy Koteka Health Educator Cook Island Family Welfare Association Liz Koteka Director of Policy & Planning Office of the Prime Minister Keith Leornard Regional Director, South Pacific
office Asian Development Bank
Vereara Maeva Former President Cook Islands Assn. for Non Govt Organisation
Janet Maki Ombudsman Office of the Ombudsman Martha Makirere Executive Member Cook Islands Assn. for Non Govt
Organisation Teresa Manarangi
Trott Consultant, Executive Member Chamber of Commerce & Creative Centre
Sir Terepai Maoate Former Deputy Prime Minister Member of Parliament Tuaine Marsters President Cook Islands Assn. for Non Govt
Organisation Pricislla Maruariki Chief Executive Officer Office of the Public Service Commissioner Rt. Hon. Jim Marurai Prime Minister Prime Minister, (Education, Police) Brian Mason Committee Member Cook Islands Budget Committee Punatau Mataio Island Council Member Rakahanga Island Administration Miimetua Matamaki 2nd National Communications
Coordinator National Environment Service
Jeanne Matenga Television Producer Pitt Media Group Ratu Mato President Ngatangiia Sports Association Maria Melei Country Specialist, South Pacific
office Asian Development Bank
Lloyd Miles Acting Chief Executive Officer Cook Islands Investment Corporation Joshua Mitchell Director Ministry of Marine Resources Myra Moekaa Director Ministry of Foreign Affairs & Immigration Ina Mokoroa Island Secretary Atiu Island Administration Mac Mokoroa Chief of Staff, Chairman NSDC Office of the Prime Minister Peter Muller Regional Disaster Response Adviser South Pacific Commission Elizabeth Munro Biodiversity Officer National Environment Service Selina Napa Business Development Officer Business Trade & Investment Board Nadine Newnham Executive Member Te Vaerua Community Rehabilitation Service Joe Ngamata Chief Executive Airport Authority Arona Ngari Manager Meterological Office Ngatoko Ngatoko Director of Biodiversity Ministry of Agriculture Nileema Noble Resident Representative United Nation Development Programme Donye Numa Policy Adviser Office of the Prime Minister Nooroa Numanga Director of Disability Issues Ministry of Internal Affairs Petero Okotai Planning Adviser Office of the Prime Minister Helen Omera Treasurer Cook Island National Council of Women Tepaeru Opo President Cook Island Family Welfare Association Sharyn Paio Head of Ministry Ministry of Education George Paniani Sports Development Officer Cook Islands Sports & Nat’l Olympic
Committee Nooroa Parakoti Consultant Ministry of Outer Islands & Planning Edward Parker Administration Officer Aid Management Division
77
First Name Surname Position Title Organisation Tahaki Paulo Island Mayor Penrhyn Richard Phelps Senior Infrastructure Specialist Asian Development Bank Tamarii Pierre Chairman Government Budget Committee Hon Apii Piho Cabinet Minister Minister for (Justice, Health & Internal
Affairs) Charles Pitt News Report/Director Cook Islands Herald Shona Pitt News Report/Director Pitt Media Group Trevor Pitt Chief Executive Officer Office of the Prime Minister Ruta Pokura Director, Gender Development Ministry of Internal Affairs Ben Ponia Head of Ministry Ministry of Marine Resources Trina Pureau Chief Executive Officer Office of the Deputy Prime Minister Daryl Rairi Technical Services Assistant Ministry of Infrastructure & Planning Fiona Ramsey Head of Section Social Sector Delegation - European Union for the Pacific Terry Rangi Chief Executive Officer Development Investment Board Hon. Wilkie Rasmussen Cabinet Minister Minister (Finance) Nikki Rattle General Secretary Cook Islands Red Cross Kori Raumea Director Ministry Marine Resources Ann Raymond Finance Officer Pacific Island Aids Foundation Tiria Rere Chief Live Stock Officer Ministry of Agriculture Dahpne Ringi Former Director, Finance & Planning Ministry of Health Teru Ringi Former Project Officer Aid Management Division Nooroa Roi Senior Policy Officer Ministry of Marine Resources Lorna Rolls Assistant Resident Representative United Nations Population Fund Teariki Rongo Coordinator Community Initiative Scheme & GEF Prog. Teariki Rouru Senior Administration Officer Atiu Island Administration Nani Samuela Manager Punanga Tauturu Incorporated Mark Short Head of Ministry Ministry of Justice Dorothy Solomona Director, Pearl Division Ministry of Marine Resources Sabati Solomona Island Secretary Aitutaki Island Administration Elizabeth Sosene Secretary Te Tup o te Manava Outrigger Canoe Club John Strickland Superintendent Cook Islands Police Henry Taiki Adviser World Monetary Organisation Mereana Taikoko Coordinator Te Kainga - Mental Health Services Tere Taio Chief Executive Officer Minister of Ministry of Outer Islands &
Planning Aukino Tairea Head of Ministry Ministry of Transport Karen Tairea Dietician/Nutritionist Ministry of Health Mahina Tairi Public Health Officer Ministry of Health Peter Taivairanga Project Officer Aid Management Division Kato Tama Board Chairman Airport Authority Edwina Tangaroa Public Health Educator Ministry of Health Ngarangi Tangaroa Programme Manager Dept National Human Resource
Development Junior Tangata NGO Officer Ministry of Internal Affairs Otheniel Tangianau Acting Head of Ministry Ministry of Infrastructure & Planning Taggy Tangimetua Chief Statistician Ministry of Finance & Economic
Management Maki Taokia Member Rutaki Women Community Group Anne Taoro Property Manager Cook Island Investment Corporation Mata Taramai Finance Officer Ministry of Health Carmen Temata Performance Analyst Office of the Public Service Commissioner Lotoika Tengere Island Mayor Pukapuka Vaine Teokotai Chief Executive Officer Office of the Minister for Finance Adrian Teotahi Director, Water Works Ministry of Infrastructure & Planning Fred Tereva Island Mayor Mitiaro
78
First Name Surname Position Title Organisation Maara Tetava Commissioner of Police Cook Islands Police Teu Teulilo Treasury Operations Manager Ministry of Finance & Economic Mgmt Peter Tierney Development Programme
Coordinator NZAID
Apii Timoti Chief Executive Officer Te Aponga Uira Jonah Tisam Chief Executive Officer Office of the Opposition Violet Tisam Quality Assurance Moderator National Human Resource Department Polly Tongia Coordinator Cook Island National Council of Women Frances Topa Head of Ministry Dept. of National Human Resource Devmt. Tai Topa Island Secretary Mitiaro Island Administration Frances Topa-Apera Acting President Cook Island National Council of Women Tangata Tou Business Manager Development Investment Board Travel Tou Ariki President House of Ariki Liane Towgood Adviser, Gender Development
Division Ministry of Internal Affairs
Gail Townsend Director, Policy & Planning Ministry of Education Angeline Tuara Consultant/Researcher Private Sector Maria Tuoro Policy Adviser Office of the Prime Minister Vaitoti Tupa Head of Ministry National Environment Service Tai Tura Island Secretary Mauke Island Administration Anthony Turua Director, Finance Ministry of Education Taunga Tuteru Island Secretary Rakahanga Island Administration Ake Utanga Executive Member Cook Island Family Welfare Association Andrew Vaeau Island Secretary Penhryn Island Administration Monique Ward Development Programme Officer NZAID Hamish Weir Senior Auditor Audit Office Henry Wichman President Outdoor Pursuits Organisation Vaine Wichman Consultant Private Sector Hon Robert Wigmore Deputy Prime Minister Deputy Prime Minister, ( Foreign Affairs) William Wigmore Director of Research Ministry of Agriculture Bob Williams Brigade Leader Avarua Boys Brigade Dallas Young Budget & Economic Policy Manager Ministry of Finance & Economic Mgmt
79
Evaluation Evidence/Assessment/Results
Summary of Matrix Rated Questionnaires’
• National
• Infrastructure
• Health
Summary of Interviews
Summary of Survey Results
Summary of Project Profiles (x6)
6 Tracer Study Profiles
80
Appendix 7: National Rated matrix
Evaluation Questionnaire Matrix Core Q1. “What are the important factors that have affected the relevance and implementation of the Paris Declaration and its potential effects on aid effectiveness and development results?” (The Paris Declaration in context)
Key Characteristics of the Country: Poverty/Development Status Sources of Evidence / Indicators
Methods / Forms of Analysis
(see Annex A, section 2)
Categories for Analysis
& Judgements
1a) What are the key characteristics of the country that have been most relevant to the implementation of the Paris Declaration?
Baseline (2000) plus changes identified between 2005 and 2010 (milestones of 2005 & 2009)
i. Human development, social and poverty conditions including gender equity.
Cook Islands specific data from the following global sources (see Country Profile, Section 3 Country Dossier), complemented by standardised/ agreed data sets in the country: • Human Development Index
(UNDP) • World Bank Poverty
Assessments: Proportion of population below national poverty line (World Bank/ National)
• MDG Progress Reports (on track/ off track) (UNDP)
• Gender and Empowerment (GEM) status (UNDP/ UN Genderstats)
• Gender and Development Index (GDI) (UNDP/ UN Genderstats)
• Gini co-efficient (UNDP)
A, B Description of broad contextual/ structural conditions of the country noting and analysing trends including the pace of change in respect to:
Economic & social conditions
Governance
Development strategy
Aid management policy
Level and source of ODA
81
Key Characteristics of the Country: Poverty/Development Status Sources of Evidence / Indicators
Methods / Forms of Analysis
(see Annex A, section 2)
Categories for Analysis
& Judgements
ii. Key economic features, issues and trends
• GDP / GNI per capita (World Bank/ National)
• Doing Business indicators (World Bank)
• Creditor Reporting System reports (OECD-DAC)
• Global Competitiveness Reports (2005/ 2010) World Economic Forum
• International Trade and tariff statistics (WTO/ UNCTAD)
A, B
iii. Governance and fragility (the rule of law and a functioning legislature, and respect of human rights are likely to be key conditions)
• CPIA (World Bank) • List of Fragile States (World
Bank)
A
iv. National development strategies18 (NSDP) • National Sustainable Development Plan
• Key priorities in the NSDP
A
18 The Term National Development Strategy (NDS) is used here as it is applied in the Paris Declaration; namely, to include poverty reduction and similar overarching strategies, as well as sector and thematic strategies.
82
Key Characteristics of the Country: Poverty/Development Status Sources of Evidence / Indicators
Methods / Forms of Analysis
(see Annex A, section 2)
Categories for Analysis
& Judgements
v. Organisation of Government; aid management, decentralisation.
• Role of key actors in the development process (Government, Civil Society & Private Sector).
• Presence/ absence and date of national Aid Management Division Policy statements/ revisions
• Constitutional arrangements; nature and extent of decentralisation.
• Self Governing in free Association with New Zealand legislation
A, F
vi. External and domestic resource mobilisation patterns
• National sources • Levels of Official Development
Assistance; International Development Statistics (OECD)
• Major development actors: International Development Statistics (OECD)
A, B
1b) What are the most important national and international events that have affected [in the country] the implementation of the Paris Declaration and the Accra Agenda for Action priorities, and how?
Changes identified between 2005 - 2010
83
Key Characteristics of the Country: Poverty/Development Status Sources of Evidence / Indicators
Methods / Forms of Analysis
(see Annex A, section 2)
Categories for Analysis
& Judgements
i. Key issues (ranked) that have / are influencing the aid arena in the Cook Islands since 2005 include:
1. Natural Disasters – Cyclones 2005 & 2010; Dengue Fever Outbreaks, Manmade - Pacific Games 2009. quite significant.
2. Economic conditions; both domestic and international macro level changes e.g. 2008/09 Global financial crisis. Increase in donor countries influencing future aid flows. – quite significant
3. Radical changes in relations with key donors, new entrants – quite significant
4. Political priorities and policy reforms, governance reforms including change of government from NZ election 2008. – quite significant
5. Pacific plan, National / international drivers e.g. upcoming High Level Forum on Aid Effectiveness limited significance.
6. New resources (internal or external) insignificant.
Donor reports
Historical Government Documents
Aid/Donor policy documents
Emergency Management Cook Islands
A, C, D Among possible list of influences, identify those that are important and rank the significance of each in terms of its implications for the implementation of the PD
• Quite significant
The Degree of change in the wider environment affecting the aid arena since 2005
• Limited/Quite significant change
1c) What is the place of aid subject to PD principles among all sources of development finance and resources? What have been the trends from early roots to 2005 and since?”
84
Key Characteristics of the Country: Poverty/Development Status Sources of Evidence / Indicators
Methods / Forms of Analysis
(see Annex A, section 2)
Categories for Analysis
& Judgements
Baseline (2000) plus changes identified between 2005 – 2010 (milestones of 2005 & 2009)
i. Pre and post PD trends in Official Development Assistance shares and components of external and overall development finance and national resource mobilisation (inc. private investment, trade receipts, remittances etc.).
OECD-DAC statistics on Official Development Assistance and other development finance19 (alongside other international sources – World Bank, UNDP, etc.)
Financial flows through South-South programmes
National budget / forecasts on revenue
Trade receipts
Remittance inflows as % of GDP / ODA
Levels / sources of new resources (internal or external)
Private sector investment (domestic / external – FDI etc)
A, B Aid / ODA as a proportion of GDP • Significant increase • Slight increase since 2001, 2005
& 2009? • Stable • Slight decline • Significant decline
Aid / ODA as a proportion of total national budget (and as a proportion of capital/ development budget)
• Significant increase • Slight increase since 2001, 2005
& 2009 • Stable • Slight decline • Significant decline
Breakdown of aid modalities:
1st level; distinction between ‘on budget’ and ‘off budget’ % on budget
ii. How the government – donor/agency landscape has evolved; how close and how important is the relationship with different donors? New entrants/ new sources of development finance?
Reports Government/ Donor Forums.
D
iii. How is the delivery of aid organised, e.g. is there a strong coordinating Ministry, is there a joint assistance strategy? What are the key modalities used?
Major shifts in Partner Government (sectoral) spending and in donor/ agency aid (sectoral) commitments
AUSAid White paper
A
19 The Core Evaluation Team will provide the compilation of best available international sources and statistics to all Country Teams in relation to these questions, for them to cross-check, widen and deepen from national sources (e.g. National ODA database) wherever possible.
85
Key Characteristics of the Country: Poverty/Development Status Sources of Evidence / Indicators
Methods / Forms of Analysis
(see Annex A, section 2)
Categories for Analysis
& Judgements
iv. What shares and types of ODA flows in turn are in practice subject to PD principles?
Note: Ensure appropriate coverage of technical cooperation, South-South and triangular cooperation, NGOs/CSOs and faith-based groups, and other sources of development cooperation not covered by the PD.
See Guidance Note on “What is Aid”
Proportion of total ODA from PD signatories
Modalities (proportion of PD-type aid using programme-based approaches (GBS, SBS etc)
Number/ volume of Global Programs.
Donor modality evaluation reports
A, B 2nd level; distinction using the categorisation adopted by the country. This is likely to cover a number of categories (that may overlap);
• Project aid – 2001 - current • Technical co-operation. 2001 -
current • Sector support (e.g. sector wide
approaches and sector ‘narrow’ approaches such as single issue support) 2001 – current
• General Budget Support – 2005 – 2010 – limited examples
• SWAP – 2005 – increasing • Other programme aid 2001 –
current • Fellowships, scholarships etc.2001
– current • Other (specify e.g. humanitarian
assistance) 2001 – current
Estimated share of total ‘aid’ provided from sources not yet endorsing PD.- China
Development and use of joint co-operation strategies / joint institutional structures
• Significant increase [volume terms] • Slight increase • Stable • Slight decline • Significant decline
86
Key Characteristics of the Country: Poverty/Development Status Sources of Evidence / Indicators
Methods / Forms of Analysis
(see Annex A, section 2)
Categories for Analysis
& Judgements
1d) Which are the key actors, in the country and among its development partners, who make major decisions on aid? What influence do the Paris Declaration and AAA commitments have on them, in relation to their other priorities and incentives?
Changes identified between 2005 - 2010
i. Chronology of major decisions taken by Partner Government and donors/ agencies affecting the level and nature of ODA
Key points in budget and spending estimates cycles, main aid consultations and pledging sessions, joint performance reviews, if applicable
A, D
Proportion of total resource flows (including ODA) to decentralised structures
• Significant increase • Slight increase • Stable • Slight decline • Significant decline
Level of delegated authority over donor / agency decisions at country level (e.g. decisions on financial approvals and decisions on new areas of/ additional support]
• Significant increase • Slight increase • Stable • Slight decline • Significant decline
ii. Identification of the relevant key decision-makers. Maps of the key objectives, interests, capacities, priorities and motivations of key actors on both sides of the aid relationships in the country, relative to the commitments of the PD and AAA. Taking into account of changing relations with key donors, parliament, local government and decentralised funding, civil society, organisations representing women and excluded groups, private sector and media actors.
This evidence is related to, but goes beyond, the “commitment, capacities and incentives” surveyed in Phase 1.
C, F
iii. Consistency / degree of decentralised decision-making between donor/agency HQs and country Field Offices.
Aid Management Division A
iv. Who takes direct responsibility for PD implementation within national government?
Government Policy Document A
v. How many different donors are active and what shares of aid are provided by each? How concentrated is the donor community (i.e. shares of total aid provided by the top third, middle third and bottom third of contributors by volume).
Government Budget A, B
87
Key Characteristics of the Country: Poverty/Development Status Sources of Evidence / Indicators
Methods / Forms of Analysis
(see Annex A, section 2)
Categories for Analysis
& Judgements
vi. Who are perceived as the five most important donors? Why? What institutional and financial resources do they direct to the implementation of the PD agenda in-country?
Government Budget and policy statements, interviews and survey data.
Donor policy docs
A, B
vii. What are the mechanisms for parliamentary, public and civil society oversight of the budget and aid allocations?
Government policy and process documents, interviews and survey data
A
1e) To what extent and where have the PD principles been implemented? Baseline (2000) plus changes identified between 2005 – 2010 (milestones of 2005 & 2009)
i. How have the different PD principles been interpreted, weighted and implemented in the country? Since when have they been implemented in the country? (e.g., pre-2005, later)
Shifts pre-2005 towards aid effectiveness discussion / processes / mechanisms, if relevant
Date of adoption / endorsement of PD
National level PD implementation strategy / targets
Cook Islands progress reports re: PD implementation plan
Brief updating of key Phase 1 evaluation results (where applicable.)
A, C, D Length and duration of engagement with PD (from 2005) in aid effectiveness agenda
• Very significant engagement • Quite significant engagement • Limited engagement • Very limited engagement • No engagement
Length and duration of engagement prior to 2005 with ‘PD like’ aid effectiveness agenda
• Very significant engagement • Quite significant engagement • Limited engagement • Very limited engagement • No engagement
ii. What have been formal statements and changes around PD implementation (implementation plan, Aid Management Policy, PD progress reports, consultations on e.g. AAA etc)
A
88
Conclusions on the Paris Declaration in context:
A. Status and relevance of the aid effectiveness agenda: • Political engagement / take-up - little • Evidence of a level of continuing interest and engagement in the aid effectiveness agenda among key
stakeholders - some
Take-up and application of the aid effectiveness agenda
• Very significant take-up and application
• Quite significant take-up and application
• Limited take-up and application • Very limited take-up and
application • No take-up and application
Evidence of continued interest / engagement in the aid effectiveness agenda
• Substantial interest / engagement • Some interest / engagement • Little interest / engagement • No interest / engagement • Reduced interest / engagement
B. What have been the main influences 2005-2010 that have affected the ways aid has worked? • Accra Agenda for Action – none/little • Changes in the aid environment (donors, government actors, laws and regulations around aid)some • Changes in national context (political change / unrest, political economy factors, change of government,
natural disasters, changing population profile etc)substantial • Changes in international context (economic volatility, impacts of climate change etc) some
Rating: • Substantial • Some • Little • None
Regression
89
C. Extent of adoption / implementation of the PD principles, and explanation • What have been the key factors influencing the extent of adoption/ implementation • Any evidence of perceived tensions / tradeoffs between principles? Yes
• Interest in and initiatives with regards to alternative approaches (not directly associated with the Paris Declaration) to aid effectiveness at country level
Evidence of continued interest / engagement in the aid effectiveness agenda
• Substantial interest / engagement • Some interest / engagement • Little interest / engagement • No interest / engagement • Reduced interest / engagement
90
Core Q2. “To what extent and how has the implementation of the Paris Declaration led to an improvement in the efficiency of aid delivery, the management and use of aid and better partnerships?” (Process and intermediate outcomes)
20 Core Question 2 is structured according to the main headings of the Accra Agenda for Action; (A) Ownership, (B) Partnerships and (C) Development Results.
PD expected outcomes Progress markers Potential indicators of change / milestones
Methods / Forms of Analysis
Judgement on progress, especially
since 2005 Key reasons
& explanation
Intended state of the aid relationship at country level in 2010 (PD)
What evidence of progress to date towards outcomes? 4 primary sources for the progress markers indicated:
• Paris Declaration on Aid Effectiveness (2005)
• Accra Agenda for Action (2008) (shaded)
• Monitoring Survey (MS) • Regional workshop and
IRG member suggestions
MS refers to Monitoring Survey (2005 & 2008) and the results under its numbered indicators (see Annex B for reference)
See Annex A, section 2
Rating: • Substantial • Some • Little • None • Regression
To justify the judgement on progress
A Country ownership over development20
i. Stronger national strategies and operational frameworks.
• Cook Islands Government lead in aid co-ordination at all levels with donors
Aid co-ordination groups led by government representatives
A, C, E • Substantial • Some • Little • None • Regression
NSDP govt led process.
AMD functions
91
21 Medium Term Expenditure Framework.
• NSDP with clear, results-oriented strategic priorities and ‘bankable programmes’ linked to MTEF21 or similar
MS Ind 1
Linkages between NSDP & annual and multi-annual budget processes
A • Substantial • Some • Little • None • Regression
NSDP in place, MTBF in place and increasing links between the two with annual budget
• Increased monitoring and scrutiny through parliamentary processes of progress with the national development strategy.
Dedicated parliamentary processes and records for NSDP progress review debate.
A, D • Substantial • Some • Little • None • Regression
No legislative requirements. Some debate through annual budget process and govt reports
• Fully consultative process (including civil society organisations and those representing women and excluded groups, local authorities and the private sector) in NSDP development
Clear frameworks for consultation
Range of consultations undertaken and range of actors involved in dialogue
Donor funded programmes / activity around engagement with non-state actors
A, C, D
• Substantial • Some • Little • None • Regression
systematic CS involvement in place for range of donor and govt national planning docs and some review/eval processes.
• NDS and sector strategies respond to international commitments on gender equality, human rights, disability and environmental sustainability
NSDP has clear analytical basis which addresses gender, rights, disability and environmental sustainability
NDS based on improved information systems, including e.g. disaggregated data around e.g. gender and disability
A
• Substantial • Some • Little • None • Regression
Yes - MDGs, NSDP. Gender disaggregated data. Not for disability and ES. not clear how feeds into National processes
92
22 See Glossary (Extranet) for a listing of vertical funds.
ii. Increased alignment of aid with partner countries’ priorities, systems and procedures, help to strengthen capacities
• Overall support based on partner NDS, specific related priorities of sectoral ministries and NSDP/ sector progress reviews
MS Ind 3
Shifts in key donor country strategies / programmes / expenditure reflect changes in government priorities (e.g. new NSDP and in progress reviews) and/or related financing gaps identified by the government
A, C,
• Substantial • Some • Little • None • Regression
Yes and increasing alignment although not picking up on all financing gaps.
Donor policy changes challenge govt.
• Increased use by donors/ agencies (and all vertical funds22) of country systems and procedures
Diagnostic reviews on Cook Islands systems / reforms undertaken
Rationale provided for non or limited use
MS Ind 6
MS Ind 5a
MS Ind 5b
A, C
• Substantial • Some • Little • None • Regression
Varied across donors. 75% - 90% target.
NZ/Aus channel 60% of programme through govt systems.
• A single framework and/or manageable set of indicators drawn from NSDP and progress reviews from which donors derive their conditions
Single framework for conditions(NSDP linkages) exists
Conditions developed transparently and in consultation with other donors / government
Joint indicator / conditions frameworks including disaggregated data
A, C
• Substantial • Some • Little • None • Regression
NSDP framework in place but has number of issues. Effort increasing by DP and Govt to develop
93
• Increased support to capacity-building of country systems
MS Ind 4
Explicit objectives / strategies for capacity strengthening of partner systems within donor strategies and programmes
Volume / proportion of support to capacity-building of partner systems
A, C
• Substantial • Some • Little • None • Regression
Varies from donor to donor.
Little data easily accessible.
iii. Defined measures and standards of performance and accountability of partner country systems in public financial management, procurement, fiduciary standards and environmental assessments, in line with broadly accepted good practices and their quick and widespread application
• Strengthened financial management capacity
MS Ind 2
Number of harmonised diagnostic reviews / performance assessment frameworks for PFM
Proportion of diagnostic reviews followed up.
Number and value of (joint) programmes addressing PFM / fiduciary reforms
Use of sex-disaggregated data and analysis in public financial management systems, e.g. gender audits, gender budget analysis
Use of tools such as gender audits, gender budget analysis to improve knowledge around PFM
Thematic group on PFM, procurement, and fiduciary standards set up
Number of audits of major development programmes accepted (without question) by Auditor General.
A, C
• Substantial • Some • Little • None • Regression
Yes and area of focus for further development.
Data not easily accessible.
PSC review and PEFA in pipeline.
Increased number of unqualified audits identified
94
• Strengthened national procurement systems
Number of harmonised diagnostic reviews / performance assessment frameworks for procurement
Number and value of (joint) programmes & value addressing procurement reforms
Number of procurement processes where local / regional firms allowed to compete
Value of [% of total ODA] procurement under International Competitive Bidding / value of direct procurement
Thematic group on procurement, standards set up?
A, C
• Substantial • Some • Little • None • Regression
Little stats available. Interview identify perceived improvements.
Tender committee process active.
• Increased use of mutually agreed fiduciary standards
Number of processes using internationally agreed standards or accepted best practices such as OECD-World Bank diagnostic tools and others
Instances of International community agreeing on a common standard
Number of audits of Government programmes approved by Auditor General
A, C
• Substantial • Some • Little • None • Regression
PEFA agreed
Need for Aid information management system
Increasing number of audits – sign off through parliament process.
95
23 See Glossary (Extranet) for definition. 24 See Glossary (Extranet) for definition.
• Increased use of mutually agreed processes to carry out diagnostics, develop fiduciary reforms and monitor implementation
Thematic group on fiduciary standards set up?
C • Substantial • Some • Little • None • Regression
No baseline
B Building more inclusive and effective partnerships for development
iv. Less duplication of efforts and rationalised, more cost-effective donor activities
• Increased use of donor comparative advantage (relative strengths / complementarity) led by government
Clear views/ strategy by Government on donors comparative advantage and how to achieve increased donor complementarity
Evidence of reprogrammed aid according to statement of relative strengths
A, C, D • Substantial • Some • Little • None • Regression
Increasing. No clear govt strategy with some effort of analysis by Donors
• Increased ‘division of labour’23 at country / sector level
Mapping process conducted / maintained
Number and type/theme of formal Division of Labour arrangements
Reprogrammed aid according to Division of Labour agreements / arrangements
Co-operative / joint work between agencies within e.g. project modalities
A, C
• Substantial • Some • Little • None • Regression
Over/under subscribed in some sectors.
NZ /Aus programme harmonised account for 62% aid flow and in virtually all sectors
• Increased delegation to lead donors for the execution of programmes, activities and tasks
Increased use of donor lead arrangements [e.g. ‘silent partnerships’24]
C • Substantial • Some • Little • None • Regression
Little movements since NZ/Aus prog. Established
96
25 See Glossary (Extranet) for definition of untying.
• Reduced fragmentation, overall
% (Proportion) of donors to total volume of aid (i.e. 80% of aid provided by 20% of the number of active donors/ agencies)
A, B • Substantial • Some • Little • None • Regression
Data limited.
• Reduced fragmentation within sectors
Number of programmes/ projects, transactions, contracts and funding arrangements compared to the total aid volume within a sector
A, B • Substantial • Some • Little • None • Regression
Only where sector plans in place, educ, health.
Data limited. No clear policy
• Increased untying of aid25 MS Ind 8
See also issues raised in Untying of Aid Report (2009)
A • Substantial • Some • Little • None • Regression
Requisite of some programmes to use donor TA/goods
v. Reformed and simplified donor policies and procedures, more collaborative behaviour
• Evidence of reforms and simplifications by individual donors in their own operations
MS Ind 10 A, C • Substantial • Some • Little • None • Regression
High level talks –NZ/Aus/CI
UNDAF
97
• Common or harmonised arrangements amongst donors at country level [for planning, funding (e.g. joint financial arrangements), disbursement, monitoring, evaluating & reporting to govt on donor activities & aid flows]
Joint Assistance Strategies / plans (national and sector level)
Joint thematic strategies on cross-cutting issues, e.g. gender, exclusion, climate and environment
Use of common procedures for pledged funds
Shared conditions for tranche funding
Joint monitoring / evaluation / reporting processes
MS Ind 9
A, C, D
• Substantial • Some • Little • None • Regression
NZ/Aus/CI
UNDAF
ADB/NZ PPTA at sector programme level only
• Strengthened incentives for harmonisation, alignment and results orientation
Extent to which action plans (Partner Governments, Donors/ Agencies) developed and implemented e.g. as part of the 2003 Rome High Level Forum
Joint accountability frameworks featuring changed incentives, e.g. (joint) annual programme performance reviews
Supportive incentives in donor agency performance management frameworks
A, D
• Substantial • Some • Little • None • Regression
Not much since NZ/Aus programme developed but government developing NSDP2 and good time to explore next phase of development.
98
vi. More predictable and multi-year commitments on aid flows to committed partner countries. [Has the nature of conditionalities been changed to support ownership in line with the AAA commitment (para. 25)]
• Increase in proportion of aid being committed through multi-year frameworks
Number of donors setting out indicative commitments within multi-year frameworks and delivering these
Proportion in terms of (total volume and the number of) donors/ agencies providing indicative aid commitments 2010-2015; based on 3 year commitment, on 5 year commitment.
A, B, C, E
• Substantial • Some • Little • None • Regression
Donors shifting towards setting out multiyear indicatives.
Draft aid policy sets clearer direction on this
• More timely and predictable aid disbursements
Number / proportion of donors with clearly set out agreed disbursement schedules with government
MS Ind 7
Share / type of aid disbursed according to schedule
Proportion of Government expenditure in line with budget.
A, B, C, E
• Substantial • Some • Little • None • Regression
Main donors set out schedules. Timing out of sync with govt budget timeframes. Process delays moving from indicative budgets to contract and disbursements
• Limited set of mutually agreed conditions jointly agreed, made public and jointly assessed
Number of mutually agreed conditions made public
Number of joint assessments
A • Substantial • Some • Little • None • Regression
No examples
99
26 See Glossary (Extranet) for typology/ listing.
vii. Sufficient delegation of authority to donors’ field staff, and adequate attention to incentives for effective partnerships between donors and partner countries
• Increased levels of delegation to country offices
Levels of decision-making authority (financial approvals, reallocating resources) of donor country offices Number of technical staff within country offices proportionate to donor/agency commitment. Number of technical staff with gender expertise working within country Field Offices
A, C, D, E
• Substantial • Some • Little • None • Regression
Mainly off shore delegation across donors.
40% of NZ staff
No gender experts positions in the field
• Increased capacity of staff in country offices to manage increased levels of delegation
Frequency of staff rotation C, E • Substantial • Some • Little • None • Regression
Recent moving from little to some. NZ, EU, UN increase in-country capacity.
• Incentives for changed behaviour in line with aid effectiveness principles
Donor performance frameworks (institutional and staff) for development effectiveness including references to aid effectiveness principles
A, C, D • Substantial • Some • Little • None • Regression
Recent AusAID incentives with MMR
viii. Sufficient integration of global programmes and initiatives into partner countries’ broader development agendas
• Global programmes work to strengthen country policy environment / institutions
Global programmes26 country implementation strategies based on NSDP
National planning / monitoring frameworks incorporating global programmes
A, C • Substantial • Some • Little • None • Regression
Linked to NSDP but with sub-regional management framework in place
100
ix. Stronger partner countries’ capacities to plan, manage and implement results-driven national strategies
• Results-oriented reporting and assessment frameworks for assessing the impact of development policies/ strategies.
MS Ind 11
MS Ind 4
Number of government plans /programmes / policies which set out clear linkages between expenditure and results over the medium term
Frameworks including manageable number of disaggregated indicators / for which data sources are available
Availability within Government of regularised socio-economic data sets
Use of disaggregated data (gender, children, outer islands) within results and assessment frameworks
A, B, C, D
• Substantial • Some • Little • None • Regression
Govt early stages of establishing its framework, Donor TA made available
Socio-econ data available through Stats office.
Links plan, expenditure and results not clearly developed
• Donor programming and resources increasingly linked to national level/ sector level development results
Proportion of donor country plans which specify links between expenditure and results
Proportion of donor results frameworks which reflect national results areas (including cross-cutting issues e.g. gender, children, outer island, climate change, environment)
A, C, D, E
• Substantial • Some • Little • None • Regression
NZ Education sector support example of links plan, $$s and results.
101
• Increased joint support (analytic and financial) to capacity development for results
Explicit objectives / strategies for capacity strengthening within the NSDP
Explicit objectives / strategies for capacity strengthening within donor support programmes
Joint initiatives for capacity development
Number of country capacity analyses undertaken / strategies developed
Volume / proportion of support to capacity-building objectives
A, B, C, D
• Substantial • Some • Little • None • Regression
Readiness assessment carried out. IPDET
C Delivering and accounting for development results
x. Enhanced respective accountability of countries and donors to citizens and parliaments
• Strengthened parliamentary role in NDS / budgets
Regular reviews by parliament of development policies, strategies, budgets and performance
National Audit reports on use of aid
A, C, D • Substantial • Some • Little • None • Regression
AMD audits
Not parliamentary review other than national budget
• Timely, transparent and comprehensive information on aid flows publicly available (donors)
Publicly available donor annual reports on aid flows
C, A • Substantial • Some • Little • None • Regression
Annual reports not easily and widely accessible at local level
102
10b) Progress towards commitments in the Accra Agenda for Action (para. 24) on transparency and accountability for development results
• Increasing accountability and transparency to the public for development results
MS Ind 12
Number of mutual assessments of ((i) General Budget Support, (ii) Sector support (iii) other programs) conducted based on country results reporting and information systems
Proportion of large Government (donor supported) programmes for which mutual assessments of an accepted quality have been completed
Number of mutual assessments which address gender and exclusion issues
A, C, D
• Substantial • Some • Little • None • Regression
Refer to 10a
• Supporting the strengthening of the existing international accountability mechanisms
Number of joint reviews of existing international accountability mechanisms e.g. number of peer reviews conducted and published
A • Substantial • Some • Little • None • Regression
Refer 10a
xi. Less corruption and more transparency, strengthening public support and supporting effective resource mobilisation and allocation
• Greater transparency in public financial management
Records of disclosure on both sides of aid disbursements, revenues, budgets, expenditures, procurement and audits
Internal and external audits reporting progress on financial management
A, C • Substantial • Some • Little • None • Regression
More country than donor disclosure info available
Govt audit reports published
103
• Increased measures to address corruption
Strategies and institutional mechanisms to tackle corruption on both sides
Number of investigations undertaken / concluded on both sides
Improved systems of investigation, legal redress, accountability and transparency in the use of public funds in partner countries.
Increased steps by donors/ agencies to combat corruption by individuals or corporations and to track, freeze and recover illegally acquired assets from donor/ agency funded programmes/ projects.
A, C
• Substantial • Some • Little • None • Regression
Increased confidence in system and action taken.
CI performance high in the region.
Conclusions on the effects of Paris Declaration implementation:
A. Report against 3 dimensions of aid effectiveness covered by Core Question 2:
• Improvements in the efficiency of aid delivery
iv. Less duplication of efforts and rationalised, more cost-effective donor activities little
v. Reformed and simplified donor policies and procedures, more collaborative behaviour [extent to which donor/ agency operations in the Cook Islands are ‘fit for purpose’ in line with honouring the commitments of the Paris Declaration and Accra Agenda for Action] little
• Substantial • Some • Little • None • Regression
• Improvements in the management and use of aid; extent of change, contributing factors (why?) and validity
ii. Increased alignment of aid with partner countries’ priorities, systems and procedures, help to strengthen capacities - some
viii. Sufficient integration of global programmes and initiatives into partner countries’ broader development agendas - little
i. Stronger national strategies and operational frameworks some
ix. Stronger partner countries’ capacities to plan, manage and implement results-driven national strategies
• Substantial • Some • Little • None • Regression
104
little/some
xi. Less corruption and more transparency, strengthening public support and supporting effective resource mobilisation and allocation – little/some
iii. Defined measures and standards of performance and accountability of partner country systems in public financial management, procurement, fiduciary standards and environmental assessments, in line with broadly accepted good practices and their quick and widespread application - little
• Better (more inclusive and effective) partnerships
x. Enhanced respective accountability of countries and donors to citizens and parliaments – little
vi. More predictable and multi-year commitments on aid flows to committed partner countries. [Has the nature of conditionalities been changed to support ownership in line with the AAA commitment (para. 25)]little/some
vii. Sufficient delegation of authority to donors’ field staff, and adequate attention to incentives for effective partnerships between donors and partner countries - little
• Substantial • Some • Little • None • Regression
B. Is there evidence of the Accra Agenda for Action triggering an acceleration of the aid effectiveness agenda in respect to progress on; (A) Country ownership over development, (B) Building of more inclusive and effective partnerships for development, and (C) delivering and accounting for development results.
Little
C. Have there been unintended consequences of the Paris Declaration for aid effectiveness (e.g. unintended impacts on particular groups including women and girls, new transaction costs due to additional meetings, reports etc)?
Not clear
D. Are there possible alternative ways of achieving more effective aid, e.g. in the experience with non-PD donors? Not clear
105
Core Q3. “Has the implementation of the Paris Declaration strengthened the contribution of aid to sustainable development results? How?” (Development outcomes) Note: Not a linear results chain – mapping [in 3a, 3b, 3c] the plausible contribution of the PD towards development results. Basis for establishing the extent to which there is evidence of PD implementation having accelerated progress towards development outcomes?
SECTOR LEVEL VIEW
3a) Were results in specific sectors enhanced through the application of the PD principles?” [minimum of the ‘Health sector’ as a case study and option of case study coverage of an additional 1-2 specified other sectors]
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Effective, efficient and sustainable progress towards sector long term development goals
Good progress against sector programmes, strategies, policies and related targets
• Evidence of progress towards policy goals?
• Sector plans implemented as intended?
• Sector strategies on track?
• Progress reviews indicate that delivery on-track to meet targets?
Sector governance improved / Institutional mechanisms operating effectively
Overall scale of committed aid within the sector, predictability and disbursement.
Active and productive policy dialogue in the sector?
Alignment of aid with sector programmes, strategies, policies and related targets
Sector strategies and plans jointly financed (government and donor) to meet agreed national targets
Barriers to achievement jointly recognised and strategies in place to address them. e.g. joint sector platforms, joint assessment
[See note on “Guidance to sector study” – Annex C for detailed set of progress markers/ indicators to work through in this column, covering;
• Efficiency in aid delivery
• Management and use of aid in the sector
• Partnerships
106
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
frameworks, strategies and reviews
Judgement on progress, especially since 2005
• Substantial- • Some - health
infra • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some – H&I • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some Health/Infra • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some - Health • Little - Infra • None • Regression
Has there been an acceleration (in the period 2005-2010 compared to 2000-2004) in progress towards development outcomes in the sector
• Substantial • Some H& I • Little • None • Regression
Has the contribution of PD implementation to any acceleration been • Substantial • Some H&I • Little • None • Not relevant
Range of operational constraints in both sectors.
Some changes in relationships and aid behaviour Limited working knowledge & capacity.
Increasing dialogue and plans.
107
MACRO LEVEL VIEW
3b) Did the implementation of the PD help countries to improve the prioritisation of the needs [beyond income poverty] of the poorest people, including women and girls?”
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Greater prioritisation of the needs of the poorest including women and girls
Increased generation and use of disaggregated data and analysis around poor and excluded groups, including women and girls?
• Extreme poverty, gender and exclusion-focused analyses conducted?
Increased recognition of extreme poverty, exclusion and gender issues within development policy and planning?
• Sector and national strategies reflecting data disaggregation by region, sex, excluded group etc?
• Policy / strategy / programmatic & and sector responses?
More effective institutional machinery to comprehensively address extreme poverty, gender and exclusion issues?
For each of these interim results:
What has been the scale of aid?
The intensity and productivity of policy dialogue?
Joint recognition of barriers to achievement jointly recognised and strategies in place to address them, e.g. joint thematic platforms, strategies and reviews?
E.g.
Joint statements / dialogue / analysis
Joint programmes / activity
Joint groups / structures
Are there more / wider entry points for productive dialogue and engagement between partner countries and donors?
Unintended impacts or effects?
Outer islands focus inherent in CI focus not necessarily clear that PD improve this focus but has potential
Limited data accessible to quantify scale of aid focused on poor/vulnerable.
Statements and programmes have focus on vulnerable.
Gender not integrated into national development agenda.
108
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
• Required level of institutional mechanisms in place, staffed and functional?
Increased resource allocations to tackle extreme poverty, gender and exclusion issues?
• Levels of gender and exclusion-related budgetary allocations and expenditure flows?
• Level of pro-poor, gender responsive priorities in national strategies, budgets?
109
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Has there been any improvement in the prioritisation by partner governments of the needs of the poorest people, including women and girls (in the period 2005-2010 compared to 2000-2004).
• Substantial • Some • Little • None • Regression
Has the contribution of PD implementation to any improvement been • Substantial • Some • Little • None • Not relevant
PD supports what already in place and has potential to do more.
110
MACRO LEVEL VIEW
3c) Has PD implementation led to sustainable increases in institutional capacities and social capital27 at all levels to respond to development challenges? Why, how and where, and what are the effects? [Guidance Note to follow]
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Increased institutional capacity at all levels
Improved administrative capacities among all development actors, including CSOs?
Improved ability to consult with and account to stakeholders?
Improved working through partnership and network formation?
Evidence of ‘learning by doing’?
Improved capacity to develop, implement and report upon comprehensive decentralisation plans?
Improved capacity to design and implement effective regulation?
Improved capacity for policy and strategic monitoring?
Improved capacity for evaluation and reporting?
For each of these interim results:
What has been the scale of aid?
The intensity and productivity of policy dialogue?
Barriers to achievement jointly recognised and strategies in place to address them e.g. joint strategies platforms, and reviews?
Country-led capacity development strategies?
Alignment of support to national objectives and strategies for capacity strengthening?
Effective use by donors/ agencies of existing capacities within partner countries?
Harmonised support for capacity development?
Joint identification of need for capacity strengthening to deliver services?
Donors strengthen own capacities and skills to be more responsive to country needs?
Jointly selected and managed technical co-operation?
Use of local and regional resources including South-South co-operation?
Promotion of
Continued government improvement
Increased demands with increased aid flows
Limited increase in local resources to address demands.
Increased capacity at central agency and some line ministry levels. Less evident in island administrations.
27 See Glossary (Extranet) for definition.
111
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
operational changes to make capacity development more effective?
Unintended impacts or effects?
Increased social capital (i.e. problem-solving networks in society) at all levels
[Guidance to follow] [Guidance to follow] [Guidance to follow] Strong social networks previously in place. PD expands scope of engagement
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Has there been a sustainable increase in institutional capacities and social capital at all levels to respond to development challenges (in the period 2005-2010 compared to 2000-2004). Has this been
• Substantial • Some • Little • None • Regression
Has the contribution of PD implementation to any sustainable increase been • Substantial • Some • Little • None • Not relevant
Overall strong development.
Builds on previous efforts.
Nature and pace slow. Foundation in place to build further on.
112
MACRO LEVEL VIEW
3d) How and why has the mix of aid modalities (including general or sector-specific budget support) evolved and what has been learnt on the development results?
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Country has a mix of aid modalities – defined as optimum by the partner country and respected by the donors/ agencies - to progress the NDS
Assessments of relative relevance and effectiveness of different modalities for different uses?
Assessment of management and absorptive capacity for the defined mix of modalities?
Government defines required mix of modalities?
Donors commit 66% of aid to programme based approaches where feasible?
Donors channel 50% or more of government-to-government aid through country fiduciary systems?
Agreement on optimal mix of modalities?
Combined response to desired mix of modalities including programme-based modalities to support common approaches?
Relative shares of allocations using agreed modalities?
Joint dialogue in and support for analysis of modalities?
Alignment to preferred national mix of modalities?
Harmonisation among donors re: mix of modalities?
Mutual accountability and progress reviews?
Yet to reach optimum mix.
Large number of project/programme led by donors .vs. country and sit at lower order.
113
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
Judgement on progress, especially since 2005
• Substantial • Some • Little • None • Regression
To what extent has the mix of aid modalities (in line with PD principles) changed in the period 2005-2010
• Substantial • Some • Little • None • Regression
What has been the contribution of PD implementation to any change?
• Substantial • Some • Little • None • Not relevant
Shifts towards earmarked budget support with sector programme, less so on full budget support.
High demands with low order modalities.
PD brings positive focus
114
Intended development
results (specify) Interim development
results
Contribution of aid (finance/ other) to the
sector Effects of PD on the
aid relationship Overall aggregate judgement Key reasons & explanations
Conclusions on the influence of improved aid effectiveness on development outcomes:
A. Has the PD enhanced ODA’s impact on achieving the goals of the national development strategy [and the MDGs]? What factors have facilitated this change?
Yes- lays a foundation to build on
B. What (plausible) contribution has the Declaration made in terms of its own statement of intended effects, to:
“Increase the impact of aid in:
1. Reducing poverty
2. Reducing inequality
3. Increasing growth
4. Building capacity
5. Accelerating achievement of MDGs” (Paragraph. 2)
C. Has the implementation of the PD had unintended consequences for development results, negative or positive?
Some positive
D. Is there evidence of different ways to make aid contribute more towards development results? South/South cooperation
115
Core Q 4. Framework for Overall Conclusions:
i. What has been the relevance of the Paris Declaration and the ways it has been implemented to the challenges of aid effectiveness? ii. To what extent has each of the five principles of the Paris Declaration been observed and implemented, and the Accra Agenda priorities
reflected? Why? Have there been conflicts or trade-offs between them? iii. What has the Paris Declaration achieved for aid effectiveness and development results? How significant are these contributions? How
sustainable? Is there evidence of better ways to make aid more effective and contribute more to development results, for women and men and for those who are excluded?
iv. What effects has the implementation of the Declaration had on the respective burdens of aid management falling on the partner country and its respective donors, relative to the changing volumes and quality of aid and of the aid partnership itself? Are these effects likely to be transitional or long term?
v. What has been the added value of Paris Declaration-style development cooperation compared with the pre-PD situation, and seen alongside other drivers of development in the country, other sources of development finance and development cooperation partners beyond those so far endorsing the Declaration?
vi. What are the key messages for a) national stakeholders, and b) donor countries and agencies? vii. What are the key implications for aid effectiveness in the future taking into account of new challenges and opportunities (e.g. climate
change) and new actors and relationships?
116
Appendix 8: Infrastructure rated matrix
Paris Declaration Evaluation Phase 2 Cook Islands Infrastructure Sector Tracer Study
A SECTOR LEVEL VIEW – Cook Islands Infrastructure
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially since 2005
Key reasons & explanation
Intended state of the aid relationship in the sector / area in 2010 (PD)
What evidence of progress to date towards outcomes? 4 primary sources for outcomes:
• Paris Declaration on Aid Effectiveness (2005)
• Accra Agenda for Action (2008) (shaded)
• Monitoring Survey (MS) • Regional workshop
suggestions
MS refers to Monitoring Survey and the results under its numbered indicators (see Annex B for reference)
See Annex A, section 2
Rating: • Substantial • Some • Little • None • Regression
To justify the judgement on progress
A Country ownership over development
i. Stronger national strategies and operational frameworks (in the sector)
• Government lead in aid co-ordination in the sector with donors
Aid co-ordination groups in the sector led by government representatives
Aid co-ordination groups led by donor representatives
A, D • Substantial • Some • Little • None • Regression
Infrastructure forum
Infrastructure committee
? policy by govt to lead coordination
117
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially since 2005
Key reasons & explanation
• Sector strategy with clear, results-oriented strategic priorities [linked to MTEF or similar]
MS Ind 1
Linkages between sector strategy & annual and multi-annual budget processes
A • Substantial • Some • Little • None • Regression
PIMP
Some costings
• Increased monitoring and scrutiny through parliamentary processes of progress with the sector strategy
Dedicated parliamentary processes and records for sector strategy progress review debate
A, D • Substantial • Some • Little • None • Regression
No legislative requirement but ad-hoc through govt annual reporting
• Fully consultative process (including civil society, parliaments, local authorities and the private sector) in sector strategy development
Clear frameworks for consultation
Range of consultations undertaken and range of actors involved in dialogue
A, D • Substantial • Some • Little • None • Regression
Cultural ethos for dialogue.
2010 forum
Rarotonga only
• Sector strategies respond to international commitments on gender equality, human rights, disability and environmental sustainability
Sector strategy has clear analytical basis which addresses gender, rights, disability and environmental sustainability
Sector strategy based on improved information systems, including e.g. disaggregated data around e.g. gender and disability
A • Substantial • Some • Little • None • Regression
NSDP, PIMP, MDGs
ii. Increased alignment of aid with partner countries’ priorities, systems and procedures, help to strengthen capacities (in the
• Overall support to the sector based on national sector strategy and progress reviews
MS Ind 3
Shifts in key donor sector strategies / programmes / expenditure reflect changes in government priorities (e.g. new policies, strategies in the sector)
A, D • Substantial • Some • Little • None • Regression
NZAID/AusAID, ADB, China, EU programme, activity shifts
118
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially since 2005
Key reasons & explanation
sector) • Increased use (in the sector) by
donors/ agencies (and all vertical funds28) of country systems and procedures
Diagnostic reviews on country systems / reforms undertaken in the sector
Rationale provided for non or limited use of country systems in the sector
MS Ind 6
MS Ind 5a
MS Ind 5b
A, D • Substantial • Some • Little • None • Regression
Partial use by China, NZ and EU with expected increase
Limited stats available
CIMRIS, ADB TAs,
• Increased support to capacity-building of country systems in the sector
MS Ind 4
Explicit objectives / strategies for capacity strengthening of partner systems in the sector within donor strategies and programmes
Volume / proportion of support to capacity-building of national sector systems
A, D • Substantial • Some • Little • None • Regression
ADB TA -sector management reform
PFTAC
B Building more inclusive and effective partnerships for development
28 See Glossary (Extranet) for a listing of vertical funds.
119
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially since 2005
Key reasons & explanation
iv. Less duplication of efforts and rationalised, more cost-effective donor activities (in the sector)
• Increased use of donor comparative advantage/ complementarity led by government in the sector
Clear view [statement] by Govt on donor comparative advantage in the sector and how to achieve donor complementarity
Reprogrammed sectoral aid according to statement of comparative advantage
A, D • Substantial • Some • Little • None • Regression
Anecdotal evidence of govt policy/ statement. Some donor analysis in strategy docs.
Modalities by grant (EU,NZ)
ADB/China by loan
• Increased ‘division of labour’29 at sector level
Mapping process conducted / maintained
Number and theme/type of formal Division of Labour arrangements
Reprogrammed aid to the sector according to Division of Labour agreements / arrangements
Co-operative / joint work between agencies within e.g. project modalities
A, D • Substantial • Some • Little • None • Regression
No clear mapping by govt, some donor analysis as above.
PPTA
• Increased delegation to lead donors for the execution of programmes, activities and tasks at sector level
Increased use of donor lead arrangements in the sector [e.g. ‘silent partnerships’30]
D • Substantial • Some • Little • None • Regression
PPTA with NZ, Aus ADB
29 See Glossary (Extranet) for definition. 30 See Glossary (Extranet) for definition.
120
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially since 2005
Key reasons & explanation
• Reduced fragmentation in the sector
Number of programmes/ projects, transactions, contracts and funding arrangements compared to the total aid volume within the sector
A, B • Substantial • Some • Little • None • Regression
Complicated and cross cutting with environment/economic development
• Increased untying of aid MS Ind 8
See also issues raised in Untying of Aid Report (2009)
A • Substantial • Some • Little • None • Regression
Shift in china approach over time x 3 projects. Use of NZ TAs tied
v. Reformed and simplified donor policies and procedures, more collaborative behaviour (in the sector)
• Evidence of reforms and simplifications by individual donors at sector level
MS Ind 10 A, D • Substantial • Some • Little • None • Regression
Donor to donor dialogue but little/no evidence of collaboration
• Common or harmonised arrangements amongst donors at sector level [for planning, funding, disbursement, monitoring, evaluating & reporting to govt on donor activities & aid flows]
Joint Assistance Strategies / plans (sector level)
Use of common procedures for pledged funds
Shared conditions for tranche funding
Joint monitoring/ evaluation/ reporting processes
MS Ind 9
A, D • Substantial • Some • Little • None • Regression
NZ/Aus harmonised programme, but ABD, EU efforts
Use of PIMP, CRRP
• Strengthened incentives for harmonisation, alignment and results orientation in the sector
Joint accountability frameworks for the sector featuring changed incentives
Supportive incentives in sector-level donor agency performance management frameworks
A, D • Substantial • Some • Little • None • Regression
121
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially since 2005
Key reasons & explanation
vi. More predictable and multi-year commitments on aid flows (to the sector). [Has the nature of conditionalities - within the sector - been changed to support ownership in line with the AAA commitment (para. 25)]
• Increase in proportion of aid being committed to the sector through multi-year frameworks
Number of donors setting out indicative commitments for the sector within multi-year frameworks and delivering these
Proportion in terms of (total volume and the number of) donors/ agencies providing indicative aid commitments to the sector 2010-2015; based on 3 year commitment, on 5 year commitment
A, B, C, E • Substantial • Some • Little • None • Regression
Donors operating with multiyear frameworks. Issue is delivering on schedules
• More timely and predictable aid disbursements to the sector
Number / proportion of donors with clearly set out agreed disbursement schedules with government
MS Ind 7
Share / type of aid in the sector disbursed according to schedule
Proportion of government aid for the sector in line with budget
A, B, ,D • Substantial • Some • Little • None • Regression
As above
Weakness with implementation
• Limited set of mutually agreed conditions in the sector, jointly agreed, made public and jointly assessed
Number of mutually agreed conditions made public
Number of joint assessments
A • Substantial • Some • Little • None • Regression
Agreed individual donor to govt
122
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially since 2005
Key reasons & explanation
viii. Sufficient integration of global programmes and initiatives into partner countries’ broader development agenda (for the sector).
• Global programmes work to strengthen sector policy environment / institutions
Global programmes31 country implementation strategies reflect relevant sector strategy
Sector planning / monitoring frameworks incorporating global programmes
A, D • Substantial • Some • Little • None • Regression
Through GEF/ environment sector
ix. Stronger partner countries’ capacities to plan, manage and implement results-driven strategies (for the sector).
• Results-oriented reporting and assessment frameworks in the sector for assessing the impact of development policies
MS Ind 11
MS Ind 4
Number of sector plans which set out clear linkages between expenditure and results over the medium term
Sector frameworks including manageable number of indicators / for which data sources are available
Availability within Government of regularised socio-economic data sets
Use of disaggregated data (gender, excluded group) within results and assessment frameworks
A, B, D • Substantial • Some • Little • None • Regression
PIMP
MDGs
Inputs and outputs focused monitoring and reporting
31 See Glossary (Extranet) for typology/ listing.
123
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially since 2005
Key reasons & explanation
• Donor programming and resources in the sector increasingly linked to results
Proportion of donor sector plans which specify links between expenditure and results
Proportion of donor sector results frameworks which reflect national results areas (including cross-cutting issues e.g. gender, exclusion, climate change, environment)
A, D. E NSDP and MDG targets
C Delivering and accounting for development results
x. Enhanced respective accountability of countries and donors to citizens and parliaments (for the sector)
• Timely, transparent and comprehensive information on aid flows to the sector publicly available (donors)
Audit reports on use of aid in the sector
Publicly available donor annual reports on aid flows
A, D • Substantial • Some • Little • None • Regression
Issues specific audits carried out.
Internet access to annual reports not accessible to all stakeholders
xi. Less corruption and more transparency, strengthening public support and supporting effective resource mobilisation and allocation (for the sector).
• Greater transparency in public financial management
Records of disclosure of aid revenues, budgets, expenditures, procurement and audits in the sector
Internal and external audits reporting progress on financial management
A, D
• Substantial • Some • Little • None • Regression
By special investigation
124
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially since 2005
Key reasons & explanation
Conclusions on the effects of Paris Declaration implementation:
A. Report against the 3 components of Core Question 2 with respect to the sector / area:
• Improvements in the efficiency of aid delivery to the sector
Efforts in sector planning, government initiated changes due to chronic implementation delays
• Substantial • Some • Little • None • Regression
• Improvements in the management and use of aid in the sector
Increasing move towards private sector delivery.
• Substantial • Some • Little • None • Regression
• Better (more inclusive and effective) partnerships in the sector
Increasing involvement of CS (includes private sector, and NGOs, village level given increased concern for environment impacts of infrastructure development.
• Substantial • Some • Little • None • Regression
B. Have there been unintended consequences of the Paris Declaration for aid effectiveness in the sector (e.g. unintended impacts on particular groups including women and girls)?
Any engagement with CS creates increased demands on women
C. Are there possible alternative ways of achieving more effective aid in the sector, e.g. in the experience with non-PD donors?
125
Appendix 9: Health rated matrix
Paris Declaration Evaluation Phase 2 Cook Islands Health Sector Tracer Study
A SECTOR LEVEL VIEW – Cook Islands Health Sector
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially
since 2005 Key reasons &
explanation
Intended state of the aid relationship in the sector / area in 2010 (PD)
What evidence of progress to date towards outcomes? 4 primary sources for outcomes:
• Paris Declaration on Aid Effectiveness (2005)
• Accra Agenda for Action (2008) (shaded)
• Monitoring Survey (MS) • Regional workshop
suggestions
MS refers to Monitoring Survey and the results under its numbered indicators (see Annex B for reference)
See Annex A, section 2
Rating: • Substantial • Some • Little • None • Regression
To justify the judgement on progress
A Country ownership over development
i. Stronger national strategies and operational frameworks (in the sector)
• Government lead in aid co-ordination in the sector with donors
Aid co-ordination groups in the sector led by government representatives
Aid co-ordination groups led by donor representatives
A, D • Substantial • Some • Little • None • Regression
Govt lead in internal coordination with In-country stakeholders , Health Conference
Engage with donors 1-1
• Sector strategy with clear, results-oriented strategic priorities [linked to MTEF or similar]
MS Ind 1
Linkages between sector strategy & annual and multi-annual budget processes
A • Substantial • Some • Little • None • Regression
2006 health strategy in place – linked to annual budget process only, NSDP,
126
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially
since 2005 Key reasons &
explanation
• Increased monitoring and scrutiny through parliamentary processes of progress with the sector strategy
Dedicated parliamentary processes and records for sector strategy progress review debate
A, D • Substantial • Some • Little • None • Regression
Only through MOH reporting and audits.
• Fully consultative process (including civil society, parliaments, local authorities and the private sector) in sector strategy development
Clear frameworks for consultation
Range of consultations undertaken and range of actors involved in dialogue
A, D • Substantial • Some • Little • None • Regression
As part of NSDP sector planning led by OPM. Thematic consultations, e.g.: TB, Disability
? account taken of CS delivery of services
• Sector strategies respond to international commitments on gender equality, human rights, disability and environmental sustainability
Sector strategy has clear analytical basis which addresses gender, rights, disability and environmental sustainability
Sector strategy based on improved information systems, including e.g. disaggregated data around e.g. gender and disability
A • Substantial • Some • Little • None • Regression
MDGs
Data collection MOH and Stats division
Data on disability register @ MOH/Intaff.
ii. Increased alignment of aid with partner countries’ priorities, systems and procedures, help to strengthen capacities (in the
• Overall support to the sector based on national sector strategy and progress reviews
MS Ind 3
Shifts in key donor sector strategies / programmes / expenditure reflect changes in government priorities (e.g. new policies, strategies in the sector)
A, D • Substantial • Some • Little • None • Regression
HSV, pandemic response, - increasing,
127
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially
since 2005 Key reasons &
explanation
sector) • Increased use (in the sector) by
donors/ agencies (and all vertical funds32) of country systems and procedures
Diagnostic reviews on country systems / reforms undertaken in the sector
Rationale provided for non or limited use of country systems in the sector
MS Ind 6
MS Ind 5a
MS Ind 5b
A, D • Substantial • Some and
increasing • Little • None • Regression
CS active, not counted in central agency
HSV, Pandemic some regional funds yes.
• Increased support to capacity-building of country systems in the sector
MS Ind 4
Explicit objectives / strategies for capacity strengthening of partner systems in the sector within donor strategies and programmes
Volume / proportion of support to capacity-building of national sector systems
A, D • Substantial • Some • Little • None • Regression
? 2007 id need for workforce development. Plan drafted and looking for donor support
WHO scholarships
ODA low to sector, some % for CB
B Building more inclusive and effective partnerships for development
iv. Less duplication of efforts and rationalised, more cost-effective donor activities (in the sector)
• Increased use of donor comparative advantage/ complementarity led by government in the sector
Clear view [statement] by Govt on donor comparative advantage in the sector and how to achieve donor complementarity
Reprogrammed sectoral aid according to statement of comparative advantage
A, D • Substantial • Some • Little • None • Regression
Few donors, NZ indicate increase. Support primarily through regional facilities
Heavily aligned to NZ service providers and stds.
32 See Glossary (Extranet) for a listing of vertical funds.
128
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially
since 2005 Key reasons &
explanation
• Increased ‘division of labour’33 at sector level
Mapping process conducted / maintained
Number and theme/type of formal Division of Labour arrangements
Reprogrammed aid to the sector according to Division of Labour agreements / arrangements
Co-operative / joint work between agencies within e.g. project modalities
A, D • Substantial • Some • Little • None • Regression
No mapping conducted.
Coordination of UN agencies. SPC.
• Increased delegation to lead donors for the execution of programmes, activities and tasks at sector level
Increased use of donor lead arrangements in the sector [e.g. ‘silent partnerships’34]
D • Substantial • Some • Little • None • Regression
NZ/Aus contribute to regional funds, SPC delivered
• Reduced fragmentation in the sector
Number of programmes/ projects, transactions, contracts and funding arrangements compared to the total aid volume within the sector
A, B • Substantial • Some • Little • None • Regression
Issues with sub- sector and cross cutting with environment, internal affairs, and
• Increased untying of aid MS Ind 8
See also issues raised in Untying of Aid Report (2009)
A • Substantial • Some • Little • None • Regression
Strong tie to NZ for provision of specialists and accepted.
33 See Glossary (Extranet) for definition. 34 See Glossary (Extranet) for definition.
129
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially
since 2005 Key reasons &
explanation
v. Reformed and simplified donor policies and procedures, more collaborative behaviour (in the sector)
• Evidence of reforms and simplifications by individual donors at sector level
MS Ind 10 A, D • Substantial • Some • Little • None • Regression
HSV managed by MOH,
UNDP - UNDAF
• Common or harmonised arrangements amongst donors at sector level [for planning, funding, disbursement, monitoring, evaluating & reporting to govt on donor activities & aid flows]
Joint Assistance Strategies / plans (sector level)
Use of common procedures for pledged funds
Shared conditions for tranche funding
Joint monitoring/ evaluation/ reporting processes
MS Ind 9
A, D • Substantial • Some • Little • None • Regression
Work by SPC,
Little other than NZ/Aus under Harmonised programme
UNDAF joint annual review to include health
• Strengthened incentives for harmonisation, alignment and results orientation in the sector
Joint accountability frameworks for the sector featuring changed incentives
Supportive incentives in sector-level donor agency performance management frameworks
A, D • Substantial • Some • Little • None • Regression
WHO
130
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially
since 2005 Key reasons &
explanation
vi. More predictable and multi-year commitments on aid flows (to the sector). [Has the nature of conditionalities - within the sector - been changed to support ownership in line with the AAA commitment (para. 25)]
• Increase in proportion of aid being committed to the sector through multi-year frameworks
Number of donors setting out indicative commitments for the sector within multi-year frameworks and delivering these
Proportion in terms of (total volume and the number of) donors/ agencies providing indicative aid commitments to the sector 2010-2015; based on 3 year commitment, on 5 year commitment
A, B, C, E • Substantial • Some • Little • None • Regression
Increasing with UN agencies subject to fund raising outcomes into fund
NZ, Aus, Un agencies.
• More timely and predictable aid disbursements to the sector
Number / proportion of donors with clearly set out agreed disbursement schedules with government
MS Ind 7
Share / type of aid in the sector disbursed according to schedule
Proportion of government aid for the sector in line with budget
A, B, ,D • Substantial • Some • Little • None • Regression
Not lined up with local budget setting rounds.
GFs reliant on local strategy in place, and HSV donor programme implemented by
Project/PBA
• Limited set of mutually agreed conditions in the sector, jointly agreed, made public and jointly assessed
Number of mutually agreed conditions made public
Number of joint assessments
A • Substantial • Some • Little • None • Regression
SPC (on behalf of donors)
GF (on behalf)
131
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially
since 2005 Key reasons &
explanation
viii. Sufficient integration of global programmes and initiatives into partner countries’ broader development agenda (for the sector).
• Global programmes work to strengthen sector policy environment / institutions
Global programmes35 country implementation strategies reflect relevant sector strategy
Sector planning / monitoring frameworks incorporating global programmes
A, D • Substantial • Some • Little • None • Regression
GF support awareness/education of HIV TB Malaria, MDGs met already.
ix. Stronger partner countries’ capacities to plan, manage and implement results-driven strategies (for the sector).
• Results-oriented reporting and assessment frameworks in the sector for assessing the impact of development policies
MS Ind 11
MS Ind 4
Number of sector plans which set out clear linkages between expenditure and results over the medium term
Sector frameworks including manageable number of indicators / for which data sources are available
Availability within Government of regularised socio-economic data sets
Use of disaggregated data (gender, excluded group) within results and assessment frameworks
A, B, D • Substantial • Some • Little • None • Regression
Linked to NSDP and MDG indicators.
35 See Glossary (Extranet) for typology/ listing.
132
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially
since 2005 Key reasons &
explanation
• Donor programming and resources in the sector increasingly linked to results
Proportion of donor sector plans which specify links between expenditure and results
Proportion of donor sector results frameworks which reflect national results areas (including cross-cutting issues e.g. gender, exclusion, climate change, environment)
A, D. E HSV, UN programmes
C Delivering and accounting for development results
x. Enhanced respective accountability of countries and donors to citizens and parliaments (for the sector)
• Timely, transparent and comprehensive information on aid flows to the sector publicly available (donors)
Audit reports on use of aid in the sector
Publicly available donor annual reports on aid flows
A, D • Substantial • Some • Little • None • Regression
No audit on HSV
Audits on UN or SPC?
Internet based. Not accessible to all health stakeholder groups
xi. Less corruption and more transparency, strengthening public support and supporting effective resource mobilisation and allocation (for the sector).
• Greater transparency in public financial management
Records of disclosure of aid revenues, budgets, expenditures, procurement and audits in the sector
Internal and external audits reporting progress on financial management
A, D
• Substantial • Some • Little • None • Regression
07/08 accounts just completed. MOH finances improved with MOH review, and development of strategy
Audits. Reduced over expenditure.
133
PD expected outcomes Progress markers Potential indicators of
change / milestones
Methods / Forms of Analysis
Judgement on progress, especially
since 2005 Key reasons &
explanation
Conclusions on the effects of Paris Declaration implementation:
A. Report against the 3 components of Core Question 2 with respect to the sector / area:
• Improvements in the efficiency of aid delivery to the sector
Time delays with decision making and processing funds into the country
• Substantial • Some • Little • None • Regression
• Improvements in the management and use of aid in the sector
Overall MOH has improved its management, service delivery as well as Aid management. MOH increasing leadership, Health cross-cutting role with Infrastructure, environment and agriculture developing .
• Substantial • Some • Little • None • Regression
• Better (more inclusive and effective) partnerships in the sector
Relatively small group of donors and programmes mainly delivered through regional agencies, facilities. Level of ODA small compared to other sectors.
• Substantial • Some • Little • None • Regression
B. Have there been unintended consequences of the Paris Declaration for aid effectiveness in the sector (e.g. unintended impacts on particular groups including women and girls)?
C. Are there possible alternative ways of achieving more effective aid in the sector, e.g. in the experience with non-PD donors?
134
Appendix 10: Summary of interviews
Interviews Key Themes
Question 1 Conclusion.
Conclusion A
Fatigue –Yes
Donors show substantial evidence while others there is little evidence. See frustrations, government is up against the wall with donors and implementers. Government show some evidence and stakeholders also show some evidence of fatigue. A lot of meetings, a lot of talking, and the process is hard work for the NGOs; report requirements are hard for them, the complicatedness of forms. AMD don’t have much people or the capacity. We don’t seem to have clear direction, bit disappointed by NZ, and too much red tape.
There is fatigue from a key donors part – informant don’t have access to the discussions that donors tend to be having themselves however, I have noticed that some donor talk about using national systems but in reality have made no effort at all to modify their legislation to allow them to use national systems of another sovereign country.
In terms of NZAID there seems to be substantial evidence of donor fatigue as aid effectiveness becomes a secondary consideration to recent changes in Policy as NZ pursues their own policy objectives. Other donors such as EU are willing to investigate budget support as an option going forward thus showing little evidence of donor fatigue and that they are still pursuing aid effectiveness agenda.
Significant delays have been experienced implementing relatively simple programs related to cyclone recovery. Lack of trust underlies the current environment and suspicion, blame and defeatisms rather than openness, creativity and optimism dominate mindsets.
Donors insist of using their systems. Incomplete projects for example harbours of outer islands and infrastructure. Yes they promise but don’t deliver quite dictatorial of management of their funds.
Fatigue from key actors in government‐ most reforms required by key actors within government are generally reforms required to get the public service performing to a decent standard and not necessarily driven by the PDA. These reforms are substantial and by themselves create organisational fatigue. For example a current ADB TA on the Public Service has identified outputs that include 1) a functional analysis of government agencies completed and adopted ii) government agencies implement a coordinated planning, management and monitoring systems and iii) PSC enforces the government wide public administrative manual. The TA activity aims to reduce the size of the public sector and aligns its structure and scope with national priorities, advance performance management government‐wide, and adopt and enforce comprehensive public service regulations and policies. This alone is going to create massive fatigue. In comparison, fatigue created by the Paris declaration will be minuscule.
In dealing with donors on a day to day basis, informant experienced aid effectiveness fatigue on a weekly basis. Often the rhetoric of donors does not actually match performances. There seems to be a shift away from a programme approach to projects in our major donor NZAID and transaction costs are consuming greater resources. Donors seem to prioritise harmonisation and mutual accountability and pay lip service to country ownership and alignment to country systems. For us the priorities are reversed.
Central agencies take positions that concur with donors that existing donor allocations can’t be spent as absorptive capacity is the problem. Private Sector capacity needs to be built. This contradicts the lack of finance discourse held in public by the same agencies. This language reveals a defensive posture of we can’t do it, it didn’t work yesterday
135
rather than an innovative posture such as what else could we do, can we do it differently, who else could facilitate in becoming effective.
There is a weakness in reporting procedures and accountability.
Fatigue from other stakeholders part – most of other stakeholders are general unaware of the Paris declaration and its implications. In most cases, they just want assistance and they are not particularly concerned about what policy framework it is delivered under. Efforts to educate them on the Paris declaration in the past are often met with glazed eyes. The difficulty in showing improved aid effectiveness as a result of the Paris declaration also reduces its public marketability.
Line agencies are underfunded, NGOs and private sector can expand to meet demand for services needed but will only be put in place as capacity if resources are made available to fund operational development. Fatigue is evident in the “not another government project” mindset which highlights the lack of confidence and transparency in national planning and funding allocation processes. For example of activists, line agencies, private actors making calls for funding for innovative approaches for high priority areas such as waste management, environment, and renewable energy.
Most funds given are merely for community projects rather than economic stimulating projects, for example the businesses.
Fatigue – No
Informant reported that working with NZAID find it a piece of cake. Another said that we don’t appreciate what effectiveness is all about as a country.
Effect of Fatigue
Informants got to stage where they don’t apply for funds anymore. They don’t get approached by NZHC, they don’t look at funding and try to do it themselves using their own resources; they are left out of the loop. There is always frustration with outer islands infrastructure and ACCC and PCC without consultations. There has to be a body so NSDC was created, arose out of frustration, criteria template that NZAID used was so bureaucratic, people on the ground didn’t have the capacity to make it happen, should be a simplified form. OIDGF couldn’t comprehend what requirements were in it with officials and down the chain. There is frustration; one donor seems to be backing away from mutual accountability. Everyone is tired with NZAID, can’t even be bothered attending functions.
Most recipients of AID are so grateful for the funds, nothing much else matters, reporting is almost non‐existent.
Delays
Harbour in the outer islands, haven’t got our act together, little people are tired and can take years to get anything to happen. There are delays in decision making. Processes are long and drawn out as policies are being developed so it takes longer. There is perception that infrastructure projects are moving slow because of donor systems; it would be quicker if we used government systems. In terms of older donors versus China, there was a quicker turnaround time once signed up. Shift of policy by NZAP, shift to bilateral from regional and economic development focus, priorities shift not necessarily consulted with Cook Islands government. Back log in OIDGF applications, delay in NZ head office because of approving criteria, this creates frustration in implementing programs. Donors saying yes when it means “not yet” we are yet to be convinced. Projects delayed for 18 months so there is delay in procurement. UNESCO takes forever, takes two years and our needs had changed by the time funding arrived, importance of subject not there anymore, spent a lot of effort and get rejected. Delays related to local counterpart, problem is with AMD, to put in place systems, follow‐up, expecting everyone to run around. E.g. Manihiki CMC renovations have taken four years, so costs increase. Donor’s process can be too slow.
136
Question 1 Conclusion
Question 1 conclusion B).
Which factors had a substantial influence on the Implementation of aid effectiveness reforms in the country over the past 5 years?
• Changing political priorities • Governance reforms • Economic conditions • Changing relations with key donors • Natural and manmade disasters
How have these factors had an influence? • Changing priorities of political actors negatively affect governance improved.
• Push for more institutional capacity e.g. the 10 year old Penrhyn lab – result of natural and man‐made
disasters, challenged by pearl disease.
Energy and Waste efficient projects became difficult to manage.
They tried to keep it politically alienated since early 1990s.
Prioritise own needs e.g. not climate change but disease related issues
• Aid effectiveness must start at government level between Cabinet and Donors. This filters down to
respective ministries. Key decisions should be made with priorities of the type of aid that we need.
• The current status of government economically – result of changing political priorities, governance reforms,
economic conditions, and changing relations with key donors.
• NZ political change shift – knowing these projects can’t be financed other than by loans – don’t see level of
analysis – not proactive. Get appreciation of aid out there, disaster – timelines to respond to aid, the influence of China, NZHC wanting to find out more of what is going on – all a result of changing relations with key donors.
General Comments
All outer islands should be included in any discussions for the NSDP or else what each island has identified and made for its own strategic plan should be incorporated in the NSDP
There is a lack of commitment by aid agencies to speeding up the process of work on each project. At AMD there have been a lot of restrictions or “red tape” in what can or cannot be done. E.g. CMS project it has taken over four years to complete.
We should be given more information on how to implement projects from our side e.g. There has been a total reliance on foreign consultants to do this rather than using our people. We know our island more than outsiders.
137
Q1d).
1dii) Identification of relevant key decision‐makers.
Decision Making Government officials identified Government as the major decision makers about accepting and allocating aid. The majority of informants know the process that AMD office as the first point of contact, then to NSDC (previously ACCC) and then Cabinet who make the final decision. Outer islands informants revealed that decisions were always made from Rarotonga from either AMD, or the head office of the various government ministries. However, there are some respondents from civil society who have direct relationship with the donors and make direct decisions about accepting and allocating aid for their organisations.
Donor Decision making HQ vs Field
Comments noted that donors take their decision/s processes off‐ shore rather than in‐country. Distinctions are made between strategic and high level decisions vs operational decisions where authority is delegated to local field office (i.e.: NZHC) or implementing agencies or decision making boards such as CIS, GEF SGP and government departments like MOH, Education and MMR.
Comments noted that final approval comes from HQ or sub‐regional offices.
CIGov – from implementing agencies comments noted that it’s not always clear what each donor’s decision making processes are or the financial delegation levels.
Positive and Negative Effects on Aid Management/Coordination
The positive effects on aid management or coordination in country were the quickness of decision making with ADB and Chinese donors.
In terms of negative effects on aid management or coordination in country the informants felt there were delays on funding to be approved as the recipients part in not filling forms correctly, some have taken nine to twelve months, some two years before funding is approved, slow process, a lot of bureaucracy, projects take much longer to be approved, the loop holes, frustrations, decision making is frustrating more time spent on coordinating agendas, projects get delayed come to standstill, shortfall in proposals and the requirement to do some more work and causes delay, battles with AMD over what is their role, lack of communication about aid and the process, little information about what is available, needs to be clear channels of communication, how to access funds, develop templates with clear parameters and conditions of aid, not easy getting the information, relying on AMD.
Capacity Needs and Resources
An informant reported that aid paperwork is a nuisance and is something that needs to changed; don’t have the right people to deal with the paper work as donors are not going to fund/s without it. The lack of matching of donor expectations and our capacity to meet is difficult. There are a lack of project design skills, need capacity to deliver plans and projects. When there is a change of personnel it creates a lack of capacity of staff, lack of information from them. The NSDP is not defined enough; there is no foreign policy and aid coordination policy by staff of the Ministry of Foreign Affairs. The failures in the infrastructure area – we tend to rely and trust external expertise without thinking about HR capacities and with limited skills because local input not known or made known in that area. No project managers, lack of specialists in country.
138
Q1e) to what extent and where have the PD principles been implemented?
1e.i) Since 2005 how have the different PD principles been interpreted, weighted and implemented in the Cook Islands.
Paris Declaration Emergence in Cook Islands
The informants had different times of awareness of the Paris declaration emergence in the Cook Islands. • One mentioned 2006 which came out of a regional meeting on aid effectiveness and developed into Pacific
principles • Another said around 2007/08. • One informant was aware in July 2007 when the Pacific Aid Effectiveness Principles was developed by the
Pacific Island Countries and Donor partners in the region. At the July 2007 Forum meeting and the Cairns Compact meeting; they helped build consensus and promotion of the Paris declaration. They provided the impetus to accelerate application of the Paris declaration in the Pacific. The donor community has become more results oriented and called for comprehensive measures for greater accountability and transparency at the country level. These motivated the application of the Paris declaration at the country level.
• To another informant it was at a regional level.
Paris Declaration Influence in Decisions making
Informants considered the PD had no influence.
Where the PD was identified as having some influence it was useful in: • We can use a point of agreement when donors don’t want or seem to be listening, aid is still very big
business. • It can be used as a tool to influence donor decisions.
Internally within government it was noted that: • A little influence internally with government because only AMD know about it across government.
PD influence in relation to Donors: • Some donor plays lip service to the Paris declaration and filters through donor policy. • Some influence provides guidelines for aid recipients. Aid donors will decline offer if we do not follow these
guiding principles. • The aid donors have their own terms and conditions and they look at our situation and they see how their
funds can meet our needs and that is how they have been operating. • With some influence donors want to be seen on the ground, for NZ and Australia they want policy capture.
We are not accessible as we are tied to the NZ framework.
Principle with Most Prominent Influence
The informants response were that the effect of harmonisation has been one of convenience rather than true intention of the principle, harmonisation because this is seen as a major issue on aid effectiveness in the Cook Islands. One informant felt that harmonisation is donor focused and donors harmonise themselves, if you have alignment then you don’t need harmonisation.
Mutual accountability • The example of a Joint Country Strategy that took two years to develop and was dropped with a NZ policy
change the question was asked how is that mutually accountable?
139
Managing for results, • To be transparent.
Alignment • How to fit the aid into their paperwork alignment because they are attempting to stop duplication, the idea is
for plans and priorities so they can move from project driven versus outcome driven. • Use alignment to country systems, alignment by donor country as to what they can do and what we can do
on our own. Alignment is important as funds need to meet our goals and fit the funds to those priorities.
Ownership • It improved a lot in terms of accountability the emphasis here is on manageable outcomes on criteria for
access and funding. • There is a lot of rhetoric for example NZAID’s change of policy has an effect on the outer islands.
Ownership and harmonisation • This is as important because NZ and Australia have merged its aid programs and NZ is managing the
programme.
Ownership, managing for results and mutual accountability • because in addition to the work of the Aid Coordination, the donors have also raised awareness about the
Paris declaration, which call for monitoring and evaluation, emphasis on documentation and data integrity. Regular interactions with partners and training in‐country through programme specific or through UN interagency work through HACT, there is now traction on the Paris declaration agenda.
Principle with Least Influence
Harmonisation was noted as least prominent as a side issue to be dealt with.
Mutual Accountability • As donors doesn’t want to look into maintenance of equipment. • They chase you up for reports we are dependent on donors.
Alignment was also noted with least influence.
Alignment and Harmonisation • These are least prominent in discussion because these require some comprehensive changes that would
require discussions at a higher level with government representatives and or in the case of the UN with headquarters and thus significant changes are slow to materialise. These changes are important but do take time to apply at both the country and donor level.
We ask that political interference to be kept out of the implementation of projects. There have been political manipulations in government, especially as to who should get aid funding. This becomes very obvious at the time of change of government leadership.
The CIS , the NGO POBOC fund from Internal Affairs and the OIDGF. In order to access any funds for any of these three agencies, groups of people come in to see us as they have difficulty filling in the forms. So we assist them in filling in the forms and submit these requests to the various funding agencies. We get feedback from them to say: you have to be part of an organisation to ask for funding, no individual funding is accepted, no funding for infrastructure e.g. Housing.
We need to help in many areas – with women groups, church groups and other groups. There are three villages in Pukapuka, and they need assistance.
140
AMD is doing a good job, the tendering process creates extra work, turn‐key solution is a good way to utilize aid funding, more utilizing of local services, sometimes there is a lot of expertise in country.
AMD are too slow at processing funds and there appears to be a top down approach, we are now able to employ our local people as technical consultants on these various projects e.g. Ben Parakoti for water works, Ata Herman for road works, there are some capacity issues with different projects we need to be able to tap into technical assistance funding. The tendering process sometimes slows down the work. If we know we have the expertise we need to be able to use it.
There needs to be locally procure services where possible – this will help with unnecessary spending. There seems to be a lot of bureaucracy with the way some of the funding arrangements are administered. If we know there are local capable people who can do the work then use them. The tendering process that is currently being used may be unnecessary spending.
It is important for aid funding to develop national sustainable growth. Tourism as a large contributor to GDP needs to be supported by government – that may mean prioritising tourism so aid money can be accessed.
2A Conclusions on the effects of Paris Declaration Implementation
Efficiency of Aid delivery (2e.1) • Overall comments positive view of improvements being made
• Seems aid to date has been TA/reports, studies and boomerang aid(goes back to funding country for equipment and consultants) as well as delivery being measured in terms of expenditure rather than achievement of deliverables
• Aid delivery is most inefficient where there are no sector strategies to coordinate priorities and implementation across sectors of government. Where government had adopted better programme planning approaches and where sector policies have been in place first.
• Having NSDP in place had strengthened capacity of central agencies and implementing agencies and provides mechanism for donors to come to since 2007.
• Delays hamper delivery and are perceived as being donor created.
• Efficiency varies between regional and national driven priorities
• Improvements have to put it down to the people involved.
• If effectiveness based on spending then not effective.
Management and use of Aid (2e.2) • Overall positive view of improvements being made.
• Still need to build our capacity in terms of project management. Risks include one person has so many jobs, focus is split and delays are caused
• Support to CS hampered by lack of information shared by government
• Financial management systems, reporting has improved.
• Use of aid remains variable
• Use of approaches and schemes like SWAp, HSV schemes put in place they work.
• AMD more coordinated and in control
141
• Bulk funding to organisations in education sector and NGOs are good
• Need to bring management under one umbrella
• Quality of the people involved is important and led to improvements
• Need to include aid in ministry business plans
Better Partnerships (2e.3) • Positive view of partnerships overall,
• Not where decisions are being made between donors and without the involvement of the country.
• ADB much improved from prior to 2005
• Partnerships improved and include the use of the private sector
Unintended effects • We are far more confident and assertive with the donor in maintaining focus at sector level and with high
levels of ownership
Different ways • donor and agencies engage in dialogue directly
• outcome driven approach is better
• China starts from the assumption that the Cook Islands operates with a democratically elected government and we know best the needs of our country. They don’t push governance issues. Assume that key responsibility retained by CIs and it’s our responsibility to maintain and operate facilities
• India use of “south‐south” cooperation and have more understanding of developing context vs traditional donors
2a. Country ownership over development
2a.i) Stronger national strategies and operational frameworks ‐Cook Islands lead in aid coordination at all levels with donors
A: Perceptions of Government leadership in aid coordination. • ‘Overall obtaining ownership is quite a challenge’.
• Government is active in taking the lead with NSDP offering an opportunity to lead discussions. Also includes Country strategies based on the NSDP.
• Government not community development focused
• Limited support to Civil Society
• Donors are viewed as advocates and picking up CS needs – e.g.: ADB funded proposal to cost CSO service delivery as a decision making tool to establish minimum service delivery standards.
• Government still to show leadership by calling round table of donors on NSDP and increase discussions
B: CIGov leadership constrained by: • Capacity issues ‐There is a lack of brains, officials not knowledgeable enough at sector, technical or aid
effectiveness. Lack of policy direction in aid effectiveness, lack of strategic planning, structure and policy, coordination is adhoc and reactive, need to confirm what is AMDs role, project preparation capacity needed,
142
capacity built up within ministries, needs to be institutionalised. M&E and reporting needs, need to look at doing things differently, high volume of transactions, AMD too busy implementing and not screening or helping line ministries process their applications so proposals are not meeting requirements.
• Some coordination of donors is poorly done.
• Gender equality agenda – not done in a planned way – taking a ‘go shopping approach’
• Need for increased support from donor to administer their programmes in‐country (not just increase the volume of aid to the country), ‐if donor can deliver – let them,
• Political influences – internal influence between officials and parliamentarians and Cabinet ministers. o With ministers working in isolation of their government and party. o Ministers and ministries’ picking and choosing priorities to present to donors comes at a cost when it
shows not clear agreed list by govt overall.
• External influence – of donor’s interests asserted in the bilateral influence on Cook Islands, donors influence with each other, and other donor influence on bilateral relations with Cook Islands.
• Mismatch between identified priorities and assistance
• Competing policy positions between CIs and donors. ‐ Need to say no to aid, look at reducing aid.
• Limited approach to leadership – government seen as ‘tagging aid to needs’ rather than leading’ ‘should be out there marking the NSDP not waiting for them” aid not matching and need to ask the question is it relevant? This is seen particularly with regional programmes which over support that is perceived not to be relevant – e.g.: the Dugong, TB
C: Perceptions of donor responsiveness to government leadership: • Different strokes for different donors – they have a different response to the government leadership.
o They are a mixed bag and each comes listening with their own agenda. o And need for correlation between donor responsiveness and importance of the importance of the
Cook Islands to the particular donors foreign policy. – o China quicker to decide and deliver but have to be through their systems, o NZ longer to decide, had major policy shifts and less responsive which not clearly articulated to
partners and seen as being pushed aggressively by NZ as the lead donor o EU is looking at high level aid modalities.
• The trust relationship is different from ‘confidence’ in our leadership... there are more risk management
issues
• Cost of doing business less responsive – e.g.: doing business in foreign currency vs in NZ dollars ends up costing CIGov more.
• There are times when donor has been able to improve local decision making by providing an alternative that is better than our planning processes. E.g. devolution of management to govt and longer term contracts/funding arrangements.
• Donors dictate where funding should go and then designed to fit funders scope, they also continually demand improvements in the systems without offering incentives or rewards for the steps taken.
“Needs to be a day when we can say we can do it ourselves”.
“We need to get away from the aid mentality.”
143
Fully consultative processes including civil society organisations and those representing women and excluded groups, in NSDP development
Views on consultation of NSDP include:
Benefits • Should be a key planning tool for central agencies and line ministries
• Can guide options and strategies for achieving objectives more effectively ‐ such as user pays rather than tax payers meeting the cost of achieving goals
• Is a focal point and guides donors as to where resources are needed
• Enables donors to meet their own PD principles – e.g. UNFPA – reached consensus at a regional level, since 2008, more effective for them to engage at the country level.
• Improved process from first NSDP with recent sector for providing input and continuity to the NSDP process.
• Donors have shifted to base priorities on NSDP. 2008 PICs requested reducing reporting intervals from quarterly to six monthly for those with good track record in order to reduce reporting burden.
Limitations of the consultations include: • Lack of involvement in the decision making process – so while many recall being involved in discussions these
were not considered to be meaningful i.e. not involved in developing the elements of the plan.
• Understated involvement of such as CSOs and outer island authorities who seek to work in partnership with the government and they question the value of their involvement in the consultations. They are not included in the allocation of resources to meet the identified priorities.
• The first plan was considered too broad, generalised and ambitious. So while many considered the process to be extensive, the difficulty in trying to extract national priorities from wide ranging stakeholders’ views is apparent with stakeholders (particularly CSO and outer islands authorities) not seeing their views or priorities reflected in the final product.
• Difficulty in managing stakeholder expectations that if they are consulted they should also be involved in the decision making.
• NSDP is a starting point, a work in progress, and should be kept at the forefront of National development
• Inability of the NSDP to be flexible to meet the changing priorities of outer islands due to external influences, policy, etc.
• Not linked to the MTBF. A useful planning document but not linked to the budget process.
• The process identifies the capacity limitations of central agency staff to produce drafts and policies. This then highlights the need for more resources to central agencies.
• Need to ensure sufficient analysis of demographic trends and potential gaps are identified.
• NSDC lack of perceived accountability back to constituencies in society
• Lack of transparency in decision making undermines ability to maintain momentum of implementation. So where each island has identified its own development plan its path into national planning and budget process is perceived to be lost.
“A plan without a budget is just daydreaming”
144
2a. Country ownership over development
2a.ii) Increased alignment of aid with partner countries priorities, systems and procedures, help to strengthen capacities
Overall support based on Cook Islands NSDP, specific related priorities of sectoral ministries and NSDP sector progress reviews.
• Conditionality’s developed in consultation with CI and donors
• Are dependent on the donor e.g.: NZ policy change excludes outer islands from direct support without consultation with the CIGov and Australia as development partners. The result leaving an unanticipated gap in the outer islands.
• Donor development of their country strategies include some form of consultation but varies in how widely this is conducted and for what purpose
Capacity building support by donors • A range of national and sector initiatives undertaken in Health and Infrastructure as well as with NSDP e.g.:
UNFPA in‐country training on programme management. Readiness assessment for developing M&E framework for the NSDP, Data management tool for Aid management. Also Education Sector Capacity assessment funded by NZAID/AusAID, UNESCO support for M&E framework for Education Sector plan.
• Capacity building is being neglected as so much work needing to be done with limited resources. Donor has indicated more of their resources are needed for their own systems due to increased aid programme and appear unwilling to invest in aid management capacity.
• Use of overseas TAs not always perceived as building capacity
• In aid management, donors took initiatives such as placing donor field staff in govt agency. Further support such as this with projects including funds to employ project manager is seen as being beneficial to improving implementation.
• Expectation that a small economy like the Cook Islands is expected to absorb increased aid flows with no additional external support increasing pressure on already underspent programmes.
Use of local systems by donors • Varies from donors to donor – “some are flexible and reporting obligations are relative reasonable (China,
India) others have very onerous requirements and insist on utilising their own systems to deliver aid which can be cumbersome and incompatible with local realities.” Are a nightmare to deal with, extremely high overheads and administrative costs. “Reform seems a distant dot in the future”
• Perceived donor lack of trust of govt systems seen as undermining significant increases in use of partner systems. Donors are seen as being locked into a cycle where poor public performance leads to increased requirements for reporting, more stringent contractual demands, more monitoring and controls, less use of local systems, less resources for innovation and leads to low morale, low performance and again lowers donor confidence.
• Perception that donors don’t trust local systems and people and therefore no real increase in use of those systems.
145
2a. Country ownership over development
2a.iii) Defined measures and standards of performance and accountability of public financial management, procurement, fiduciary standards and environmental assessments, in line with broadly accepted good practices and their quick and widespread application.
A: Strengthened financial management
Lack of improvements • Some wouldn’t see a difference like in CS where they don’t have the capacity to manage funds, make poor
decisions and are open to abuse
• Recent audit investigations interpreted as poor management and government has little control over funds
• Audit of govt and crown accounts still out of date
• Improvements needed in budget bidding, preparation and establishment of MTBF.
• AMD unqualified audits – comes to the attention of Audit NZ and negative rating for donor/MFAT and issue to manage.
• Don’t take time to do the M&E. Reviews offer opp to identify lessons and learn from them
• Improvements in financial management process and at operational level
Operational issues • Policy is in place and controls are tight – compliance by ministries’ the issue
• Issue of depreciation a stumbling block???
• Need for improved aid information on flows and sources
• Budget process not seen the same – linked to the NSDC and NSDP process.
• As part of developing medium term BF, need to continue to improve aid information into budget process as issue linked to quality of information available by donors/AMD during this process isn’t reliable
• Budget process shows lack of decisiveness about priorities and limited ability to judge value for money.
• Looking at functional review of PSC and builds on PSC act.
• PERCA act in place
• Extent to which donors are bought into strengthening govt systems not clear
• Decisions taken by cabinet are not linked to MTBF. E.g. when cabinet approved the EMP but not the costings of the EMP. This is what needs to be linked to the budget process. Therefore cabinet can make decisions and give assurances to minutes that decisions made with funding implications in mind
• No plans for PEFA, but welcome opportunity for external audit and review.
• Double posting create increased reconciliation
• Levels of thresholds when purchasing material needs consideration along with the way exemption rule is being applied as the norm rather than exception.
• Issue with costings for tenders and actual tender prices that come in.
Capacity strengths • Central agency systems are improving with OPM working on costing 2011‐15 NSDP, moving towards MTBF
and audit providing improved oversight of performance.
• also have own financial investigation unit and clean audits,
• have qualified staff on hand who can assist and advice on procedures
146
• increased knowledge about FMPPM
• AMD systems improving and provide option for line ministries to direct all external funding through them to reduce constraints on their capacity to manage and comply
• Lack of understanding of software, late completion of financial statements, inaccuracies e.g. with payroll, but looking to new approaches such as using the internet and central management of accounts (recentralisation an option).
• 70‐80% improvement of quality of work by ministries. Others don’t see the value of making the effort.
• MFEm have staffing policy of graduates.
Capacity issues • AMD needs to increase qualified capacity
• problems arise when it can’t be completed on time
• need to strengthen forecasting and reporting to avoid stock shortages at OI level
• outer islands staffing capacity poor and PS not having the right staff to comply with general accounting practices, so working closely with some reporting ‐ weekly, monthly, six monthly and annually
Operational gains • Increased accountability within each ministry
• Increased Donors level of trust
• Improved systems, planning
• Audit investigations undertaken seen positively as complaints will be followed up
• Donors follow procurement system
• Donor funded activities have lead to improvements in capacity through support such as PSTAF
• Improved from cash basis to full accrual accounting
• CIGov have endorsed a rationale promoting fiscal discipline around public debt levels and public service employment
• Commonwealth Pacific governance facility identified our procurement procedure weaknesses for us to act on
2b. Building more inclusive and effective partnerships for development 2a.iv) less duplication of efforts and rationalised, more cost‐effective donor activities
Government steering donors to their strengths is limited • No donor roundtables planned or undertaken systematically
• Limited lead taken by AMD to call stakeholders together to discuss priorities and funding opportunities at national, sector or agency levels prior to government engaging with donors formally.
• Donors engage in Donor to donor talks that exclude country partner
• Regional opportunity to steer donors as PIC member. o Pacific Regional Infrastructure Facility an example of one set of rules, conditions, grants, loans and
contact point that harmonises countries and partners
• Capacity has improved with better coordination coming from some sectors such as environment, infrastructure and education
• Better coordination coming from central agencies such as AMD, MFEM and OPM
147
• AMD takes steps to match donors to priorities as well as different modalities o E.g. towards budget support o E.g. SPC, NZAID/AusAID has its JCS; ADB has 2010‐12 Pacific ops plan, EU focused on energy. UNDP
and the UNDAF o This also includes steering donors away from areas such as NZ away from infrastructure o NZ “steered ourselves towards waste management. No one asked us to do that”
Examples of donors channelling resources via other donors • Few examples with silent partnerships limited to UN global funds
• NZ and Australia have harmonised bilateral programme with Australia not involved in strategic bilateral activities
• Regional scholarships
• The Pride project in education
• FAO used NZAID/CIGov OIDGF for funding support
• UN trying to deliver as one
• Regional examples with RRRT, SOPAC and SPBEA coming under SPC umbrella
2b. Building more inclusive and effective partnerships for development 2a.v) Reformed and simplified donor policies and procedures, more collaborative behaviour
Joint Analytical work • Lack of progress with what’s perceived by the CI as third party interference.
• Donors agree on the generalities but not the specifics
• In general donors do their own
• Some examples include: o NZ/Aus joint country strategy developed in 2006‐2007 o UNDAF done with other UN agencies and Joint mission planned for 2011 o NZAID and GEF‐ SGP assessment of GEF – Small grants programme o Pride evaluation o PPTA
Policy Reforms by Donors • EU planning on shifting to budget support
• Refer to DA02 response to 2a.5 for example
2b. Building more inclusive and effective partnerships for development
2b.vi) More predictable and multi‐year commitments on aid flows to committed partners counties
Indicative allocation process • Indicative budgets data collected by AMD during budget process are less reliable
148
• Timeframes for donors confirming allocation not timely with government budgeting process and timeframes
• Not much progress as focus on annual programmes vs multi year. Perceived as a mixed bag.
• Certainty of funding not guaranteed
Confirmed budgets for implementation • Process of getting from indicative to contracted budget difficult e.g. UNDP fundraise to add to their core
funds. So actual allocation can vary.
• Aid flow unpredictable due to delays in receiving funds after initial approval e.g. GEF – SG
• Previously had multi‐year with NZ/Aus for programmes but then with change of government moved to year by year renewals, making planning difficult
UNFPA have taken concerted efforts to have detailed annual work plans and give indicative budgets
2b. Building more inclusive and effective partnerships for development
2b.vii) Sufficient delegation of authority to donors’ field staff and adequate attention to incentives for effective partnerships between donors and partner countries
Continuity • High staff turnover with regional programmes
• Increase local staff turnover within bilateral programme, if local staff are retained then get continuity
• Dealing with different staff and building relationships ongoing
• Moving backwards with NZAID
Adapting resources and skills of field staff • NZAID developed local contractor scheme and then moved to minor contracting scheme, this enabled
development of local contractors to undertake contract work without having to recruit from overseas.
• Creating team approach to contractor enabling skill transfer and increase knowledge and skill of local contractor capacity
• Familiarity with the aid environment but not local context and ways of doing business
Technical staff working for more than one donor
• Not on harmonised projects/activities but some examples of contractors developing enough experience and
then able to work for other donors in‐country and the region.
149
2b. Building more inclusive and effective partnerships for development
2b.viii) sufficient integration of global programmes and initiatives into partner countries broader development agendas
Some strengthening with use of local coordinator – no clear links identified
2b. Building more inclusive and effective partnerships for development
2b.ix) stronger Cook Islands capacities to plan, manage, and implement results‐driven national strategies
Limited Improvements of government monitoring frameworks • Improvements at the sector level – e.g. MOH, Education and national level such as MDG & NSDP monitoring.
• CIS/AMD is monitoring more
• Government is perceived as trying to focus on results with OPM as part of NSDP having collected some baseline data and trying existing data up to the outcome level.
• Gap connecting national and sector level data
• Agency level data is the job of PSC.
• Since 2008, seeing government templates for monitoring against policy templates. This includes changes in PSC reports, templates and more responsive advice provided. ‘MFEM say NO report then no bulk funding’
• Limited absorptive capacity for M&E. In 2008 $1M given from AusAID. By the time it arrived, in the October, we couldn’t spend the money so not successful.
• NSDP at a stage of 5 year review of national plan and associated sectoral plans, lessons becoming apparent in relation to establishing clear more measurable indicators
• Seems to be pressure to measure regional indicators and measure of the Pacific plan for e.g. rather than strengthening national ones.
• There seems to be a focus on the governance principle interpreted as ‘does the county meet our (donor) view of what the governance system should be rather than the effectiveness of implementation’
Donor support to strengthen CI own monitoring system • Donors systems perceived as being focused on results and outcomes
• Support from donors include o SPC with statistical data and a updates o UNESCO M&E support of EMP – includes partnership with SPBEA and UNICEF o CIMRIS, WHO and UNFPA all assist with collecting and analysing data
Government greater focus on results • Moves being made through business plans being outcomes driven
• Planning towards M&E being driven at national level – but there were issues of capacity
• Capacity issue with personnel fully understanding this across all sectors
Donors working together to support government monitoring of national priorities • Cyclone recovery package for 2010 Aitutaki shows donors combined support
• GEF PACC project linkages to the climate change country team
150
2C. Delivering and Accounting for development Results 2c.x) Enhanced respective accountability of countries and donors to citizens and parliaments Change in trust, mutual respect, open dialogue and flexibility between donors and government
• Positive changes for the government in having the NSDP, sector plans and clarity about priorities gives greater confidence to the country in making requests of donors
• Relationship between donor and CI is important with both being accountable
• Mixed degree of changes depending on donor o NZ less dialogue before, delays with programme talks, performance reporting, decisions & implementation o Increased dialogue with ADB, UN o EU more dialogue and use of country systems. o Less engagement with China
• Difficulties come with maintaining relationships with donors who aren’t present
• Mixed messages with donor HQ/ high level views not reflected on the ground in field staff. There is more openness on the ground with staff the public were well aware
• Changes made but still a long way to go
• Donors also seek out CI view of other donors and seem to exclude others E.g. China left out
• Over time the dialogue has changed. In 2005 conversations were easier and level of trust less. Now conversations much harder and more trust needed.
• Clarification of the nature of the relationship needed trade based vs aid based. CI imports over $100m in goods from NZ
Government accountability changes to citizens and parliament • Improvement includes increase in auditing, media reporting (education and environment report cards), public
feedback and consultation processes through review and development of next NSDP, recent sector forum e.g. infrastructure, environment,
• No examples of parliamentary debate of aid effectiveness identified
• Need to understand that donors and countries are equally accountable for the way aid is used.
• Room for improvement with one central body as well as implementing agencies and recipients have some role to play.
Information on aid flows and use of aid in budgets accessible/available • Overall need for improvement as it is important to let people know about what support is available, how to
access it and be kept informed on implementation progress and results
• While Aid flow are published in donor annual reports, websites and CIGov budget policy statement, more information needs to be made available to stakeholders
• Need for greater transparency on Aid related activities to the public and better use of the media to do this
• Donor who have no presence on the ground need to create presence through better communication of their work on the ground
• AMD role to inform stakeholders along with Umbrella organisations in Civil Society
• Factors that limit dissemination and coordination of information includes: o Key agencies under resourced for this such as AMD, CIANGO, line ministries o Agencies lack the skills needed such as working with the media, networking and consultation skills
151
2C. Delivering and accounting for development results
2c.xi) Less corruption and more transparency, strengthen public support and supporting effective resource mobilisation and allocation
Measure to tackle corruption
• In general and not just specifically to aid, there is a reasonably efficient system in place from the Audit office that has raised the issues in the past and these have been investigated by police and taken to court. This included 2003 conviction of public servant of not following PFPPM.
• MFEM continue to improve systems and includes internal audit system to be carried out during a financial year, while staff is on hand to address any issues that arise. This will give a level of confidence for when audit office come to do full audit at end of FY. Given two years behind in ministry audits it reduces the risk of dealing with issues within a year rather than potentially a two – three years worth of issues.
• Measures also include official information act, National anti corruption committee, participation in regional body and seeking membership to international anti corruption body, PSC act, MFEM act in place,
• More public awareness created from high profile; audit reports made public and more complaints made to authorities
• Cook islands off the off‐shore banking financial black list, large numbers of unqualified audits of aid projects, KPMG/Standards and Poors ratings
• Internal controls have tightened and follow through happening to address weaknesses.
• Other crown agencies play an important role and include: o FIU o Ombudsman office o Crown Law o Audit office
Limitations
• No anti corruption policy in place
• Credibility of audit office, police and court system called into question when cases taken to court fail o Need clarification of what is corruption and what level of corruption exist in here? e.g. low level
where people get jobs that aren’t advertised, o some cases of fraud in community project level (have been investigated) o kickbacks and bribes
• Issue of needing to manage self interest vs natural good where individuals are looking for what is in it for them, cover up turn a blind eye and don’t share information
• Need for leadership code of conduct amongst officials and politicians
• Need for changing attitude of ‘this is the way we have done things’ and resistant to change amongst ministries
• Audit office working outside its scope with too broad an interpretation of its role.
• Capacity of CSO to govern and manage aid funding directly needs strengthening.
• Need to resource anti‐corruption team to operate effectively
• Issues to address with qualified audits include:
152
o Fixed assets, inventory, not able to produce OI supporting documentation, poor record keeping, lack of understanding of financial statements
Rising/Diminishing levels • Corruption is not a major issue and generally perceived that measures are being taken to deal with it.
Role of aid in rising/diminishing level • Perceptions that aid has no role and that donor controls are a lot tighter vs aid is an important role. Like
government, then need to keep vigilant
• Aid can contribute to the corruption related practices if aid is not well managed and if mechanisms for accountability and transparency are not in place
Q3. 3a) Were results in specific sectors enhanced through the application of the PD principles? Donor support for innovative approaches
• Overall there is little evidence
• MMR and AusAID fisheries incentives
• Health sector : o Consolidating programmes and integrating into core MOH public health care services such as the
consolidation of the Adolescent Health development into the reproductive health programme. This helps to bring in youth perspective, participation and engagement to mainstream review and planning at national level
o Climate change agenda – Food security and MOA and MOH working on integrated work plans to address issues raised by Climate change.
o ADRA, MOA and TGA collaboration o DAT project linked to National Disability strategy
• Infrastructure: o China involvement o EU community based sanitation o ADB and PIMP
Aid helping leverage Cook Islands policy and programmes to achieve national objectives • MDGs and pacific plan used to leverage change
3b) Did the implementation of the PD help Cook Islands to improve the prioritisation of the needs of the poorest people, including women and girls
Alienated or disenfranchised groups of society • Outer islands isolated and lower standards of living, facing challenges of limited transportation, economic
opportunities, communication, cost of shipping, airfares, imported goods and foods, have few convertible assets to build livelihood like Rarotonga leasehold land
• People with disabilities – despite receiving a degree of support from donors including NZAID with the DAT project, they have ongoing needs, so sustainability is the issue to address the needs of PWD
o children with disability excluded and need access to better services
153
• Limited opportunities for small business development on Rarotonga and outer islands leading to family to choose to migrate.
• women and children continue to be vulnerable o Women remain on the edge of political system which marginalises their ability to influence decision
making about resources and allocation of resources. o Access to quality housing for outer islands families in Rarotonga
• Young men not well catered for in education system
• Semi‐unskilled foreign workers emerging as group with needs that need to be addressed in employment conditions, resident entitlements and addressing workforce shortage and migration issues.
• The Elderly, mental health consumers needs access to services and resources for care and treatment
• Gay population disenfranchised
• Those least likely to benefit are those outside of the cash economy or whose economic contribution is not counted in the formal economy
Productive policy dialogue • Up until 2008 NZ were a key support to outer islands development. There has been no dialogue regarding
their policy change
• Other donors policy not outer islands focused
• Dialogue needed on capacity development for excluded groups
• Dialogue on implications of decision making that requires taking loans to provide services/infrastructure with the cost of these passed onto or excluding groups such as the elderly, disabled, mentally ill, youth etc.
• Sports are a sector which has the potential to provide benefits to the country economy and development but unsupported by government and only becoming a discussion topic amongst some donors.
• Regional discussion e.g.: RH symposium in July initiated attention on gap in RH services and information between Rarotonga and outer islands.
• Dialogue yes .... Capacity to implement ‐ No.
• When donors engage in dialogue with outer islands, local community despondent when little results from visit or decisions are made without including the community in the discussions or decisions
PD style aid contribution to promoting social inclusion and gender equality • Working with donors who have gender equality built into approaches and activity cycles, appraisal, reporting
etc. Need for consistent data cross ministries’ and access to the information.
• Donors have support review of national women’s policy and recommended a national gender policy – not necessarily linked to the PD
• There are activities that target
• UN agencies joint programmes and missions for gender related work
3c) Has PD implementation led to sustainable increases in institutional capacities and social capital at all levels to respond to development challenges.
Increased institutional capacity
• Progress considered slow and little
154
3d) How and why has the mix of modalities evolved and what has been learnt on the development
results?
Mix of modalities to progress NSDP • More programme/project orientated because of mismanagement in previous years i.e. 1996 economic crisis
• Sector based support in education sector initiated and much better move away from discrete project driven aid.
• Activities that are outcomes based and linked to plan with targets
• PRIF
• SWAp
Effect on Development results • Bureaucracy has increased and results delayed
• SWAp allows for focus on longer term outcomes for sectors
• Allows for longer term focus and flexibility
• Programme approaches are flawed for operating in small country economies taking too long to get off the ground.
• Projects are much more responsive and CIS is an example of this.
155
Appendix 11: Summary of survey results
Paris Declaration Evaluation – Cook Islands
Survey Questionnaire
Introduction The Cook Islands is an international partner to the Paris Declaration on Aid Effectiveness. This survey is part of an Evaluation to examine how the Paris Declaration has been applied in the Cook Islands. The focus of the survey questionnaire is to gather in your views, ideas, knowledge, experience on aid effectiveness and development results, including poverty reduction in the Cook Islands.
The information you provide in this survey is confidential. It will be reported in a way which will not identify any organisation or person.
Instructions 1. This survey should be completed by you
2. Please answer honestly and accurately.
3. Please return your completed survey to the survey facilitator as soon as you have completed it
4. It should take you 10 to 15 minutes
5. For assistance contact your Survey facilitator (Liz Ponga ph: 79786 , Nooroa Short Ph: 79782 or Tina
Newport ph: 79785)
Paris Declarations has Five Principles of Aid Effectiveness: Ownership Developing countries set their own strategies for development, improve their
institutions and tackle corruption. Alignment Donor Countries bring their support in line with these objectives and use local
systems. Harmonisation Donor countries co‐ordinate their action, simplify procedures and share information to avoid duplication. Managing for results Developing countries and donors focus on producing – and measuring – results. Mutual accountability Donor and developing country partners are accountable for development results.
156
A: General Questions A1. Do you know what the Paris Declaration Agreement is about? (Please tick one) Full knowledge Some knowledge A little knowledge No knowledge
A2 Do you know about Aid effectiveness in the Cook Islands? (Please tick one) Full knowledge Some knowledge A little knowledge No knowledge
A3 What involvement do you have with the Aid effectiveness Agenda? (Please tick one or more of the following boxes)
National/sector/agency planning Aid Management & coordination Policy development Project Programme Implementation Monitoring & Evaluation Financial Management Other Please specify for “other”:________________________
157
B: Important factors that have affected the relevance and implementation of the Paris Declaration and its potential effects on aid effectiveness and development results
B1. Who makes the major decisions about accepting and allocating Aid on behalf of the country? (Please tick one or more of the following boxes)
Government Politicians Traditional leaders Donors NGO leaders Civil Society Other Please specify for “other”:________________________
158
B2 Who are the most influential donors in the Country?
(Please list the Top Three with numbers 1, 2 and 3 in terms of ranking) Asian Development Bank ____ Australian Aid ____ China ____ Donor/NGO ____ European Union ____ UN Agencies (WHO, UNESCO, UNFPA, etc) ____ Other ____ Please specify for “other”:________________________ “Straight input”
“Equal weighting per input”
159
“Weighted Input (more accurate”)
“Top Three (Weighted Average)”
B3 What influence does the Paris Declaration have with aid effectiveness discussion in the Cook
Islands? Substantial Some a little No Moving Influence Influence Influence Change Backwards
160
B4 How important are the Paris Declaration principles in aid effectiveness discussions and
implementation? Ownership: (The Cook Islands set their own strategies for development, improve their institutions and
tackle corruption). Very Some a little No Important Importance Importance Importance
161
Alignment: (Donor Countries bring their support in line with these objectives and use local systems).
Very Some a little No Important Importance Importance Importance
Harmonisation: (Donor countries co‐ordinate their action, simplify procedures and share information to
avoid duplication).
Very Some a little No Important Importance Importance Importance
162
Managing for results: (Developing countries and donors focus on producing and measuring results).
Very Some a little No Important Importance Importance Importance
Mutual accountability: (Donor and developing country partners are accountable for development)
Very Some a little No Important Importance Importance Importance
163
B5 Is there evidence of ‘aid effectiveness fatigue’ (i.e. finding the processes of reform heavy and slow
and questioning the value of results achieved) on the part of:
a. Key donors Substantial Some a little No evidence evidence evidence Change
b. Key actors in government Substantial Some a little No evidence evidence evidence Change
164
c. Other stakeholders
Substantial Some a little No evidence evidence evidence Change
C: To what extent and how has the implementation of the Paris Declaration led to an improvement in the efficiency of aid delivery, the management and use of aid and better partnerships?
Ownership and Alignment
C1 In general, would you say that the Government leads in Aid Co‐ordination with Donors? Fully Mainly Partly Not Moving
Leading Leading Leading Leading Backwards
165
C2 How well do you think Donors as a group are responsive to Government Leadership in Aid Co‐ordination?
Fully Mainly Partly Not Moving Responsive Responsive Responsive at all Backwards
C3 How would you rate the development of the National Sustainable Development Plan (NSDP) in terms of being consultative with stakeholders in the Cook Islands? Fully Mainly Partly No at all Moving Consultative Consultative Consultative Consultative Backwards
166
C4 Do you see an improvement in Donors co‐ordinating their aid programmes with Cook Islands Priorities? Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards
C5 To what extent have conditions on aid been developed transparently and in consultation with CI government and donors?
Fully transparent Mainly transparent Partly transparent No at all transparent Moving Consultative Consultative Consultative Consultative Backwards
167
C6 In the last two years, have you seen strengthened capacity building support by donors of Cook Islands systems? Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards
C7 Have you seen improvements in the Cook Islands Local systems i.e. Public finance, local purchasing/supplies of goods and services. Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards
168
C8 Are Donors using these local systems more to deliver aid? Substantially Some a little No Moving more more more more backwards
Harmonisation C9 To what extent is the government steering donors more to use their particular strengths in specific
areas? Substantial more Some more a little more No Moving Steering Steering steering Change Backwards
169
C10 Have you seen donors adapting the resources and skills of their field staff to the new ways of managing aid? Sufficiently Partly Insufficiently Moving Adapting Adapting Adapting Backwards
Managing for Results C11 Have you seen an improvement in the way the Cook Islands Government monitors progress towards
achieving its national priorities? Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards
170
C 12 Are donors working to strengthen and use the Cook Islands own monitoring frameworks? Fully Mainly Partly Not Moving
Working Working Working at all Backwards
C13 Have you seen Donors working together with the Cook Islands Government to improve systems to
achieve better results with aid programmes? Fully Mainly Partly Not Moving
Working Working Working at all Backwards
171
Mutual Accountability C14 Have you seen changes in trust, mutual respect, open dialogues and flexibility between Donors and
the Cook Islands Government? Substantial Partial a Little No Moving
Increase Increase Increase Change Backwards
C15 Have there been changes in the Government accountability towards citizens and parliament? (e.g. parliamentary debate on aid effectiveness, increase dialogue with Civil Society)
Substantial Partial a Little No Moving Increase Increase Increase Change Backwards
172
C16 Are the Government and Donors taking increased measures to tackle corruption? Substantial Partial a Little No Moving
Increase Increase Increase Change Backwards Measures measures measures
C17 Overall have there been improvements in the efficiency of aid delivery over the past five years?
Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards
173
C18 Overall have there been improvements in the management and use of aid over the past five years?
Substantial Some a little No Moving Improvement Improvement Improvement Change Backwards
D: Has the implementation of the Paris Declaration strengthened the contribution of aid to sustainable
development results? How? D1 Do you think there are groups who are excluded from mainstream economic and social development in
the Cook Islands? Yes No Do not know
174
D2 Are there productive policy discussions between Cook Islands Government and Donors to include these groups in the mainstream economic and social development? Yes No Do not know
E: Finally, some questions about you (tick)
E1: Gender Male Female
E2: Your current position: ____________________________
175
E3: Tick one of the following:
Government central agency
Government line ministry/department
Government owned enterprise
Non Government organisation
Donor Agency
Other:
Please specify for ‘other’: ______________________
176
Appendix 12: Summary of six project profiles
Summary of Health and Infrastructure Sectors Initiatives
Official Title Donor Budget (NZ) Implementatn. Agency
Total Budget (NZ) ODA Type PBA PIU Type Tied
Aid
Term
Outcome data
Multi-functional indoor sports stadium, repairs & Upgrade of existing sports facilities & amenities and Ministry of Education complex.
China 13,000,000 CIIC 13,000,000 Concessional Loan
No Integrated Yes One off NSDP
Cyclone Recovery and Reconstruction Plan
NZ, ADB, UNDP, EU
12,519,000 MFEM 16,158,000 Grant Yes Integrated No Multi year NSDP
Preparing the Infrastructure development project
ADB 500,000 MOIP 500,000 Technical cooperation
Yes Stand alone No Multi year
Pandemic Influenza/H1N1 Vaccine Deployment Plan
WHO
NZAID/AusAID
133,000 Ministry of Health
140,000 Grant Yes No No
One off Yes
Pacific Island Regional Multi-Country
Programme to fight Tuberculosis
Global Fund HIV/AID, TB & Malaria
300,000 Secretariat of the Pacific Community
Ministry of Health
300,000
Grant Yes Yes No Multi year Yes
Health Specialists Visit Scheme NZAID/AusAID 355,000 Ministry of Health
355,000 Grant No
No Yes Multi year Yes
177
Appendix 12a: Health ‐HSV
Individual Profiles of Health and Infrastructure Sector Initiatives
Health Sector Initiative Profile 1 – Health Specialist Scheme Budget 355,000 Objectives: to provide equitable access to health
services
Development Partner NZAID/AusAID Harmonised
Programme
Terms of Assistance 2 year funding arrangement (established since 2004, reviewed and revised)
Description:
Series of scheduled and prioritised teams of visiting specialists deployed to provide tertiary and secondary level healthcare in specialist fields. For e.g. Orthopaedics, Ophthalmology, Mammography, Gynaecology, Paediatrics, Cardiology, mental Health. Also includes capacity building support.
Implementing Agency
Ministry of Health
Type of Assistance Grant Geographic Focus/Intended Beneficiaries:
Northern and Southern Groups – total population with focus on outer islands in Northern and Southern Groups screened and receive treatment.
Aid Delivery Modality
Donor Programme. Prior to 2004 visits by visiting New Zealand medical specialists were approve by MFAT on a case by case basis.
Q2 – Effectiveness of Aid Delivery Q3 – Achievement of Development Outcomes
Country Ownership • Alignment with national development priorities: Yes • Use of public financial management system: Yes –
AMD/MOH system • Use of government procurement system: Yes • Use of standalone/integrated PIUs: No – Managed by
MOH staff
Development Outcomes Achieved
Contributes to addressing challenges of providing good coverage of health services nationwide. Improved screening, early intervention and treatment as part of
MOH output and NSDP Goal 1 strategy - Maintain Medical Specialists programme achieved
Inclusive and Effective Partnerships • Use of PBAs and PBA-like aid delivery arrangements:
Yes • Use of delegated cooperation arrangements among
donors: Yes • Use of delegated decision making authority to field
staff: Yes • Use of joint missions and other development
activities: No
Supporting Performance Indicator Data
HSV programme M&E indicators by islands, specialist field, gender and age.
MDG report, NSDP review report, project documents, interviews
Accounting for Development Results • Involvement in mutual assessment reviews: Yes • Use of common frameworks: yes • Use of joint monitoring indicators: Yes • Use of joint institutional or program evaluations:
NZAID led • Use of publically available annual
progress/performance reports: No
Contribution to Health Sector Performance
Contributes to NSDP Goal 1 strategy to improve and protect the health of all cook Islanders including women, children and the elderly.
Contribution to TKN/NSDP – supports achievement of NSDP Goal 1 strategy to maintain adequate levels of advice to all Cook Islands people.
178
Appendix 12b: Health ‐ Pandemic Plan
Health Sector Initiative Profile 2 - Pandemic influenza/H1N1 Vaccine Deployment Plan
Budget 133,000 Objectives: to protect lives and save people from second wave of H1N1 pandemic
Description: implement deployment plan to outer islands and Rarotonga during a 1 month campaign.
Geographic Focus/Intended Beneficiaries: National focus for all Cook Islanders excluding infants under six months.
Development Partner WHO, NZAID/AusAID
Terms of Assistance One off – 3 month activity
Implementing Agency Ministry of Health
Type of Assistance Grant
Aid Delivery Modality Programme Based Approach – developed in response to Pandemic threat in 2008/9
Q2 – Effectiveness of Aid Delivery Q3 – Achievement of Development Outcomes
Country Ownership • Alignment with national development priorities: Yes • Use of public financial management system: Yes • Use of government procurement system: Yes • Use of standalone/integrated PIUs: No
Development Outcomes Achieved
Partial achievement to immunise 100% of the population.
Visits were completed and vaccines administered no reported extreme side effects.
Implemented against newly developed National prevention and response strategy.
Inclusive and Effective Partnerships • Use of PBAs and PBA-like aid delivery arrangements: Yes • Use of delegated cooperation arrangements among donors:
Yes • Use of delegated decision making authority to field staff:
Yes • Use of joint missions and other development activities: No
Supporting Performance Indicator Data
Vaccines administered refusals and any adverse side effects. (Data not yet available).
NSDP, MDG Report, MOH strategy and proposal, NZAID contract, Interviews
Accounting for Development Results • Involvement in mutual assessment reviews: Yes • Use of common frameworks: Partial • Use of joint monitoring indicators: Yes • Use of joint institutional or program evaluations: no • Use of publically available annual progress/performance
reports: no, CI use of media during rollout of project.
Contribution to Health Sector Performance
Supports NSDP goal 1 target to reduce the incidence of communicable diseases with an emphasis on STI/HIV/AIDS, vertor borne diseases, hepatitis, tuberculosis and other new communicable diseases.
Contribution to TKN/NSDP
Goal 6 – develop and implement a national Pandemic Prevention and Response strategy for the management of pandemics
179
Appendix 12c: Health ‐ GF – TB
Health Sector Initiative Profile 3 - Pacific Islands Regional Multi-Country to fight Tuberculosis
Budget $300,000 Objectives: reduce TB burden through increased detection
and improved surveillance, training and monitoring.
Description: A regional delivered programme across ten countries with strategies to increase detection, surveillance and referral systems, community awareness, human resource capacity.
Geographic Focus/Intended Beneficiaries: National Focus – primarily Rarotonga based
Development Partner Global Fund HIV/AID, TB & Malaria
Terms of Assistance Multi-year 200
Implementing Agency Ministry of Health
Type of Assistance Grant
Aid Delivery Modality PBA
Q2 – Effectiveness of Aid Delivery Q3 – Achievement of Development Outcomes
Country Ownership • Alignment with national development priorities: Yes • Use of public financial management system: Yes • Use of government procurement system: Yes • Use of standalone/integrated PIUs: integrated
Development Outcomes Achieved
Cook Islands remains TB free
Improving Disease management with developed policies and guidance,
Staff training conducted
Improved disease prevention and increase awareness about TB symptoms and treatment.
Inclusive and Effective Partnerships • Use of PBAs and PBA-like aid delivery arrangements: Yes • Use of delegated cooperation arrangements among donors:
Yes • Use of delegated decision making authority to field staff:
Mixed – local & sub-regional • Use of joint missions and other development activities: No
Supporting Performance Indicator Data
PIRMCCM documents and reports. NSDP, MDG report, SPC/CI project documents. Interviews.
Accounting for Development Results • Involvement in mutual assessment reviews: Yes • Use of common frameworks: yes • Use of joint monitoring indicators: yes • Use of joint institutional or program evaluations: yes • Use of publically available annual progress/performance
reports: reports produced
Contribution to Health Sector Performance
Supports Heath strategic policy
Contribution to TKN/NSDP –
Adds to NSDP goal 1 target to reduce the incidence of communicable diseases with an emphasis on STI/HIV/AIDS, vertor borne diseases, hepatitis, tuberculosis and other new communicable diseases.
180
Appendix 12d: Infrastructure ‐ Sports Stadium
Infrastructure Sector Initiative Profile 1 – Multi-functional Indoor Sports Stadium, Sports Facilities Upgrade and Ministry of Education Building Construction
Budget 13, 000,000 Objectives: Construct multi-functional indoor sports stadium, and Ministry of Education complex. Conduct repairs & upgrade of existing sports facilities and amenities
Description: Construction of building of facilities by China including China labour and materials. Some works and materials contracted locally for amenities and MOE building. Technical assistance was also provided by China to assist with Team Cook Islands athletes with Pacific Mini Games preparation.
Geographic Focus/Intended Beneficiaries:
Rarotonga based facility, Rarotonga General Populations with opportunities to include outer islands
Development Partner
China
Terms of Assistance
One-off
Implementing Agency
CIIC
Type of Assistance
Concessional Loan
Aid Delivery Modality
Projects within PBA
Q2 – Effectiveness of Aid Delivery Q3 – Achievement of Development Outcomes
Country Ownership • Alignment with national development priorities: Yes • Use of public financial management system: Yes for the
Cooks Islands component • Use of government procurement system: Partial • Use of standalone/integrated PIUs: Integrated
Development Outcomes Achieved
One of three strategies under Goal 1 of NSDP met – Provide international accredited sports facilities, robust training and competitive programmes.
Identified as a ‘nation building’ priority.
Inclusive and Effective Partnerships • Use of PBAs and PBA-like aid delivery arrangements: Yes • Use of delegated cooperation arrangements among donors:
Yes • Use of delegated decision making authority to field staff:
Yes • Use of joint missions and other development activities: No
Supporting Performance Indicator Data
Begin implementation in 2007, and monitor to 2009, the Pacific Mini Games Master Plan – Achieved
Upgrade existing sporting facilities, and the construction of new facilities targeting major sporting fixtures in 2009, which are accessible to all (including the disabled). - Achieved
NSDP Review Report, ADB strategy, Interviews.
Accounting for Development Results • Involvement in mutual assessment reviews: Yes • Use of common frameworks: No • Use of joint monitoring indicators: Yes • Use of joint institutional or program evaluations: • Use of publically available annual progress/performance
reports: No
Contribution to Infrastructure Sector Performance
Not included in the PIMP or infrastructure goal 5 of the NSDP – infrastructure to support national development.
Contribution to TKN/NSDP
Not stated - links to potential to generate revenue in the tourism industry. Support private sector led economy
181
Appendix 12e: Infrastructure – CRRP
Infrastructure Sector Initiative Profile 2 – Cyclone Recovery & Reconstruction Plan
Budget 12, 519,000 Objectives: Meet the need for emergency shelter for the displaced and homeless; revival of livelihood and sustainable economic recovery and reconstruction of homes and infrastructure to replace assets lost or destroyed.
Description: An investment programme focused on 12 sectors that coordinates efforts and processes to effect regeneration of communities following a cyclone disaster
Geographic Focus/Intended Beneficiaries: National with priority focus on islands damaged by 2005 cyclone season.
Development Partner
NZ, ADB, UNDP, EU
Terms of Assistance Multiyear – 2006 - 2009
Implementing Agency
MFEM
Type of Assistance Grant
Aid Delivery Modality
PBA – led by CIGov
Q2 – Effectiveness of Aid Delivery Q3 – Achievement of Development Outcomes
Country Ownership • Alignment with national development priorities: Yes • Use of public financial management system: Yes • Use of government procurement system: Yes • Use of standalone/integrated PIUs: Integrated
Development Outcomes Achieved
Partial achievement - harbour projects, cyclone centres, red cross centre (NZAID) shelter construction (Pukapuka – EU) in progress.
Atiu slipway, rainwater harvesting (ADB) completed.
Inclusive and Effective Partnerships • Use of PBAs and PBA-like aid delivery arrangements:
Yes • Use of delegated cooperation arrangements among donors:
Yes NZ/Aus • Use of delegated decision making authority to field staff:
Yes – NZAID, CEAP TA, EU. • Use of joint missions and other development activities: not
between donors
Supporting Performance Indicator Data
NSDP – 6: improve community cyclone shelters and disaster management facilities on all islands
NSDP 7: implement priorities in the PIMP in relations to airports and harbours facilities in Mauke & Mitiaro (NZAID funded)
NSDP review Report, OID DPA evaluation, OID TA monitoring report. MDG Report. Interviews. Planning documents.
Accounting for Development Results • Involvement in mutual assessment reviews: Yes • Use of common frameworks: No • Use of joint monitoring indicators: Yes • Use of joint institutional or program evaluations: no
evidence of co-funded studies but involves info sharing. • Use of publically available annual progress/performance
reports: No
Contribution to Infrastructure Sector Performance – majority projects still in progress.
Contribution to TKN/NSDP - NSDP Goal 7 Strengthened national resilience to natural disasters, including the effects of climate change, sea level rise and climate variability
NSDP Goal 6 – Establish a coordinated and effective national disaster risk reduction and disaster management system.
182
Appendix 12f: Infrastructure – PPTA
Infrastructure Sector Initiative Profile 3 – Preparing the Infrastructure Development Project
Budget NZ$500,000 Objectives: improve governance and institutional arrangements for infrastructure management
Description: Preparatory work as part of developing Infrastructure governance framework and priority projects.
Geographic Focus/Intended Beneficiaries: national infrastructure users, Public and Private sectors – Tourism industry.
Development Partner
ADB
Terms of Assistance Multi Year
Implementing Agency
MOIP
Type of Assistance Technical Cooperation
Aid Delivery Modality
PBA
Q2 – Effectiveness of Aid Delivery Q3 – Achievement of Development Outcomes
Country Ownership • Alignment with national development priorities: Yes • Use of public financial management system: No • Use of government procurement system: No • Use of standalone/integrated PIUs: Standalone
Development Outcomes Achieved
Improved governance and institutional arrangements for infrastructure management partially achieved
Inclusive and Effective Partnerships • Use of PBAs and PBA-like aid delivery arrangements: No • Use of delegated cooperation arrangements among donors:
Yes • Use of delegated decision making authority to field staff:
Yes • Use of joint missions and other development activities:
ADB lead donor for PPTA
Supporting Performance Indicator Data
Prepare proposals for an infrastructure governance framework and reform of institutional arrangements to be agreed with the government
ADB partnership strategy, ADB business plan, Interviews. NZAID/ADB contract arrangement.
Accounting for Development Results • Involvement in mutual assessment reviews: Yes • Use of common frameworks: yes • Use of joint monitoring indicators: Yes • Use of joint institutional or program evaluations: yes • Use of publically available annual progress/performance
reports: No
Contribution to Infrastructure Sector Performance – improving governance of sector in transition. Providing management framework to the infrastructure master plan and improved prioritisation of projects.
Contribution to TKN/NSDP – sector governance as a whole not identified as an indicator or strategy within the NSDP but picked up through the 07/08 review process.
183