Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a...

42
Global Evaluation of UNICEF’s Drinking Water Supply Programming in Rural Areas and Small Towns 20062016 Country case study report Ethiopia Drafted by: Evaluation team Reviewed and edited by: Jérémie Toubkiss, UNICEF Evaluation Office

Transcript of Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a...

Page 1: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

Global Evaluation of UNICEF’s Drinking Water Supply Programming in Rural Areas and Small Towns

2006–2016

Country case study report – Ethiopia

Drafted by: Evaluation team

Reviewed and edited by: Jérémie Toubkiss, UNICEF Evaluation Office

Page 2: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 2

Contents

A. Executive summary .......................................................................................................... 4

A.1. Background and objectives ....................................................................................... 4

A.2. Brief presentation of the sector context and of the evolution of UNICEF RWS programming ............................................................................................................. 4

A.3. Main evaluation findings ............................................................................................ 5

A.4. Main recommendations ............................................................................................. 8

B. Overview of the global evaluation ................................................................................. 11

B.1. Rationale and objectives ......................................................................................... 11

B.2. Evaluation design and criteria ................................................................................. 11

B.3. Role of country case studies ................................................................................... 12

B.4. Ethiopia country case study methodology ............................................................... 13

C. Country specificities regarding RWS ............................................................................ 15

C.1. Economic and physical context ............................................................................... 15

C.2. Progress in RWS coverage ..................................................................................... 15

C.3. RWS policy and institutional setting ......................................................................... 17

D. UNICEF RWS programme description .......................................................................... 18

D.1. Overview of the UNICEF RWS programme ............................................................. 18

D.2. Evolution of the UNICEF programme ...................................................................... 19

E. Evaluation findings ........................................................................................................ 20

E.1. Relevance ............................................................................................................... 20

E.2. Effectiveness ........................................................................................................... 21

E.3. Efficiency ................................................................................................................. 22

E.4. Sustainability ........................................................................................................... 24

E.5. Equity ...................................................................................................................... 26

E.6. Innovation ............................................................................................................... 28

F. Recommendations to UNICEF ....................................................................................... 29

F.1. Continue to adapt its RWS programming, with increased linkages between downstream and upstream work ............................................................................. 29

F.2. Further support the sustainability of RWS services ................................................. 30

F.3. Increase focus on equity issues .............................................................................. 31

G. Annexes .......................................................................................................................... 33

Page 3: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 3

Acronyms

Acronym Developed

CWA Consolidated WASH Account

DFID International Development Agency (United Kingdom)

DHS Demographic and Health Survey

GTP Growth and Transformation Plan

JMP WHO/UNICEF Joint Monitoring Programme

MDG Millennium Development Goals

MICS Multiple Indicators Cluster Survey

MoWIE Ministry of Water, Irrigation & Electricity

MoE Ministry of Education

MoFEC Ministry of Finance and Economic Cooperation

MoH Ministry of Health

NGO Non-governmental organization

NUWI Netherlands and UNICEF WASH Initiative

OFDA Office of US Foreign Disaster Assistance

OWNP One WASH National Programme

RWS Rural and small town drinking water supply

SDG Sustainable Development Goal

SNNPR Southern Nations, Nationalities, and Peoples’ Region

SWAp Sector-wide approach

UAP Universal Access Plan

UNDAF United Nations Development Assistance Framework

UNESCO United Nations Educational, Scientific and Cultural Organization

USD United States dollars

WASH Water, sanitation and hygiene

WASHCO WASH committees

WHO World Health Organization

WSP Water and Sanitation Programme

Page 4: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 4

A. Executive summary

A.1. Background and objectives This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming in Rural Areas and Small Towns’, which was commissioned by the UNICEF Evaluation Office to assess UNICEF’s experience and contributions in this area between 2006 and 2016. The global evaluation is informed by seven other country case studies, which all follow the same structure and methodology, and by additional sources of evidence, including a review of UNICEF and non-UNICEF documents and databases, a global online survey, and semi-structured interviews with diverse sector stakeholders.

The country case study is not a full evaluation as it is less comprehensive in terms of scope, data collection and analysis. Its main objective is to feed into the global evaluation report by gathering information about UNICEF’s programming in rural and small town water supply (RWS) and about the evolution of UNICEF’s contribution to the sector at the country level. The case study also presents findings and recommendations that are intended to be useful to UNICEF Ethiopia and its partners in-country.

A.2. Brief presentation of the sector context and of the evolution of UNICEF RWS programming

Ethiopia has experienced strong economic growth since the mid-2000s, underpinned mainly by public sector-led development. While Ethiopia has relatively abundant water resources and ample per capita availability, it is considered water stressed due to significant geographical and temporal variations in rainfall and lack of storage facilities.

The country achieved the Millennium Development Goal (MDG) target to halve the proportion of people without access to safe water, despite rapid population growth. The number of people in rural areas with access to water supply increased from 3% in 1990 to 49% in 2015. During that time period, approximately 30 million people gained access to an improved water source.

However, Ethiopia continues to face considerable challenges with regard to achieving the new Sustainable Development Goal (SDG) of universal and equitable access to safe and affordable drinking water by 2030. The extension of access to water supplies in rural areas has only kept pace with the population growth. The rate of expansion of water coverage thus needs to increase. In addition, improvements in access have not been equitable across all regions (with arid and semi-arid areas having poorer access), woredas (districts), population groups, and wealth quintiles. Finally, the SDG standards stipulate that water supply must not only be safe in terms of water quality and risk of contamination, but must also be sustainable. Yet, evidence indicates that between 20–35% of the hand pumps installed in Ethiopia are currently dysfunctional.

Successive sector policy and institutional reforms have significantly strengthened the enabling environment for water, sanitation and hygiene (WASH) in general and RWS in particular. They have delegated responsibility for service delivery to the woreda authorities and have given them the mandate for investment planning, monitoring and providing technical assistance to service providers. Ministries in charge of water, education, health, and finance and economic cooperation have been working together under the One WASH National Programme (2010–

Page 5: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 5

2020), which aims to better coordinate government efforts and harmonize and align planning and investments with development partners. A pooled funding mechanism has been established, and it has received contributions from several major donors including UNICEF.

The Government has placed considerable emphasis on low-cost technologies, such as spring protection and hand-dug wells equipped with hand pumps. Piped supplies (including multi-village reticulated schemes) account for only about 1% of all improved rural water sources. The Government has set the ambitious target of increasing this figure to 20% by 2020, and to achieving universal access in the same year, 10 years ahead of the 2030 SDG deadline.

During the evaluation period, UNICEF was one of the major development partners supporting the Government’s RWS programme. And within UNICEF, Ethiopia has been, and still is, one of the largest WASH/RWS programmes globally.

Over the course of the evaluation period, the focus of UNICEF’s downstream project implementation support shifted from promoting self-supply and low-cost technologies, which result in community-owned and operated assets, to more complex networked systems that are owned by woreda authorities and managed by public utilities operating at the woreda level. In addition, UNICEF has increased its upstream work to help strengthen the Government’s enabling environment for WASH services.

A.3. Main evaluation findings

Strengths Weaknesses

Relevance

UNICEF is a well-positioned and credible partner in the sector due to its:

- Considerable financial contribution (UNICEF accounts for almost half of sector progress towards the RWS-related MDG, based on estimates);

- Capacity to mobilize resources quickly;

- Leadership in both development and emergency programming;

- Presence throughout the country at the sub-national level;

- Long-standing collaboration with the government; and

- Role in strengthening the enabling environment at the upstream level: contribution to sector policies/plans and sector-wide approach (SWAp); support to coordination and review mechanisms; co-chairing of working groups, etc.

Decrease in financial contribution to the RWS sector is a potential threat to UNICEF’s future positioning and credibility in the sector

Page 6: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 6

Strengths Weaknesses

Effectiveness

Intended/planned results achieved or exceeded

Successes in institutional strengthening

Major investment to increase the implementation capacities of regional and local government water departments

Relevant and effective evolution towards more upstream support over the evaluation period

Evidence on RWS produced through evaluations, surveys and sustainability checks

Limited impact (especially at the community level) and institutionalization of capacity-building efforts

Weaknesses in the dissemination and utilization of generated evidence for informing sector policies/strategies

Efficiency

Series of measures taken over time to identify programmatic risks and improve efficiency, including:

- As part of the setting up of the SWAp: fiduciary risk assessment of the government administration conducted and code of conduct signed for contributors to the Consolidated WASH Account (CWA);

- Continuing direct cash transfers to sub-national departments and introducing direct payment modality to service providers;

- Conducting capacity assessments of sub-national authorities and developing/implementing capacity building plans, which reportedly resulted in an improvement in the quality of construction;

- Shifting towards local procurement of spare parts (notably hand pumps); and

- Increasing programmatic integration of RWS interventions with other sectors, including nutrition and education.

Inefficiencies and delays resulting from the choice to channel funds and implement interventions through government entities at the national and sub-national levels and through the CWA

Page 7: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 7

Strengths Weaknesses

Sustainability

Conduct of periodic sustainability assessments/checks and other studies and surveys to identify sustainability issues and monitor the evolution of RWS outputs and outcomes over time

Conduct of hydrogeological studies in regions with high levels of water stress to ensure proper siting and development of boreholes

Recent shift from low to high technology, scale and service levels to promote more reliable and climate-resilient services

Willingness and ability to pay for studies carried out and business plans developed for the water operators in small towns, intended to inform tariff setting and support the financial viability of piped water schemes

Relevant downstream and upstream work at the sector level, which has laid the groundwork for addressing sustainability

Weaknesses in tariff-setting, especially for boreholes (not based on an appropriate analysis of the life cycle costs of the water points and of the users’ ability to pay)

Concerns about the recovery of operation and maintenance costs as well as the capital replacement costs of boreholes and piped network systems: UNICEF’s primary focus seems to have been the development of services rather than the sustainability of existing services

Lack of institutionalized post-construction training and follow-up support to WASH committees, service providers and woredas; and weak human resource capacity at the woreda level

Community-based management model used for the multi-village schemes probably inadequate/ineffective, with communities being involved in their management rather than in oversight/regulation/accountability activities

Equity

UNICEF’s approach to geographical targeting in RWS consistent with the Government’s approach, which is focused on reducing coverage inequalities among regions (and also taking into account vulnerability to climate change)

Leader in adopting and promoting a gender lens in the RWS sector: gender audit conducted, action plans and guidelines developed, presence and roles of females in WASH committees at community level made mandatory

Concerns about the observed increase (nationally) in the gap between the richest and poorest wealth quintile in the use of improved water sources

Pro-poor tariffs and cross-subsidy mechanisms not actively promoted by UNICEF

Arrangements not made to promote participation of the poor, elderly, disabled, transient populations, ethnic or religious minorities in community-based WASH management committees

Lack of equity-lensed monitoring and evaluation

Innovation and scaling up

Innovative aspects within UNICEF programming: shift from direct cash transfers to direct payment with turnkey contracts; remote sensing technologies combined with traditional hydrogeological study techniques to enable mapping of large areas of the country; promotion of the SWAp and pooled funding mechanism; and introduction of sustainability checks/assessments

Limited evidence of replication/scaling up by the national and regional governments

No innovation identified in the following areas: management models for RWS services and involvement of the private sector; and accountability and regulation mechanisms – despite needs/opportunities

Page 8: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 8

A.4. Main recommendations The Ethiopia RWS sector and its enabling environment have evolved quickly over the past 10 years. The MDG target has been met. The SDGs now set a new, even more ambitious agenda for the country with a 2030 horizon. The objective of boosting the coverage rate will continue to drive the sector, but additional challenges will increasingly draw the attention of all stakeholders, including: improving the level of service (in terms of quantity, accessibility, reliability and quality), ensuring the financial viability and affordability of the service, and reaching the most remote and vulnerable communities and households.

The WASH strategy in the current UNICEF country programme involves both upstream sector strengthening work and downstream implementation using higher technology/higher cost solutions to improve the level of service and build resilience to climate change.

The following recommendations are based on both the findings of this case study and the wider context of the SDGs, the RWS sector and the UNICEF country programme. These recommendations should be considered as a menu of options rather than a plan of action. The most appropriate options for UNICEF will depend on numerous factors including the dynamics in the sector, the wider strategic vision of the UNICEF country office regarding its assistance to the country, the resources at its disposal, the positioning of other RWS donors and stakeholders, and UNICEF’s analysis of its own comparative advantages and added value.

Continue to adapt its RWS programming, with increased linkages between downstream and upstream work

1. The high technology/high cost approach chosen by UNICEF will likely reach fewer direct beneficiaries than over the past 10 years (for the same budget). Yet the Government needs to double the rate of increase in RWS access to achieve the SDGs. Therefore, UNICEF should use its direct implementation projects as opportunities for testing innovations and as learning laboratories to document/evaluate what is working and what needs to be improved in water scarce areas, and to inform its policy advocacy and upscaling/mainstreaming efforts. This will strengthen the linkages between UNICEF’s downstream and upstream work, and help the organization increase its impact on the RWS sector with limited funding – thus maximizing the value for money of its programming. This approach implies:

Designing field interventions with an innovation lens and a clear research/evaluation agenda, based on a well formulated theory of change.

Continuously refining the intervention design based on research/evaluation findings and lessons learnt, with the view to developing models of RWS services that work for various contexts.

Documenting/publishing the results and disseminating them throughout the sector. This means ensuring that those who require this information are able to access it and know how to effectively use the data for decision-making, policymaking, etc. UNICEF is recognized globally and within Ethiopia for its contribution to learning and knowledge, but its full potential in this area has not been realized in Ethiopia. UNICEF should ensure that the research/evaluation process used is fully understood and can be replicated (i.e. sharing not just the results but also the cleaned data sets and methods used to collect, analyse and present the information)

Advocating to the government and other sector stakeholders for the replication, scaling up, mainstreaming and institutionalization of successful models and innovations, in a coordinated effort with its partners.

Page 9: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 9

In doing so, UNICEF should focus on the key bottlenecks and challenges identified internally and in the RWS sector (at the field, sub-national and national levels). This approach requires UNICEF to use its considerable human resources in a more analytical and upstream manner.

The findings of this country case study indicate that sustainability and equity are two priority areas of work, both for UNICEF and the broader RWS sector. The more specific recommendations related to these two topics are summarized below and further developed at the end of the report.

Further support the sustainability of RWS services

In order to strengthen the sustainability of RWS services, UNICEF should (continue to) work with the Government and other RWS sector stakeholders to:

2. Develop/validate/implement a long-term financing strategy for water supply services that determines an appropriate planning, mix and allocation of tariffs, government budget and external aid to ensure that both operations and maintenance and capital replacement costs are covered. Determine how tariffs can better contribute to the viability of the service while remaining realistic and affordable to the poorest.

3. Continue to innovate, test and document/evaluate alternative RWS service delivery/management models.

4. Institutionalize capacity-building efforts to ensure training and technical support opportunities are available to all stakeholders (especially WASH committees, small-scale private service providers, woredas and zonal and regional water bureaus) on a continuous basis.

5. Continue current efforts to improve the quality of RWS services through improved regulation, both at community/service and sector/central level.

6. Support the supply side through the promotion of local construction and manufacturing capacities.

7. Put systems and capacities in place to ensure that water resources are effectively and efficiently managed.

8. Institutionalize periodic, sector-wide and government owned/led sustainability checks/assessments designed to inform decision-making.

Increase the focus on equity issues

In order to strengthen equity in the RWS sector and services, UNICEF should (continue to) work with the Government and other RWS sector stakeholders to:

9. Ensure that all groups (including underprivileged groups) are taken into account and involved in RWS planning, implementation and management processes, at both the sector and field implementation levels. This is likely to become more critical as the move towards more highly engineered systems will probably result in an increase in top-down approaches in the sector, driven by technologies and financial resources. UNICEF has an important role to play in providing the Government with evidence and advice on how to fulfil the equity agenda set by the SDGs (examples of such activities are provided in the recommendation section at the end of the report).

10. Document and disseminate among the Government and development partners the lessons learnt on low technology/low cost solutions, and how they can particularly benefit the poorest and most remote communities that cannot access/use more sophisticated (and costly) water

Page 10: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 10

schemes. It will be important for some other institution or partner to fill any gaps left by UNICEF’s change in approach.

11. Improve the sustainability checks/assessment methodology and the sector monitoring and evaluation system by including a stronger equity lens.

Page 11: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 11

B. Overview of the global evaluation

B.1. Rationale and objectives This country case study report is a component of the global evaluation of UNICEF’s drinking water supply programming in rural areas and small towns between 2006 and 2016. The global evaluation was commissioned by the Evaluation Office at UNICEF Headquarters in New York. It was designed to assess UNICEF’s experience in drinking water supply programming in rural areas and small towns to fill specific knowledge gaps, draw lessons and improve the appropriateness of UNICEF strategies globally and the quality of its programming in the field. In doing so, it will inform the development of the water, sanitation and hygiene (WASH) component of the next Strategic Plan 2018–2021 and guide UNICEF into the new Sustainable Development Goal (SDG) area. The evaluation was designed to contribute to global learning and promote UNICEF’s accountability to internal and external stakeholders. It examines both downstream work in service delivery and upstream work in strengthening the enabling environment for the rural and small town water supply (RWS) sector at the national and global levels.

B.2. Evaluation design and criteria The global evaluation is structured around six main evaluation criteria and six key evaluation questions, listed in the first two columns of Table 1. The eight country case studies use the same six evaluation criteria and key evaluation questions. They form one component of the evidence base for the global evaluation, which also includes a review of UNICEF and non-UNICEF documents and databases, a global online survey, and semi-structured interviews with a diversity of sector stakeholders. The areas of particular interest for the country case studies are listed in the third column of the table.

Table 1: Key global evaluation questions and areas of interest for the country case studies

Criteria Key global evaluation question Areas of particular interest for country case studies

Relevance Has UNICEF been a well-positioned, credible partner for national governments and major development agencies, demonstrating alignment and complementarity both globally and within countries?

UNICEF’s position in the field of drinking water supply in rural areas and small towns

The credibility, adaptation and complementarity of UNICEF’s activities with those of its partners and of the other major in-country players

Effectiveness To what extent has UNICEF a) achieved its global and country output and outcome level targets through quality programme implementation; and b) effectively contributed to the water-related Millennium Development Goal (MDG)?

Achievement of output and outcome level targets in-country, and contribution to the MDGs

Success of policy advocacy, capacity-building and knowledge generation/management activities

Quality of programme implementation

Page 12: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 12

Criteria Key global evaluation question Areas of particular interest for country case studies

Efficiency Has UNICEF maximized the costs-results relationship by systematically integrating efficiency considerations into its activities at the global, regional and country levels, notably by promoting programmatic integration and partnerships with other WASH and non-WASH initiatives?

Relationship between costs and results

Use of cost-efficient approaches and measures, including engagement in integrating the water supply intervention with other WASH (e.g. sanitation, hygiene) and non-WASH interventions

Equity What has been the level of equity-sensitivity in the design, implementation and monitoring and evaluation of RWS activities at the global, regional and country levels?

Geographical targeting at the country level

Equity-sensitivity of UNICEF RWS programming, including in the monitoring and evaluation systems

Sustainability Has UNICEF integrated appropriate measures and tools at all levels and achieved a satisfactory level of sustainability in its drinking water supply programming in rural areas?

Evidence on the actual level of sustainability of past interventions

Extent to which the technical, financial, social, institutional and contextual factors known to support water supply sustainability have been taken into account in UNICEF’s water supply programming

Sustainability-lens in UNICEF monitoring and evaluation and information management systems

Innovation / upscaling

Has UNICEF been able to identify and test new programmatic approaches and take them to scale if successful?

Private sector participation in the management of RWS services

Real-time monitoring

Sector regulation and accountability mechanisms

Innovative financial mechanisms to support access to the service

B.3. Role of country case studies Country case studies are not country evaluations. They are used to document some – but not all – of the overall evaluation questions and indicators. The objective of the country case studies is to bring additional evidence from the country/field level and document some country specificities, feeding into the global evaluation report. At the same time, the evaluative country case study draws findings and recommendations intended to be useful at the country level.

Page 13: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 13

B.4. Ethiopia country case study methodology

B.4.1. Data collection methods

Two members of the global evaluation team undertook a two-week visit to Ethiopia, accompanied by a local consultant and by UNICEF Ethiopia staff. The mission to place in October/November 2016. See Annex 1 for a full itinerary of the mission.

a) Semi-structured interviews and (focus) group discussions

Thirty-three semi-structured interviews and six group discussions (one group discussion per kebele) were carried out with UNICEF staff, key partners and beneficiaries, as well as with other RWS stakeholders at the local and national level. See Annex 2 for a complete list of the stakeholders interviewed.

b) Field visits and meetings/interviews with local stakeholders

Field visits to four woredas (districts) enabled the evaluation team to build a better understanding of the local context, interview local/sub-national stakeholders and rights holders, obtain an overview of UNICEF interventions during the period under review, triangulate information from documents and interviews, and collect (limited) field data, particularly on sustainability and equity aspects. In each of the six communities visited (listed in Annex 3), the evaluation team used a structured observation protocol, which included qualitative information such as pictures, videos and interviews with local stakeholders, the community’s natural leaders, WASH committee (WASHCo) members, and the direct users of the services.

c) National and sub-national workshops

Workshops make it possible to collate feedback from a larger group of stakeholders than individual interviews and help build a certain level of consensus in responding to the evaluation questions, or at least ensure triangulation. It was for this reason that two workshops were carried out: one at the sub-national/local level and another at the country/national level.

A sub-national-level workshop was held in Tigray on 5 November 2016. It was attended by 45 participants from the region (3 women, 42 men), including representatives from a public enterprise, a public utility, 29 representatives from 9 woredas (7 individuals involved in the implementation of the UNICEF 2006–2015 country programme and 2 involved in the current country programme), and 11 WASHCo members from 3 kebeles.

A national workshop was held in Addis Ababa on 9 November 2016, which was attended by 15 people. Attendees included representatives from Coffey, the Netherlands Development Organisation SNV, CoWASH, UNICEF, WHO, Ministry of Water, Irrigation & Electricity (MoWIE) and the Ministry of Health. Alongside UNICEF, these institutions have significant experience in RWS in Ethiopia. Individual participants were targeted due to their familiarity with UNICEF programming.

Annex 4 has the list of participants in each workshop.

d) Document review

Several national government documents, national WASH sector documents, and UNICEF public and internal documents were reviewed. These are listed in Annex 5.

Page 14: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 14

e) Wrap-up meeting between the evaluation team and the UNICEF country office

A wrap-up meeting was held on 10 November 2016 at the end of the country visit. The evaluation team presented the key preliminary findings and general trends identified during the field visits, stakeholder/group interviews and workshops, in order to gather and incorporate feedback.

f) Review of the draft country case study report

The draft country case study report was quality reviewed by both UNICEF Ethiopia and the UNICEF Evaluation Office, and further edited by the Evaluation Office before finalization.

B.4.2. Limitations

Due to political violence and government restrictions, the site visits were limited to Tigray Region. The Amhara and Oromia regions have had the largest number of UNICEF interventions. However, field observations and interviews were not intended to be representative of all UNICEF RWS interventions during the period under review. Moreover, the evaluation team met with most key institutions and individuals with significant knowledge of the UNICEF programme.

Page 15: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 15

C. Country specificities regarding RWS

C.1. Economic and physical context Ethiopia has experienced strong economic growth since the mid-2000s, underpinned mainly by public-sector-led development.

While Ethiopia has relatively abundant water resources and ample per capita availability, it is considered water stressed due to significant geographical and temporal variations in rainfall and a lack of storage facilities: water is often not available where and when needed. Moreover, only a very small proportion of water resources are actually used, of which only between 10% and 20% is for domestic water supply. As Ethiopia’s economy develops and the population continues to grow, demands on available water supplies will increase. The majority of the rural community water supply relies on groundwater through shallow wells, deep wells and springs. Rainwater harvesting is also common. The rainy season is from the end of June to the end of September.

C.2. Progress in RWS coverage Ethiopia has achieved the MDG target to halve the proportion of people without access to safe water. In 2015, the WHO/UNICEF Joint Monitoring Programme for Water Supply and Sanitation (JMP) reported that overall coverage, in both rural and urban areas in Ethiopia, was 57% against 13% in 1990. The coverage in rural areas was 49% against only 3% in 1990. The rural population increased substantially during the same time period. In 1990, there were 42 million people living in rural areas, a figure that grew to 79.7 million in 2015. Thus, there has been substantial growth in the absolute numbers of rural people with access to improved water supplies – rising from less than 10 million in 1990 to more than 40 million in 2015. The percentage of the population accessing surface water decreased between 1990 and 2015, from 54% to 16% and, in absolute terms, there were 9.9 million fewer people using surface water in 2015 than in 1990, despite the population growth. This is a laudable achievement.

However, Ethiopia continues to face considerable challenges with regard to achieving the new Sustainable Development Goal (SDG) of universal and equitable access to safe and affordable drinking water by 2030 (the government of Ethiopia’s own target is to reach 100% by 2020). The absolute number of people using unimproved water sources remained unchanged over the MDG period: 40.7 million in 1990, and 40.6 million in 2015. Considering the rate of progress to date, Ethiopia is not on track to meet the SDG by 2030. At the current pace, universal access should be achieved around 2042 (see Figure 1). The marginal cost/effort required to reach the population currently without access will increase. The Government will not only need to increase the rate of expansion of access to between 3 million to 4 million people per year, but will also likely need to invest more per capita for access in order to meet the SDG.

The variability of rainfall and the decreasing availability of groundwater resources (especially in the lowlands) as a consequence of climate change will likely result in more costly investments. Moreover, the SDG standards stipulate that water supply must not only be safe in terms of quality and risk of contamination but must also be sustainable. In addition to extending water supply coverage that meets the new standards, it will be necessary to maintain acceptable service levels for those who already have access, which will also be a challenge. A recalibration of current coverage levels based on the new SDG standards means that the starting point in 2016 is considerably lower than 49%.

Page 16: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 16

Figure 1: Access to improved water supply in rural areas of Ethiopia

Source: JMP, 2015

Some of the issues that need to be addressed to ensure the delivery of sufficient, reliable, sustainable and safe rural water supply services include the reduced yield of water sources during the dry season; the difficulty in developing deeper wells and boreholes; the affordability of motorized pumps; and the increasing cost and limited availability of fuel and spare parts in some remote rural areas. For example, evidence suggests that between 20%–35% of the hand pumps installed in Ethiopia are currently dysfunctional.

The Government has placed considerable emphasis on promoting low-cost technologies such as spring development/protection, and hand-dug wells equipped with hand pumps. The overwhelming majority of improved access in rural areas is to non-piped services, with piped supplies (including multi-village reticulated schemes) accounting for only about 1% of all improved sources.

Inequalities persist between urban and rural areas, as well as between/within regions and woredas. Inequalities between the rich and poor are also significant and growing. In 2013, the JMP reported coverage data disaggregated by wealth quintiles (see Figure 2). This data shows that the gap between the richest quintile and the poorest quintile was 14 percentage points in 1995 and increased to 20 percentage points in 2012. However, this gap remains relatively modest compared to other

countries in sub-Saharan Africa.

1% 1%1%

3% 9%19%

28%38%

48%43%42%

40%

39%

37%

35%54% 49%

41%32%

24%16%

77%

0%

20%

40%

60%

80%

100%

1990 1995 2000 2005 2010 2015 2020 2025 2030 2035 2040 2045

Piped onto premises Other improved Other unimproved

Surface water Total improved Current Trend

Necessary Trend

Universal access in 2042

Figure 2: Drinking water coverage in rural areas of Ethiopia by wealth quintile from 1995 to 2012

Source: JMP, 2012

Page 17: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 17

C.3. RWS policy and institutional setting Sector reform began in the early 1990s when the institutional arrangement for basic service provision was

devolved to Regional Water Bureaus. In 1995, the Ministry of Water Resources was established and assigned policymaking, coordination and regulatory functions. The National Water Resource Management Policy (1998) and the National Water Strategy (2000) set out the basic principles and objectives for extending services, including institutional roles and social and financial aspects. Additional reforms in the 2000s delegated responsibility for service delivery to the woreda authorities and gave them the mandate for investment planning, monitoring and providing technical assistance to service providers. In urban areas, autonomous utilities were established. Important developments and elements in sector policy achieved during the recent past include the following:

1) Recognizing water and sanitation as economic and social goods;

2) Devolving ownership and management autonomy to the lowest possible local level;

3) Moving towards ensuring that water tariffs enable full cost recovery for urban schemes and the recovery of operation and maintenance costs for rural schemes; and

4) Integrating sanitation and hygiene promotion with water supply.

In 2005, following engagement with international donors, the Government of Ethiopia developed an ambitious plan for universal access to water and sanitation that brings together the four main federal ministries dealing with WASH services. In addition to the Ministry of Water, Irrigation & Electricity (MoWIE), these are the ministries of Education (MoE), Finance and Economic Cooperation (MoFEC) and Health (MoH). The One WASH National Programme, launched in 2013, is an inter-ministerial, coordinated government-led response to the targets set out in the Government’s Growth and Transformation Plan (GTP) 2010–2015. It aims to coordinate ministry efforts and to harmonize and align planning and investments with development partners, based on a sector-wide approach (SWAp). The OWNP builds on the 2011 WASH Implementation Framework and includes a pooled funding mechanism, the Consolidated WASH Account (CWA). The CWA receives contributions from the Department for International Development (United Kingdom) (DFID), World Bank, African Development Bank and UNICEF.

The OWNP is an ambitious programme with a total envelope of approximately 4 billion USD over a seven-year period, implemented in two phases. Phase I ran from July 2013 to June 2015 and Phase II began in July 2015 and will run through June 2020. This phased approach is to allow for any important changes in government policies, strategies and plans to be accommodated as the programme progresses. For the first phase, approximately 1.03 billion USD was earmarked for the rural sector for rural and pastoral WASH programming, two-thirds of which was for RWS (representing 47% of the entire planned investment). Around 85% of the rural water component is for the provision of new water facilities, rehabilitation and design, and the remainder is for capacity support and software; just over 2% has been set aside for post-construction support and supply chain development. In addition, there is renewed interest in multi-village schemes and rural piped water supply is expected to receive more attention under GTP II with an ambitious target to reach 20% of the rural population through piped supplies.

This set of policies, frameworks and goals represents a strong political commitment to not only increasing access, but also improving the capacity of institutions to support long-term rural water service delivery. On paper at least, there are clearly established functions, coordination mechanisms and guidance for implementation. These policy measures, combined with the country’s strong commitment to decentralization down to the lowest levels, provide a sound framework for the rural water sub-sector.

Page 18: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 18

D. UNICEF RWS programme description

D.1. Overview of the UNICEF RWS programme Within UNICEF, Ethiopia has been, and still is, one of the largest WASH/RWS programmes globally. During the evaluation period, UNICEF was one of the major development partners supporting the Government’s RWS programme. This period covers three UNICEF country programmes – the third, fourth and fifth country programmes developed in Ethiopia. The current (i.e. sixth) country programme covers the period 2016–2020. A summary of these programmes is given in Figure 3.

Figure 3: Overview of the UNICEF country programmes (CP) during the evaluation period (2005–2016)

Source: Data from UNICEF Ethiopia

Since 2010, UNICEF has been working mainly through local government. Over the evaluation period, UNICEF worked with several government counterparts including the MoWIE, MoH, MoE, MoFEC, Ministry of Urban Development and Construction, the Water Resources Development Fund and the National Disaster Risk Management Commission. Main donors to the WASH programme included the European Union; DFID; the German government-owned development bank KfW; the Government of Japan; the Government of the Netherlands; and the Office of US Foreign Disaster Assistance (OFDA). UNICEF has also partnered with various United Nations agencies, non-governmental organizations (NGOs), academic institutions and private sector firms.

Page 19: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 19

D.2. Evolution of the UNICEF programme Over the course of the evaluation period, the focus of UNICEF’s downstream project implementation support shifted from promoting self-supply and low-cost technologies, which result in community-owned and operated assets, to more complex networked systems that are owned by woreda authorities and managed by public utilities operating at the woreda level. During this period, UNICEF support evolved towards prioritizing the development of more resilient water sources and reinforcing government ownership. In addition, UNICEF has increased its upstream work to help strengthen the Government’s enabling environment for WASH services.

This shift in approach is illustrated in Figure 4.

Figure 4: Evolution of the UNICEF country programme between 2005 and 2016

Source: UNICEF, 2016

There were several other significant strategic shifts initiated during the evaluation period that have been rolled into the current UNICEF country programme (2016–2020). These are described in the relevant findings sections below.

Page 20: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 20

E. Evaluation findings

E.1. Relevance UNICEF’s positioning, role, credibility, comparative advantage and added value in the RWS sector over the last 10 years

During the past 10 years, the majority of WASH funding from UNICEF has been for RWS project implementation. This has made UNICEF the Government’s most important partner for achieving the RWS MDG target. For example, during the fifth country programme (2012–2016), it is estimated that UNICEF was directly responsible for providing access to approximately 1 million of the country’s 2.2 million new beneficiaries per year. On average, over the 2006–2016 period, it is estimated that UNICEF funded almost half of the RWS sector results in terms of new users of improved water sources (44% in 2007–2011; 50% in 2012–2016). UNICEF is recognized as having the capacity to mobilize resources quickly. This demonstrates the strong contribution and positioning of UNICEF in the sector.

The Government and development partners consider UNICEF to be an important and credible partner in the RWS sector. This is not only because of its financial contribution but also because of its long-standing collaboration with the Government and strong presence at both the national/federal level and throughout the country’s regions. Donors highlighted that UNICEF’s main comparative advantage in RWS project implementation is having WASH staff based at the sub-national/field level to monitor and supervise donor investments and provide reporting. UNICEF’s work in water supply in emergencies is considered even more important. This is significant because many, if not most, of the UNICEF WASH teams work on both development and emergency RWS (including coordination of WASH/RWS in emergencies). This is not necessarily the case with development partners.

This allows UNICEF to play a significant role not only in the field (increasing RWS coverage) but also in strengthening the enabling environment at the national/sector level. UNICEF was a key partner of and support to the Government (along with other development partners such as the World Bank) in developing a WASH sector-wide approach (SWAp) and the related One WASH National Programme in 2013, which promotes one budget, one reporting system and one plan, and in updating the Universal Access Plan (i.e. UAP 2). UNICEF is also very active in sector-level coordination and dialogue with government and development partners as the agency has co-chaired the National WASH Working Group as well as various technical working groups (e.g. emergency, urban WASH, and hygiene and sanitation). As a result of its contribution to the development of national policies, strategies, plans and coordination mechanisms, UNICEF’s programming has been well aligned with these sector-level arrangements. For instance, UNICEF’s WASH programme is aligned with the WASH component of the Government of Ethiopia’s national GTP II (2015–2020) as well as with its technology choices, and feeds into the CWA, which is the pooled fund of the OWNP. However, it is not clear how UNICEF ensured that its upstream work was strategic, targeted and systematic.1

1 For example, UNICEF has not developed a situation analysis document, bottleneck analysis (the WASH bottlenecks analysis tool – WASH BAT – promoted by UNICEF helps analyse strengths and weaknesses in the WASH sector building blocks), strategy, or concept note in the period under review. In 2017, however, UNICEF conducted a bottleneck analysis and developed a revised UNICEF WASH concept note/strategy for Ethiopia, as part of the mid-term review of the UNICEF country programme.

Page 21: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 21

As a result of the decrease in donor funding to UNICEF and UNICEF’s shift to higher cost RWS solutions, the current Country Programme Document calls for far less direct contribution to increase RWS access (750,000 USD over a four-year period, or less than 200,000 USD per year, as compared to the over 1 million USD a year contributed by UNICEF over the past 10 years). The implications are that UNICEF may become less well-positioned and relevant in terms of direct contributions to increased access.

Given this situation, UNICEF could gain further relevance if it continues to provide upstream support strategically targeted at the sector-level bottlenecks that could impede the achievement of the RWS-related Government or SDG targets.

Conclusion on relevance: UNICEF has been well positioned, credible and overall relevant in Ethiopia’s RWS sector during the evaluation period. The organization has been a (if not the) major external support agency in the sector for the last 10 years, playing a key role in the achievement of the RWS MDG target and in the strengthening of the enabling environment.

E.2. Effectiveness The degree to which UNICEF-funded programmes have produced the desired results through quality implementation, including in capacity building, policy advocacy and knowledge management

Based on the document review, UNICEF has met or exceeded planned results (number of communities or beneficiaries with increased accessed to RWS in the third and fourth country programme). The number of UNICEF direct beneficiaries reported by the organization shows that it delivers results for the interventions it funds and supervises – especially in terms of constructing new water schemes.

The level of results achieved by UNICEF through direct contributions is largely dependent on donor support – which includes a combination of availability of donor funding, competing priorities for those funds and the fundraising ability of the Government and UNICEF – and on the effectiveness and efficiency of its implementing partners.

In interviews, regional and local governments recognized and appreciated UNICEF’s financial investment not only in new water infrastructure, but also in capacity building (by purchasing computers, motorcycles, office equipment, software, etc.). This contribution has reportedly improved their ability to procure materials and supervise project construction. Indeed, UNICEF’s capacity-building programme during 2009–2013 (which ran parallel to the capacity-building fund managed by the federal government) involved the professional development or (re)training of tens of thousands of stakeholders currently active in the WASH sector. This included technicians, artisans and health extension workers at the local level, staff in zonal and regional water bureaus at the meso level, as well as professionals in federal and regional ministries. This programme was also successful at the upstream level, specifically the institutional strengthening and strategic sector support components.

However, despite intensive investment in local stakeholder capacity building, the capacity of communities to manage water schemes remains low and their ability to carry out major repairs is limited, as corroborated by the site visits, key informant interviews and regional-level workshop. This has often resulted in limited/basic water services in areas where there was potential for higher service levels. Furthermore, it appears that the capacity-building investments made by UNICEF at the regional and local government levels were not institutionalized, as illustrated by the fact that UNICEF needed to repeatedly build capacity on a project-by-project basis.

Page 22: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 22

The evolution of UNICEF’s RWS programming towards more upstream work reflects the overall needs and changes in the sector (e.g. development of the ONE WASH National Programme) and provides an opportunity for UNICEF, together with other development partners, to play an increased role in coordination and support at the federal level.

There was consensus among the development partners interviewed that UNICEF was not as effective as it could have been with regard to knowledge management, sharing lessons learnt and developing evidence-based knowledge products to inform government and development partner policies and programmes. However, UNICEF Ethiopia did produce a significant amount of evidence over the evaluation period. Between 2006 and 2016, it notably commissioned four evaluations, three sustainability assessments, and several field surveys related to RWS, making UNICEF Ethiopia one of the country offices with the highest number of such evidence-generating activities. The corporate system of developing a management response to evaluations and tracking the implementation of evaluation recommendations demonstrates that the evidence generated by UNICEF has been used to improve UNICEF’s operations.

Stakeholders recognized that UNICEF had a lot of institutional and implementation experience but that more could have been done to disseminate some of the evidence and lessons learnt, both through the various strategic/coordination platforms and through the creation of a national-level repository clearinghouse. UNICEF’s focus on implementation (making sure funds were being spent correctly), and staffing constraints have been the main obstacles in this area. This was noted in UNICEF stakeholder interviews at the federal and regional levels. Moreover, there was no clear sense among UNICEF staff that other development partners were ‘clients’ for their evidence and lessons learnt and/or that they should work together to achieve collective best practice to inform government programmes. UNICEF staff mentioned some deficiencies in information sharing within the sector more generally, and a tendency to regard other WASH development partners as competitors was noted during interviews.

Conclusion on effectiveness: During the evaluation period, UNICEF achieved its objectives in terms of number of beneficiaries, reached its capacity-building targets, and significantly engaged in upstream work. However, more could have been done to institutionalize capacity-building efforts and to ensure the effective dissemination and management of internal and external knowledge.

E.3. Efficiency Degree to which UNICEF maximized the costs-results relationship by systematically integrating efficiency considerations into its programme design and management

Since 2010, UNICEF has worked mainly with and through government entities at the national and sub-national levels (as opposed to NGOs). This approach creates ownership and supports sustainability. However, it comes with a higher level of financial and operational risk, due to inefficiencies and delays. For example, multiple interviewees recognized that the CWA mechanism introduced additional delays because CWA-funded activities had to pass through the federal level before reaching the regional level (i.e., CWA to MoFED and then to BoFED). UNICEF took a series of measures to identify and mitigate these risks:

As part of the establishment of the SWAp (One WASH programme, WASH Implementation Framework, etc.), a fiduciary risk assessment of the government administration was conducted and a code of conduct was signed for contributors to the CWA.

Page 23: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 23

UNICEF continued to fund some activities through direct cash transfers to sub-national departments, alongside its contribution to the CWA.

Under the direct cash transfer modality, UNICEF made partial advance payments to government entities for professional services to be conducted within a few months. Under this model, the government has to account for the expenditure of these funds before receiving subsequent instalments or additional transfers. The payment, reporting and accounting processes often impeded overall progress of the work. To reduce delays, UNICEF introduced the direct payment modality to service providers, whereby government entities procured goods and services from third parties and then supervised the delivery of these services, approving payments that were made directly from UNICEF to the third party. This resulted in efficiency gains.

In some instances, UNICEF has directly procured the materials and services required for RWS interventions, bypassing federal government processes, and minimizing risks and reducing delays. During the 2000s, UNICEF had a policy of procuring spare parts (notably hand pumps) through UNICEF’s central supply depots in Copenhagen (through long-term agreements with global suppliers). In 2009/10, as local supply chains were improving, this policy changed. This helped reduce costs and supported local supply chains. Another positive step is the recent approach of clustering contracts for deep groundwater exploration as a way to reduce costs and encourage the international private sector to invest in building national capacity.

In partnership with WaterAid and SNV, UNICEF conducted capacity assessments of sub-national authorities, including a financial risk assessment of procedures,2 and facilitated the preparation of related capacity-building plans. In addition, technical assistance, and direct monitoring and quality assurance by UNICEF staff based at the sub-national/regional level have helped reduce financial inefficiencies. UNICEF staff working closely with regional and local government staff has increased staff capacity, and some stakeholders felt that one benefit of this has been an improvement in the quality of construction (this could not be independently verified).

In the earlier years of the evaluation period, UNICEF did not integrate RWS with other non-WASH programmes. More recently, however, when constructing multi-village water schemes in rural areas, UNICEF ensured that surrounding schools, health posts and health centres were also provided with piped water supply. However, in practice there have been difficulties in agreeing upon the price that institutions should pay for water – often the community is not willing to subsidize schools/health posts.

Since 2011, UNICEF has been more effective in integrating WASH projects with other sectors, including child/maternal and community nutrition, livelihood and education (e.g. community-based nutrition programme to reduce stunting in children). With its significant geographical presence through programmes in other sectors, UNICEF has a unique ability to achieve synergistic outcomes and reduce operational costs through integrated programming.

Conclusion on efficiency: Working with and through government entities and processes implies a certain level of risk. During the evaluation period, UNICEF took a series of relevant measures to minimize these risks and increase operational efficiency.

2 UNICEF conducted capacity assessments of 16 Technical and Vocational Education and Ttraining Committees: four in Amhara, two in Benishangul Gumuz, one in Gambella, four in Oromia and five in the Southern Nations, Nationalities, and Peoples’ Region.

Page 24: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 24

E.4. Sustainability UNICEF’s contribution to RWS sector strengthening and sustainable services

Based on the first national inventory of water points, only 67% of water points were functional in 2012. Following these results, UNICEF commissioned several studies that examined both the functionality and sustainability of water supply services.3 According to the 2013 sustainability assessment, the functionality rate of 44 systems supported by UNICEF and other development partners was 64% (Ripple 2013). A 2015 study found a similar functionality rate (68%) for the taps in seven towns and surrounding rural communities (Adank, 2016). 82% of households surveyed as part of this study had access to an improved source, but the service levels in the communities were very poor. However, only 10% had at least 20 litres per person per day reliably available within 500 metres of their dwelling (Butterworth, 2016). After the sustainability study of 2013, two additional sustainability assessments were carried out on UNICEF-funded systems implemented under the One WASH National Programme (2015 and 2016).4 Because of the limited statistical representativeness of the sample size, we cannot conclude from these studies whether RWS schemes constructed with direct UNICEF support and supervision are more or less sustainable than those built by the Government without UNICEF support or by NGOs.

Despite the methodological limitations of these assessments and surveys, overall, the stakeholders interviewed had a very positive perception of the various monitoring sustainability activities that UNICEF has supported or led over the years. The Government was directly involved in these initiatives. UNICEF convened a workshop on sustainability to disseminate the findings of these sustainability-related assessments to sector stakeholders. It drafted sustainability plans with local government in response to the results. It is working with the Dutch non-profit IRC to produce various academic papers and briefing notes on this topic. It is as yet unclear if/how these checks will be integrated into UNICEF’s policy advocacy agenda and guide Government efforts to improve national monitoring systems, and whether sufficient capacities have been developed within the Government to use the monitoring data (both strategically and operationally) that has been collected. Other initiatives related to sustainability monitoring supported by UNICEF include the National WASH Inventory, the pilot in Somali Region using smartphones, the global pilot of a country-level water quality study (i.e. ‘Random Assessment of Drinking Water Quality’), WASH Report Cards, Living Standards Measures Study (for setting the SDG baseline), and other monitoring initiatives. Monitoring is an important building block of sustainability, as is overall sector strengthening.

In conjunction with UNESCO, UNICEF has also made a significant contribution to the sector through the hydrogeological studies that have been carried out in three woredas in two regions (Afar and Tigray). These regions have high levels of water stress, and the proper siting and development of boreholes is a prerequisite for sustainable services. More recently, in 2015/16, UNICEF has been shifting from low to high technology, scale and service levels as part of the current country programme. Replacing the promotion of shallow sources of drinking water with the construction of higher cost, more complex piped network technology to access deeper water sources has also been a relevant shift to more climate resilient RWS programming, especially in arid and semi-arid regions. This approach is likely to reduce the vulnerability of populations in the lowlands (assuming water resources are catalogued and managed sustainably). It reflects

3 Earlier, in 2009, UNICEF had created WASH report cards in several woredas. These reported on water supply access, system functionality, use and water safety. 4 UNICEF is currently (2017) planning for two annual sustainability checks (in the dry and wet season) stratified by climate zone to investigate the impact on different rural water supply technologies.

Page 25: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 25

the increase in standards stipulated by the Government (i.e. between Growth and Transformation Plan I and II). 5

The current Country Programme Document calls for UNICEF to provide direct project implementation support to regional governments so as to focus more on climate change-affected water scarce regions. This support will come in the form of more complex and more costly technologies, such as deep wells, motorized pumps, multi-village schemes and household/yard connections, with services provided by professional service providers (as opposed to community management). This high tech and high cost approach makes sense as deep groundwater aquifers will be more resilient to climate change impacts than shallow aquifers. Furthermore, a recent life cycle cost analysis commissioned by UNICEF found that in the long run, per capita costs are lower than the costs associated with point sources and water trucking solutions. However, appropriate management arrangements and business models are needed for these more sophisticated systems to be financially sustainable, as communities and households may not be able to afford to pay higher tariffs to cover the associated operation and maintenance costs.

Over the evaluation period, the Government and UNICEF’s approach to tariff setting and recovery of operation and maintenance costs has differed between rural areas and small towns. In rural areas, the local government works with the community to determine water tariffs, based (or not) on a minimum tariff set by the regional government (at least in Tigray). As per Government policy, operation and maintenance costs, including major repair costs, are to be borne by communities. However, in practice, the tariffs are generally not set to reflect these actual costs (WaterAid, 2015). Rather than an attempt to ensure the financial viability of the service, tariffs reflect the users’ willingness to pay. In small towns, under the new One WASH+ programme that was started in 2013, willingness and ability to pay studies were carried out with UNICEF support, and this information was used to inform the business plans for the water operators. This approach was subsequently not transferred to rural areas.

Moving forward, if tariffs are increased as a result of the promotion of more sophisticated, higher cost systems, and if service providers do not have a clear understanding of the financing needs and access to the required credit/finance, then it may not be possible to ensure the long-term financial viability of schemes. During the period under review, UNICEF did not conduct enough/any advocacy to increase tariffs.6 In general, the primary focus within the RWS sector during the evaluation period was to develop the service rather than ensure its long-term sustainability. Financing of direct support to community management committees and other operators (usually provided by both local government and/or the local private sector), and capital maintenance and replacement costs both constitute major gaps.

From the field visits and regional-level stakeholder interviews, it appears that the management models used for the multi-village schemes developed (at least in Tigray) were largely based on the community management model (WASHCo) used for small reticulated schemes. Given the increased complexity of the multi-village schemes, the service delivery model is inadequate without further institutional support. The limited capacity of service providers jeopardizes the sustainability of services over time. UNICEF has supported town utilities/municipalities in expanding their operations to surrounding rural areas to showcase a new management model in rural areas.7 The scalability of the model eventually selected to replace community-based

5 More recently, at the upstream level, UNICEF has been the key partner supporting the development of a new Climate Resilient WASH strategy (endorsed at the end of 2017), which will form a key pillar for Phase II of the OWNP. 6 UNICEF is now advocating for a change to the Government’s policy that all maintenance, including major repairs, must be borne by the community in rural areas. 7 UNICEF is now (2017) supporting the establishment of rural public water utilities in Ethiopia, as an alternative management model for the country context.

Page 26: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 26

management will require different capacities at all levels (e.g. implementation, technical support, monitoring, etc.). UNICEF started working with formal service providers in small and medium towns, and can learn from this experience.

During the evaluation period, UNICEF support to the participation of beneficiary communities in RWS interventions in the field was relatively effective. The woreda governments oversee the choice of technology, design and construction of the systems. The community has not necessarily been involved in the choice of technology, however, the choice is relatively limited and dependent on the technical restrictions of the site (i.e. the depth to groundwater and the available power source). Community representatives have been involved in the design and preparations phases, following the government's protocols. During the evaluation period, there was a shift away from leveraging in-kind contributions for projects. As UNICEF is moving towards more heavily engineered technology choices, the role played by the community will have to evolve alongside, with less focus on direct management and maintenance, and a greater role in holding the service operators/providers to account.

UNICEF’s investment in building the capacities of public and private sector institutions and individuals has been laudable, as highlighted earlier. The capacity-building efforts included a training of trainers on operation and maintenance practices, which was given to government officials at the federal and regional levels.8 A large gap still remains, however. Human resource capacity at the woreda level and below is a critical constraint to sustaining rural water systems. Current estimates indicate a shortfall of some 40% across all technical cadres, from artisans and water technicians to professional engineers, equating to some 47,000 people. The CWA financing/interventions have included capacity-building support activities. Currently, around 6% of the CWA budget is for capacity building, post-construction support, support to supply chain development and other long-term service delivery activities. These have been largely one-off events rather than part of a systematic support and long-term strategy.

Conclusion on sustainability: Over the past 10 years, UNICEF has focused more on constructing new hardware. Some key building blocks of sustainability have not been addressed during this period, such as financial and institutional sustainability.

E.5. Equity UNICEF’s consideration for the population groups most in need in its geographical and beneficiary targeting, in programme design and implementation (both downstream and upstream) and in monitoring and evaluation

The Government’s approach sought to ensure that funds for RWS were shared equitably across all the country’s regions with a focus on those lagging behind in terms of RWS coverage and vulnerable to climate change. This was based on a formula for geographical targeting that takes into consideration the region’s population and current RWS access. This formula also takes into account areas with scarce water resources and the degree of vulnerability at the community level. UNICEF’s RWS programming was in line with this approach. Over the course of the 2012–2016 country programme, a large part of UNICEF’s contribution to water supply was channelled to regions with the lowest coverage, such as Afar, Somali, SNNPR, Oromia and Amhara, as well as to arid regions that were not well served by other development partners (see Figure 5).

8 More recently, UNICEF has provided significant technical support to the MoWIE to develop/update an operation and maintenance manual for rural piped water schemes. This guideline was completed in September 2017 and is currently being rolled out to the entire country through training workshops.

Page 27: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 27

Resources were also spent in areas with relatively higher coverage (Gambella, Tigray, Benishangul), which may have been due to other types of vulnerabilities.

Source: UNICEF, Evaluation of UNICEF WASH country programme 2012-2016, 2016

While this geographical targeting approach aims to address inequalities at the regional, woreda and community levels, it does not ensure that inequalities are addressed at the household level. As noted earlier, JMP data shows that while the coverage for the poorest rural quintile increased over the past 10 years, inequality has grown. Given that UNICEF has been a (if not the) major contributor to RWS over this period, this raises the question as to whether UNICEF has put in place appropriate approaches and measures, both in its downstream and upstream work, to ensure the poorest households get access to the service. Although UNICEF is bound, to a certain extent, by the prioritization processes used by the Government when working through regional/local government, it is still possible to influence these processes to ensure that UNICEF’s institutional mandate is met. Noting that 99% of RWS systems in rural areas provide services accessed through public taps/ community water points (JMP 2015), it is also the responsibility of the community WASHCo to ensure that services reach the poor and other disadvantaged households.

In 2009, UNICEF, together with all other United Nations organizations and the Government, took part in a gender audit. An action plan and guidelines were subsequently developed to address WASH-related gender issues. UNICEF is recognized as a leader in adopting and promoting a gender lens in the RWS sector. An example of its work is the bill assigning legal accountability to local WASHCos, including the mandatory presence and roles for females in these committees.

Little to no evidence was found that UNICEF has advocated for measures targeting other vulnerable, marginalized or disadvantaged populations. Guidance was not provided to ensure poor, elderly, disabled people, transient populations, ethnic or religious minorities were represented in community-based WASHCos. Reduced tariffs and cross-subsidy arrangements for them have not been promoted. Decisions regarding these vulnerable sub-groups have been

Figure 5: UNICEF WASH country programme (2012–2016) water supply

Page 28: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 28

left to the WASHCos. Moving forward, this will be an issue as the SDG agenda includes ensuring accessibility and affordability of the service for all, leaving no one behind.

During the evaluation period, UNICEF did not systematically collect disaggregated data to track marginalized groups. None of the evaluations commissioned during the evaluation period included a dedicated section on and analysis of equity. UNICEF should, at a minimum, try to develop evidence to inform decision-making and other sector stakeholders. Lately, UNICEF has been working in coordination with consultants appointed by DFID to design the WASH Monitoring Information System (which may or may not include equity-related indicators).

Conclusion on equity: UNICEF’s geographical targeting has been equity-lensed. The organization’s role in promoting gender equity is recognized and documented. Areas of weakness include lack of directive given to ensure the poorest and most vulnerable households are included in WASHCos and that tariffs are affordable for them. There is also a lack of equity-lensed monitoring and evaluation.

E.6. Innovation UNICEF’s introduction of innovation and scale up, in particular in the following areas: innovative financial mechanisms; private sector participation in the management of RWS services; real-time monitoring; sector accountability; and regulation arrangements

Several initiatives within UNICEF and within Ethiopia’s RWS sector are considered to be innovative. Focusing on the four areas of innovation that this evaluation looked at, these initiatives are:

Involvement in the SWAp and pooled funding mechanism (CWA);

Piloting of real-time inventory/mapping/monitoring of water points;

Introduction of sustainability checks/assessments; and

Use of remote sensing technologies, combined with traditional hydrogeological study techniques, to map large areas of the country. This study was only recently completed and it is expected that it will help develop water resources. In addition, it is critical to helping the Government understand and manage its water resources.

There is limited evidence that innovations have been replicated and scaled up by the federal and regional governments. The potential for scaling up successful innovations could be increased were UNICEF to adopt a more strategic and powerful policy advocacy and mainstreaming agenda.

During the period under review, no innovation was identified in the following areas: management models for RWS services; involvement of the private sector; and accountability and regulation mechanisms. There is no independent regulator of water services for rural areas and small towns, which is a barrier to consumer and service provider performance and accountability. Regulatory functions provided by local government are weak at best. Recently, UNICEF launched an initiative to support the establishment of rural public water utilities in Ethiopia (a new management model for the country context), and the establishment of the country’s first national regulator for the water sector, which will assess and support both urban and rural water utilities.

Conclusion on innovation and scaling up: UNICEF has introduced some innovative approaches in Ethiopia in the areas of financing mechanisms and (real-time) mapping (sustainability monitoring), which were replicated by other agencies and/or taken to a larger scale. There is potential for innovations in other areas such as RWS management models and regulation and accountability mechanisms.

Page 29: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 29

F. Recommendations to UNICEF

The Ethiopia RWS sector and its enabling environment have evolved quickly during the past 10 years. The MDG target has been met. The SDGs now set a new, even more ambitious agenda for the country with a 2030 horizon. The objective of boosting the coverage rate will continue to drive the sector, but additional challenges will increasingly draw the attention of all stakeholders, including: improving the level of service (in terms of quantity, accessibility, reliability and quality), ensuring the financial viability and affordability of the service, and reaching the most remote and vulnerable communities and households.

The WASH strategy in the current UNICEF country programme involves both upstream sector strengthening work and downstream implementation using higher technology and higher cost solutions to improve the level of service and build resilience to climate change.

The following recommendations are based on both the findings of this case study and the wider context of the SDGs, the RWS sector and the UNICEF country programme. These recommendations should be considered as a menu of options rather than a plan of action. The most appropriate options for UNICEF will depend on numerous factors including the dynamics in the sector, the wider strategic vision of the UNICEF country office regarding its assistance to the country, the resources at its disposal, the positioning of other RWS donors and stakeholders, and UNICEF’s analysis of its own comparative advantages and added value in the sector.

F.1. Continue to adapt its RWS programming, with increased linkages between downstream and upstream work

1. The high technology/high cost approach chosen by UNICEF will likely reach fewer direct beneficiaries than over the past 10 years (for the same budget). Yet the Government needs to double the rate of increase in RWS access to reach the SDGs. Therefore, UNICEF should use its direct implementation projects as opportunities for testing innovations and as learning laboratories to document/evaluate what is working and what needs to be improved in water scarce areas, and to inform its policy advocacy and upscaling/mainstreaming efforts. This will strengthen the linkages between UNICEF’s downstream and upstream work, and help the organization increase its impact on the RWS sector with limited funding – thus maximizing the value for money of its programming.

This approach implies:

Designing field interventions with an innovation lens and a clear research/evaluation agenda, based on a well formulated theory of change.

Continuously refining the intervention design based on research/evaluation findings and lessons learnt, with the view to developing models of RWS services that work for various contexts.

Documenting/publishing the results and disseminating them throughout the sector. This means ensuring that those who require this information are able to access it and know how to effectively use the data for decision-making, prioritization, policymaking, etc. UNICEF is recognized globally and within Ethiopia for its contribution to learning and knowledge, but its full potential in this area has not been realized in Ethiopia. UNICEF

Page 30: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 30

should ensure that the research/evaluation process used is fully understood and can be replicated (i.e., sharing not just the results but also the cleaned data sets and methods used to collect, analyse and present the information).

Advocating to the government and other sector stakeholders for the replication, scaling up, mainstreaming and institutionalization of successful models and innovations, in a coordinated effort with its partners.

In doing so, UNICEF should focus on the key bottlenecks and challenges identified internally and in the RWS sector (at the field, sub-national and national levels). This approach requires UNICEF to use its considerable human resources in a more analytical and upstream manner.

The findings of this country case study indicate that sustainability and equity are two priority areas of work, both for UNICEF and the broader RWS sector. More specific recommendations related to these two topics are proposed below.

F.2. Further support the sustainability of RWS services

In order to strengthen the sustainability of RWS services, UNICEF should (continue to) work with the Government and other RWS sector stakeholders to:

2. Develop/validate/implement a long-term financing strategy for water services that determines an appropriate planning, mix and allocation of tariffs, government budget, and external aid to ensure that both operations and maintenance and capital replacement costs are covered. Such a strategy should encompass the various types of water services/technologies and management models. UNICEF should determine how tariffs can better contribute to the viability of the service while remaining realistic (depending on users’ ability to pay) and affordable to the poorest (e.g. through cross-subsidy mechanisms within or between communities/schemes).

3. Continue to innovate, test and document/evaluate alternative RWS service delivery/management models, including the establishment of rural water utilities currently supported by UNICEF, private sector participation arrangements, and multi-use water service models (i.e. for residential, industrial and agricultural purposes), particularly for multi-village and small town piped water schemes.

4. Institutionalize capacity-building efforts to ensure training and technical support opportunities are available to all stakeholders (especially WASHCos, small-scale private service providers, woredas, and zonal and regional water bureaus) on a continuous basis, i.e. not only during project implementation but also during the post-implementation phase. This could be funded through the CWA, for example.

5. Continue current efforts to improve the quality of RWS services through improved regulation.

At the community/service level: Depending on the type of service and the management model, communities should be officially recognized and appropriately trained either for operating/managing point sources or for providing feedback on service performance and holding the water operators to account (for piped water schemes).

At the sector/central level: As the RWS sector progresses toward a higher level of service, more piped systems, and possibly the establishment of public water utilities,

Page 31: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 31

there is an opportunity and a need for instituting an adequate performance monitoring and regulation mechanism. UNICEF’s recent advocacy for the establishment of a regulation agency for urban and rural water utilities seems to be in the right direction. Lessons can be learnt from other development agencies (e.g. World Bank) and countries (e.g. Rwanda, Mauritania, and Mozambique).9

6. Support the supply side through the promotion of local construction and manufacturing capacities, including through the combined procurement of spare parts and handpumps in areas where hand pumps are a sustainable solution, in order to motivate private suppliers.

7. Put systems and capacities in place to ensure that water resources are effectively and efficiently managed. This is in light of the shift to more climate-resilient water supply services, which could have a greater impact on water resources. This is also the focus of

several SDG targets, most notably targets 6.4 and 6.5.10 To this end, UNICEF should work

with the Government to:

improve integrated water resource management capacities and the effective planning and management of future rural water supply schemes; and

ensure that the WASH sector monitoring and information system includes water resource management (or is linked to an integrated water resource management-specific monitoring and information system).

8. Institutionalize periodic, sector-wide sustainability checks/assessments, which are led by the Government so it owns the process and plays a leading role in conducting them. The results should be used to inform decision-making.

F.3. Increase focus on equity issues

In order to strengthen equity in the RWS sector and services, UNICEF should continue to work with the Government and other RWS sector stakeholders to:

9. Ensure that all groups (including underprivileged groups) are taken into account and involved in RWS planning, implementation and management processes, at both the sector and field implementation levels. This is likely to become more critical as the move towards more highly engineered systems will probably result in an increase in top-down approaches in the sector, driven by technologies and financial resources. UNICEF has an important role to play in providing the Government with evidence and advice on how to fulfil the equity agenda set by the SDGs. For example, by:

Documenting inequities in access within communities (i.e. intra-community service levels) as well as pro-poor tariff arrangements; and

Assessing and reinforcing the identification and prioritization processes for site selection with the view to better addressing the needs of the poorest, most remote and marginalized populations. This applies to selecting geographic areas, communities and

9 See also arrangements described in the UNICEF Headquarters document, ‘Accountability for sustainability: Reference Guide for WASH Programming’, and accompanying tools published in 2015/16. 10 Target 6.4 calls for substantially increasing water-use efficiency across all sectors, ensuring sustainable withdrawals and supply of freshwater to address water scarcity and substantially reduce the number of people suffering from water scarcity. Target 6.5 advocates for applying integrated water resource management at all levels of government.

Page 32: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 32

specific locations within communities. The recently endorsed Climate Resilient WASH Strategy is a significant step in this direction: it focuses on arid and semi-arid regions and is therefore likely to increase the resources allocated to these vulnerable and neglected areas.

10. Document and disseminate among the government and development partners the lessons learnt on low technology/low cost solutions, and how they can particularly benefit the poorest and most remote communities that cannot access/use more sophisticated (and costly) water schemes. It will be important for some other institution or partner to fill any gaps left by UNICEF’s change in approach.

11. Improve the sustainability checks/assessment methodology and the sector monitoring and evaluation system by including a stronger equity lens. Sufficient data needs to be generated on a regular basis in order to monitor progress/impacts on poor, remote and marginalized groups, so that programmes can be adjusted to effectively target these groups. This implies disaggregating data to examine access and use issues from various perspectives (e.g. the elderly, poor, single parent households, disabled, ethnic minorities, etc.). Currently, the data used for access estimates comes from household surveys (i.e. MICS, DHS and other Government surveys), which are not sufficiently disaggregated.

Page 33: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 33

G. Annexes

G.1. Timeline of the country visit Date Activity Time frame Who

Monday, 31 October 2016 (Morning)

Security briefing 09:30-10:30 Ryan Schweitzer

Orientation/introduction to UNICEF WASH staff and review of agenda 10:30-11:00 Ryan Schweitzer, Yemarshet Yemane, Getachew Hailemichael

Meeting with senior management (Ryan Schweitzer to prepare 5-6 slide overview of evaluation)

11:00-11:30 Ryan Schweitzer, Yemarshet Yemane

Meeting with WASH section (Ryan Schweitzer to prepare 10-15 slides with details on scope of evaluation, methods, outputs etc.)

11:30-12:00 Country Representative, Deputy Representative

Overview presentation of the UNICEF RWS portfolio (2006–2015) 12:00-12:45

Discussion 12:45-13:30 WASH section (Getachew Hailemichael, Jane Bevan, Michele Paba, Kalkidan Gugsa)

Monday, 31 October 2016 (Afternoon)

Bilateral meeting with WASH team (30 minutes x 4 people) 14:30-16:30 WASH section (Getachew Hailemichael, Jane Bevan, Michele Paba, Kalkidan Gugsa)

Finalize agenda and any other logistics issues 16:30-17:00

Tuesday, 1 November 2016 Semi-structured interviews with representatives of the MoWIE AM Ryan Schweitzer, Yemarshet Yemane

Tuesday, 1 November 2016 Semi-structured interviews with representatives of the partners of the projects that will be visited during the field trips.

PM

Wednesday, 2 November 2016

Semi-structured interviews with UNICEF staff AM Eddy Perez, Ryan Schweitzer, Yemarshet Yemane

Wednesday, 2 November 2016 (Afternoon)

Safety debriefing (security clearance) 12:00-13:00 Eddy Perez

Travel to Mekele

Eddy Perez, Ryan Schweitzer, Yemarshet Yemane, Getachew Hailemichael

Page 34: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 34

Date Activity Time frame Who

Meeting with Tigray Region water bureau 14:30-16:00

Semi-structured interviews with UNICEF staff 16:00-17:30

Thursday, 3 November 2016 (Morning)

Group 1: Field visit to Raya Azebo woreda AM

Ryan Schweitzer, Yemarshet Yemane, Leul Fesseha Semi-structured interviews with woreda staff

Group 2: Field visit to Kilte Awelaelo woreda

AM Eddy Perez, Getachew Hailemichael, Getachew Asmare

Semi-structured interviews with local stakeholders and beneficiaries

Semi-structured interviews with WASH sector

Thursday, 3 November 2016 (Afternoon)

Group 1: Site visit to two randomly selected kebeles

PM Ryan Schweitzer, Yemarshet Yemane, Fesseha

Semi-structured interviews with community leaders, WASHCo and beneficiaries

Observation of infrastructure

Group 2: Field visit to small town water supply project Wukro PM

Eddy Perez, Getachew Hailemichael, Getachew Asmare Travel back to Mekele

Friday, 4 November 2016 (Morning)

Group 1: Travel to Emba Alaja woreda AM

Ryan Schweitzer, Yemarshet Yemane, Fesseha Semi-structured interviews with woreda staff

Group 2: Travel to visit RWS (around Kilte Awelaello or nearby woreda)

AM Eddy Perez, Getachew Asmare Semi-structured interviews with local stakeholders and beneficiaries

Semi-structured interviews with WASH sector offices (NGOs)

Friday, 4 November 2016 (Afternoon)

Group 1: Site visit to two randomly selected kebeles

PM Ryan Schweitzer, Yemarshet Yemane, Fesseha

Semi-structured interviews with community leaders, WASHCo and beneficiaries

Observation of infrastructure

Drive back to Mekele

Group 2: Site visit to two randomly selected kebeles

PM Eddy Perez, Getachew Asmare Semi-structured interviews with community leaders, WASHCo and beneficiaries

Observation of infrastructure

Page 35: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 35

Date Activity Time frame Who

Drive back to Mekele

Saturday, 5 November 2016 (Morning)

Sub-national workshop at Mekele for about 30 people

09:00-14:00 See list of participants With implementing partners (NGOs, private sector), local governments, community-level organizations

Travel back to Addis Ababa in the evening

Sunday, 6 November 2016 Data collation and analysis All day Eddy Perez, Ryan Schweitzer, Yemarshet Yemane

Monday, 7 November 2016 Semi-structured interviews with stakeholders from CoWASH, IRC, Regional Bureau of Water-Oromia

All day Eddy Perez, Ryan Schweitzer, Yemarshet Yemane

Tuesday, 8 November 2016 Semi-structured interviews with SNV All day Eddy Perez, Ryan Schweitzer

Wednesday, 9 November 2016

Semi-structured interviews (World Bank, DFID) All day Eddy Perez, Ryan Schweitzer

Thursday, 10 November 2016

National-level workshop (stakeholders to be defined)

Semi-structured interview with UNICEF WASH Section Chief

09:00–16:00

16:00–17:30

See list of participants

Eddy Perez, Ryan Schweitzer, Yemarshet Yemane

Friday, 11 November 2016

Brainstorming on main observations and findings, preparation of wrap-up meeting

09:00–10:00

10:00–12:00

Ryan Schweitzer, Eddy Perez, Yemarshet Yemane

WASH section staff Feedback meeting on evidence collected during the visit, preliminary findings and discussion

Saturday, 12 November 2016 (Morning)

Ryan Schweitzer and Eddy Perez fly out early morning

Page 36: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 36

G.2. Semi-structured interview key informants

# Name Level Organization Responsibility

1 Nuredin Mohamed National MoWIE Director, Water Supply and Sewerage Directorate

2 Tamene Hailu National MoWIE Manager, National WASH Inventory project

3 Belay Seyoum National MoWIE UNICEF focal point at MoWIE/coordinator of National Fluorosis Mitigation Project Office

4 Mesfin Mulugeta National Water Sector Working Group Secretariat

Coordinator

5 Dr Zufan Abera National MoH Director, Health Extension and Primary Health Service Directorate

6 Wolday Zewde National MoH Hygiene and environmental health expert (sanitation market and CLTS)

7 Abayew Mussei National MoH Hygiene and environmental health expert

8 Abreham Misganaw

National MoH Environmental health case team

9 Ababu Tadesse National MoFEC WASH focal person

10 Martha Solomon National DFID WASH advisor

11 Teferi Menker National AfDB Programme Manager

12 John Butterworth National IRC-Netherlands Country Director

13 Tesfaye Bekalu National WSP/World Bank Senior water and sanitation specialist

14 Gulilat Berhane National WSP/World Bank Senior water and sanitation specialist

15 Arto Suominen National COWASH-Finland Programme Manager

16 Getachew Fekadu Sub-national Tigray Water Bureau Bureau/Water Supply Core Process Successor

17 Tamene Mengistu Local Kilite Awlalo woreda water office, Tigray Region

Office Head or Deputy

18 Hayelom Hadefom Local Raya-Azebo woreda water office, Tigray Region

Engineer and Water Supply Head

19 Aberha Berhanu Local Raya-Azebo woreda water office, Tigray Region

Electromechanical technician

20 Mola Haddis Local Raya-Azebo woreda education office, Tigray Region

Plan expert

21 Tsehay Seid Local Raya-Azebo Accountant

Page 37: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 37

# Name Level Organization Responsibility

Woreda finance office, Tigray Region

22 Godif Aklom Local Amba Alaje woreda water office, Tigray Region

Office Head

23 Frew Sisay Local Amba Alaje woreda water office, Tigray Region

Engineer and Water Supply Head

24 Meri Kebede Local Amba Alaje woreda water education office, Tigray Region

Head

25 Nure Hussein Local Amba Alaje woreda water finance office, Tigray Region

Accountant

26 Samuel Godfrey National UNICEF Ethiopia WASH Section Chief

27 Jane Bevan National UNICEF Ethiopia Rural WASH Manager

28 Ali Regah National UNICEF Ethiopia WASH specialist-Rural

29 Michele Paba National UNICEF Ethiopia Urban WASH Manager

30 Kalkidan Gugsa National UNICEF Ethiopia WASH specialist-C4D

31 Clara Dube Sub-national UNICEF Tigray Manager, field office

32 Getachew Asmare Sub-national UNICEF Tigray WASH specialist

33 Leul Fesseha Sub-national UNICEF Tigray WASH officer

G.3. List of sites visited during field work

Woreda Kebele Site/Village

Wukro Wukro Town n/a

Tahtay Maychew Tahatay n/a

Embaalji Woreda Ayba Kebele Kresher Adihabi village

Embaalji Woreda Betemera Ketema/town n/a

Raya Azebo Woreda Hurta Kebele Melheeso village

Raya Azebo Woreda Ebo Kebele Sheshehro village

G.4. Workshop participants

G.4.1. Sub-national workshop

A total of 45 participants (3 women, 42 men) attended the sub-national workshop.

Participants included representatives from 3 regions, a public sector enterprise, a public utility, 29 representatives from 9 woredas (7 individuals involved in the implementation of the UNICEF 2006–2015 country programme and 2 involved in the current country programme), and 11 WASHCo members from 3 kebeles.

Page 38: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 38

Page 39: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 39

G.4.2. National workshop

Name Organization

Arto Souminen COWASH

Wondeayehu Webe Ministry of Health

Kassahun Beyene Ministry of Water Irrigation and Electricity (MoWIE)

Lakech Haile MoWIE

Belay Seyum MoWIE - UNICEF focal person

Getachew Abdi Private sector (Coffey)

Andualem Anteneh SNV

Jane Bevan UNICEF

Samuel Godfrey ` UNICEF

Abebe Ketema UNICEF

Gezahegn Laurecha UNICEF

Michele Paba UNICEF

Ali Regah UNICEF

Bethelehem Mengistu WaterAid Ethiopia

Waltaji Terfa Kutane WHO

Page 40: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 40

G.5. List of documents reviewed

G.5.1. UNICEF documents

United Nations Development Assistance Framework (UNDAF), 2007–2011

UNDAF, 2012–2015

UNDAF, 2016–2020

UNDAF Action Plan, 2012–2015

UNDAF Monitoring and Evaluation Plan Volume II, 2012–2015

UNICEF Country Programme Action Plan 2007–2011 and annexes

UNICEF Country Programme Document 2012–2015

UNICEF Country Programme Document 2016–2020

UNICEF Country Programme Document results matrix, 2012–2016

G.5.2. Proposals

UNICEF, ‘Community-based Nutrition Programme’, 2010

UNICEF, ‘DfID WASH Business Case’, 2013

UNICEF, ‘Integrating WASH, Multiple Use Services and Community-based Nutrition for Improved Food Security and Reproductive and Sexual Health’, proposal to the Government of the Netherlands, 2011

UNICEF, ‘The Netherlands and UNICEF WASH Initiative’ (NUWI), proposal for funding, 2006

UNICEF, ‘Water, Sanitation and Hygiene Capacity-building Project’, proposal to the Government of Finland, 2010

G.5.3. Reports

UNICEF, Capacity-building Programme Final Report, 2014

UNICEF, Atsbi Hydrogeological Final Report, 2015

UNICEF, Elidar Hydrogeological Final Report, 2015

UNICEF, Erebti Final Report, 2015

UNICEF, Annual Reports to the Government of the Netherlands: 2015, 2016

UNICEF, Acceleration of Water Supply, Sanitation and Hygiene towards Ethiopia’s MDGs, Final Report to the Government of the Netherlands, 2010

UNICEF, WASH Capacity-building Programme Reports: 2012, 2015

UNICEF, Baseline survey NUWI, 2013

UNICEF, Final Report Evaluation of the Netherlands Water Initiative (NUWI), 2010

UNICEF, Wet Nutrition 2013 Baseline Survey Report, 2015

UNICEF, Learning Notes – One WASH Plus Program [e.g. Build Capacity–Build Transfer (BCBT), Sustainability]

UNICEF, Woreda selection criteria for the new country programme, 2012–2015

Page 41: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 41

UNICEF, Strategy Note, Supporting government capacity and systems to plan, manage and monitor integrated and sustainable decentralized WASH services in 86 learning woredas, 2012

UNICEF, Evaluation of the UNICEF Ethiopia Water, Sanitation and Hygiene (WASH) Country Programme Document

G.5.4. Key WASH documents

a) Policies/Strategies

Government of Ethiopia, Health policy of Ethiopia, Addis Ababa, 1993, www.cmpethiopia.org/page/301

Government of Ethiopia, WASH Implementation Framework, Addis Ababa, March 2013, www.cmpethiopia.org/page/301

Government of Ethiopia, One WASH National Program (OWNP), Addis Ababa, August 2013, www.cmpethiopia.org/page/562

Ministry of Finance and Economic Development (MoFED), Growth and Transformation Plan (GTP) 2010–2015, Addis Ababa, September 2010, www.ethiopians.com/Ethiopia_GTP_2015.pdf

Ministry of Health, National Drinking Water Quality Monitoring and Surveillance Strategy, Addis Ababa, May 2011, www.cmpethiopia.org/page/301

Ministry of Water Resources, Ethiopian Water Sector Strategy, Addis Ababa, 2001, www.cmpethiopia.org/media/ethiopian_water_sector_policy_2001

Ministry of Water and Energy, Rural water supply Universal Access Plan (UAP I), Addis Ababa, December 2011, www.cmpethiopia.org/page/301

Ministry of Water and Energy, National hygiene & sanitation strategic action plan (UAP II), Addis Ababa, June 2011, www.cmpethiopia.org/page/301

b) Guidelines, Standards and Norms

Ministry of Water and Energy, ‘Gender Mainstreaming Guidelines and Checklists for the Water Sector’, Addis Ababa, 2001, www.cmpethiopia.org/page/301

Ministry of Water and Energy, ‘Region Specific Supply Chain for Hand Pump and Spare Parts in Ethiopia Guideline’, Addis Ababa, May 2010, www.cmpethiopia.org/page/301

Ministry of Water, Irrigation, and Energy, ‘WASHCO Legalization Guideline’, Addis Ababa, 2012, www.cmpethiopia.org/page/401

Ministry of Water, Irrigation and Energy, ‘Community Generic CMP Investment Fund Management Guideline’ (final draft), Addis Ababa, 7 March 2012, www.cmpethiopia.org/page/224

Ministry of Water Resources, ‘Gender Mainstreaming Field Manual’, Women’s Affairs Department, Addis Ababa, December 2005, www.cmpethiopia.org/media/gender_mainstreaming_field_manual_2005

National WASH Coordination Office, ‘WASH M&E Framework and Manual Version 1.0’, Addis Ababa, www.cmpethiopia.org/page/301

National WASH Coordination Office, ‘National WASH Coordination Office ToR’, Addis Ababa, 2009, www.cmpethiopia.org/page/301

G.5.5. Other resources

Aboma, G. and S. Gobena, Lifecycle Cost and Affordability of Communal Rural Water System: Evidence from selected local governments, WaterAid Ethiopia Report, 2013.

Adank, M., ‘Measuring factors that predict if WASH services are sustainable’, 2015, www.unicef.org/ethiopia/ECO_Learning_note_measuring_factors_that_predict_if_WASH_services_are_sustainable.pdf

Page 42: Global Evaluation of UNICEF’s Drinking Water Supply ......This country case study report is a component of the ‘Global Evaluation of UNICEF’s Drinking Water Supply Programming

HYDROCONSEIL-AGUACONSULT-CGSW

UNICEF – Global RWS evaluation: Ethiopia country case study Page 42

Alemu, G., et al, ‘Assessment of budget in the water sector: a case study of two selected woredas in Oromia Regional State (Babile and Goro-Gutu woredas)’, RiPPLE Working Paper 19, RiPPLE, Addis Ababa, 2010, http://r4d.dfid.gov.uk/Output/186296/

African Ministers' Council on Water (AMCOW), Water Supply and Sanitation in Ethiopia Turning Finance into Services for 2015 and Beyond, Water and Sanitation Program, World Bank, Washington DC, 2010, www.wsp.org/sites/wsp.org/files/publications/CSO-Ethiopia.pdf

Butterworth, J., ‘Findings of independent monitoring and review’, 2017, Powerpoint presentation available at www.ircwash.org/sites/default/files/irc_-_findings_of_independent_monitoring_and_review.pdf

Butterworth, J., et al, ‘Monitoring access to rural water supplies in Ethiopia’, National WASH Inventory seminar: lessons learned and maximising value, Addis Ababa, 8 April 2013.

Federal Democratic Republic of Ethiopia, ‘One WASH National Program: A Multi-Sectoral SWAp’ (final programme document), August 2013.

One WASH National Program Consolidated WASH Account, 2nd Quarterly Report, March 2016.

Tadesse, D., et al, Rapid Assessment of Drinking Water Quality in the Federal Democratic Republic of Ethiopia, World Health Organization, Geneva, 2010, www.wssinfo.org/fileadmin/user_upload/resources/RADWQ_Ethiopia.pdf

WHO/UNICEF, ‘Joint Monitoring Program Update’, World Health Organization, Geneva, 2011, www.wssinfo.org/