Www.3ieimpact.org Hugh Waddington What works in WASH? Evidence from systematic reviews Hugh...

www.3ieimpact.orgHugh Waddington

What works in WASH? Evidence from systematic

reviews

Hugh Waddington

Geneva Evaluation Week

7 May 2015

International Initiative for Impact Evaluation


3ie is managing 3 linked work streams

Impact evaluations•Funding available for 2 studies•RFQ (2-stage awards)•Review of proposals from June 7th

•Presentation of baseline findings in Kathmandu (Nov-Dec 2015)•Draft report in December 2016

Mid-term review•WSSCC medium-term strategy•IFMR contracted•Draft report by January 2016

Systematic reviews•2 questions•Call closes May 12•Contracting in June•Draft reports in 2016


Call for SR proposals: deadline 12 May

2 questions developed with WSSCC and 3ie-WSSCC programme advisory group

•What is the effectiveness of interventions aiming to promote sanitation and hygiene behaviour change in communities?

•To what extent has the sanitation and hygiene sub-sector taken into account the life-cycle approach in the design, implementation, maintenance and use of programmes during the MDG period?


“Literature reviews are like

sausages... I don’t eat

sausages as I don’t know

what goes into them”

Dean Karlan


3ie Evidence gap map

gapmaps.3ieimpact.org


A sector-wide logic model

Inputs Activities Outputs Outcomes Impacts

WASH sector programmes:

Improving operator performance

Private sector delivery, contracting out

Pricing reforms e.g. subsidies

1.1 1.2

1.3 Decentralized delivery (e.g. CDD)

Water supply services

Water supply facilities

Better, more reliable access

Time use, reduced

travel time

Improved child nutritional

status, safety Reduced

mortality

Better educational attainment

Higher income and consumption

Reduced costs of health care

Poverty reduction

Happiness

1.4 Information and education communic-ation (IEC) services

Water treatment

Better quality water: less fecal contamin-ation in water source

1.5 Improved water use practices: less fecal contamin-

ation in drinking water

1.6 Reduced morbidity, and better maternal

and child health

outcomes

Behavior change campaigns (BCC) about water use

BCC about sanitation practices

1.7 1.8

1.9 Access to safe sanitation

Improved sanitation practices: less fecal contamin-

ation in environment

1.10

Sanitation services Sanitation

facilities User

satisfaction Empowerment:

dignity, safety


Key evidence gaps• 138 Impact and 26 SRs

of WASH evidence in low and middle income countries.

• Mostly measure diarrhoeal health outcomes (not very rigorously)

• Very few studies examining demand-side approaches (eg CLTS, san marketing)

• Few estimate impacts for sanitation and hygiene programmes as part of scale up

• No rigorous prospective studies of sanitation programmes in SSA

• Few on governance


3ie systematic reviews

BEHAVIOURAL CONTEXT (Pre-project period)

Individual characteristics and abilities: Age, gender, pre-existing WASH practices, knowledge, skills, self-efficacyHousehold demographics: Socioeconomic status, household structure, religionCommunity/socio-political structure: Social norms, access to WASH technologies and materials, health system structure, governmental policies and financial support

Early introduction

Communities and individuals are first introduced to WASH technologies and promotion messages. • May have previously been exposed, or may be new to campaigns.• WASH technologies or messaging may be new or exciting• Promoter may be viewed as an influential person

Maintenance during project period

Promotion continues to end of intervention period. • Health promoters assist with problems• Participants able to practice behaviours independently

Continued use (short-term) after end of project period

Intervention period ends. Participants practice WASH behaviours without study support. • Messaging and education still “fresh” in participants’ minds• Lack of reminders from promoter visits

Sustained adoption (long-term) after end of project period

Ranges from 6 months to years after project period ends. Captures true behaviour practice without influence of intervention promotion.• Habitual behaviour established• New families unfamiliar with WASH interventions may not practice

behaviour• Cost may be unsustainable

PRO

JECT

PER

IOD

POST

PRO

JECT

PER

IOD

WASH Technology Knowledge/skills of WASH behaviour

www.3ieimpact.orgHugh Waddington9

Methodology

• Extensive search of published and unpublished sources (updated in 2012-13)– PubMed, Embase, LILACs, Web of Science – JOLIS, IDEAS, British Library for Development Studies (BLDS),

Cochrane Library, scholar.google– Personal communication with leading researchers– Hand-search, back-referencing and citation tracking

• Inclusion criteria: – IEs measuring impact of intervention on diarrhoea morbidity

using experimental (RCTs) and quasi-experimental methods– reported specific water, sanitation, and/or hygiene

intervention(s); – were conducted in low- or middle-income countries; – use an infant or child as the unit of observation; and – estimate impact on diarrhoea morbidity, measured under non-

outbreak conditions.


WASH systematic reviews search process74,181 records identified through database search

Scre

enin

gIn

clud

edEl

igib

ility

Iden

tifica

tion

1,024 records identified through other sources

49,472 records after duplicates removed

1,869 records screened 47,603 records irrelevant

225 full-text articles assessed

120 full-text articles excluded

137 studies included in quantitative

synthesis of effects

44 studies included in qualitative synthesis

‘Factual’ evaluation studies (eg qualitiative,

ethnographic etc)

‘Counterfactual’ impact evaluation studies


Diarrhoea impacts: evidence from 71 studies update ongoing)

NOTE: Weights are from random effects analysis

Water supply interventions

Subtotal

Water quality interventions

Subtotal

Sanitation interventions

Subtotal

Hygiene interventions

Subtotal

Multiple interventions

Subtotal

ID

Study

0.98 (0.89, 1.06)

0.58 (0.50, 0.67)

0.63 (0.43, 0.93)

0.69 (0.61, 0.77)

0.62 (0.46, 0.83)

ES (95% CI)

0.98 (0.89, 1.06)

0.58 (0.50, 0.67)

0.63 (0.43, 0.93)

0.69 (0.61, 0.77)

0.62 (0.46, 0.83)

ES (95% CI)

Ratio favours intervention 1.1 .5 .75 1 2


Sustainability 1: less impact over longer periods


Water supply (12 months or more)

Subtotal

Water quality (under 12 months)

Subtotal

Water quality (12 months or more)

Subtotal

Sanitation (12 months or more)

Subtotal

Hygiene (under 12 months)

Subtotal

Hygiene (12 months or more)

Subtotal

Multiple (under 12 months)

Subtotal

Multiple (12 months or more)

Subtotal

ID

Study

0.82 (0.71, 0.96)

0.56 (0.47, 0.66)

0.81 (0.67, 0.97)

0.64 (0.37, 1.10)

0.72 (0.60, 0.86)

0.67 (0.49, 0.91)

0.41 (0.23, 0.74)

0.77 (0.70, 0.85)

ES (95% CI)

0.82 (0.71, 0.96)

0.56 (0.47, 0.66)

0.81 (0.67, 0.97)

0.64 (0.37, 1.10)

0.72 (0.60, 0.86)

0.67 (0.49, 0.91)

0.41 (0.23, 0.74)

0.77 (0.70, 0.85)

ES (95% CI)



Sustainability 2: compliance falls over time

• Ceramic filter provision in Cambodia; 3 years later only 31% households were still using the filters (Brown et al, 2007)

• Pasteurisation in Kenya; 4 years later only 30% continued to pasteurise their water (Iijima et al, 2001)

• Programme promoting POU water disinfectant in Guatemala 1 year later; repeated use among only 5% of households from original trials (Luby et al, 2008).

• Water filters in Bolivia; compliance 67%; assessment made 4 months after trial ended (Clasen et al, 2006)

www.3ieimpact.orgAuthor name

Sustainability 3: lack of WTP (Kremer et al. 2012)

In Kenya, access to free chlorine increased uptake to over 60 percent, whereas coupons for even a 50 percent discount had a minimal effect


Sustainability of hygiene impacts


Water supply (12 months or more)

Subtotal

Water quality (under 12 months)

Subtotal

Water quality (12 months or more)

Subtotal

Sanitation (12 months or more)

Subtotal

Hygiene (under 12 months)

Subtotal

Hygiene (12 months or more)

Subtotal

Multiple (under 12 months)

Subtotal

Multiple (12 months or more)

Subtotal

ID

Study

0.82 (0.71, 0.96)

0.56 (0.47, 0.66)

0.81 (0.67, 0.97)

0.64 (0.37, 1.10)

0.72 (0.60, 0.86)

0.67 (0.49, 0.91)

0.41 (0.23, 0.74)

0.77 (0.70, 0.85)

ES (95% CI)

0.82 (0.71, 0.96)

0.56 (0.47, 0.66)

0.81 (0.67, 0.97)

0.64 (0.37, 1.10)

0.72 (0.60, 0.86)

0.67 (0.49, 0.91)

0.41 (0.23, 0.74)

0.77 (0.70, 0.85)

ES (95% CI)


Source: Waddington et al 2009 3ie Systematic Review


Scalability of hygiene promotion

• Vietnam (Chase & Do, 2012):– Handwashing BCC (not soap) scale-up

through 1) mass media & 2) inter-personal communication – No health or productivity effects

• Peru (Galiani & Gertler):– Hygiene promotion through 1) mass media & 2) community

level (health facilities, schools)– No impacts health or nutrition (compliance measured)

• Bangladesh (Huda & Luby 2012):– SHEWA-B local community hygiene promoters– Low compliance rates & no impacts on diarrhoea or ARIs


Supply doesn’t create its own demand

• Reduction in child disease rates not observed by carers or seen as substantial enough benefit to warrant costs (money/time) – what role for health education?

• Adoption of innovations (social change) is a slow process (early adopters vs. laggards) – what role for ‘triggering’ (CLTS)?

• Perceived benefits important (e.g. user satisfaction, time-use, safety) – role for appropriate technology/maintenance?


Thank you

• Systematic review evidence library:http://www.3ieimpact.org/evidence/systematic-reviews/

Www.3ieimpact.org Hugh Waddington What works in WASH? Evidence from systematic reviews Hugh...

Documents

Transcript of Www.3ieimpact.org Hugh Waddington What works in WASH? Evidence from systematic reviews Hugh...