RESEARCH ARTICLE Open Access

Cluster-randomised trial to test the effectof a behaviour change intervention ontoilet use in rural India: results andmethodological considerationsWolf-Peter Schmidt1* , Kavita Chauhan1, Priya Bhavsar2, Sandul Yasobant3, Vaibhav Patwardhan2, Robert Aunger1,Dileep Mavalankar2, Deepak Saxena2 and Val Curtis1

Abstract

Background: Effective and scalable behaviour change interventions to increase use of existing toilets in lowincome settings are under debate. We tested the effect of a novel intervention, the ‘5 Star Toilet’ campaign, ontoilet use among households owning a toilet in a rural setting in the Indian state of Gujarat.

Methods: The intervention included innovative and digitally enabled campaign components delivered over 2 days,promoting the upgrading of existing toilets to achieve use by all household members. The intervention was testedin a cluster randomised trial in 94 villages (47 intervention and 47 control). The primary outcome was theproportion of households with use of toilets by all household members, measured through self- or proxy-reportedtoilet use. We applied a separate questionnaire tool that masked open defecation questions as a physical activitystudy, and excluded households surveyed at baseline from the post-intervention survey. We calculated prevalencedifferences using linear regression with generalised estimating equations.

Results: The primary study outcome was assessed in 2483 households (1275 intervention and 1208 control).Exposure to the intervention was low. Post-intervention, toilet use was 83.8% in the control and 90.0% in theintervention arm (unadjusted difference + 6.3%, 95%CI 1.1, 11.4, adjusted difference + 5.0%, 95%CI -0.1, 10.1. Thephysical activity questionnaire was done in 4736 individuals (2483 intervention and 2253 control), and found noevidence for an effect (toilet use 80.7% vs 82.2%, difference + 1.7%, 95%CI -3.2, 6.7). In the intervention arm, toiletuse measured with the main questionnaire was higher in those exposed to the campaign compared to theunexposed (+ 7.0%, 95%CI 2.2%, 11.7%), while there was no difference when measured with the physical activityquestionnaire (+ 0.9%, 95%CI -3.7%, 5.5%). Process evaluation suggested that insufficient campaign intensity mayhave contributed to the low impact of the intervention.

(Continued on next page)

© The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you giveappropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate ifchanges were made. The images or other third party material in this article are included in the article's Creative Commonslicence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commonslicence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtainpermission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to thedata made available in this article, unless otherwise stated in a credit line to the data.

* Correspondence: Wolf-Peter.Schmidt@lshtm.ac.uk1Environmental Health Group, Department of Disease Control, LondonSchool of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT,UKFull list of author information is available at the end of the article

Schmidt et al. BMC Public Health (2020) 20:1389 https://doi.org/10.1186/s12889-020-09501-y

(Continued from previous page)

Conclusion: The study highlights the challenge in achieving high intervention intensity in settings where theproportion of the total population that are potential beneficiaries is small. Responder bias may be minimised bymasking open defecation questions as a physical activity study. Over-reporting of toilet use may be further reducedby avoiding repeated surveys in the same households.

Trial registration: The trial was registered on the RIDIE registry (RIDIE-STUDY-ID-5b8568ac80c30, 27-8-2018) andretrospectively on clinicaltrials.gov (NCT04526171, 30-8-2020).

Keywords: Sanitation, Behaviour change, Bias

BackgroundOpen defecation is thought to contribute to the trans-mission of gastro-intestinal infections especially in youngchildren. The direct health benefits of high coveragewith and use of toilets in rural areas have not beenestablished and may be small [1, 2]. Toilet use may how-ever be associated with indirect and non-health benefitssuch as reducing psycho-social stress in women [3].India had, until recently, more than 60% of the global

population that defecates in the open [4]. For more thanthree decades, the Government of India has made effortsto improve sanitation in rural India mainly by providingsubsidy for toilet construction with some information,education and communication activities [5]. However,while coverage with toilets increased, there were oftenlimited improvements in toilet use [5–8]. In 2014,through the launch of Swachh Bharat Mission-Gramin(SBM-G), the pace of toilet construction has increased[9]. Current strategies include a decentralised approachto improving sanitation coverage and use, by augment-ing the capacity of state governments to undertake be-haviour change activities and incentivizing governmentperformance [10]. SBM-G focuses on mass media cam-paigns and village level events to address people’s toiletuse behaviour [9]. Recent surveys show improvement inprovision of toilets and toilet use [9]. However, theSwachhta Status Report as recent as 2015 [11] foundthat around 52% of the Indian population still defecatedin the open. Similarly, a 2014 survey in rural villages inthe north Indian states of Bihar, Madhya Pradesh, Rajas-than, and Uttar Pradesh found that 70% of the popula-tion defecated in the open. A resurvey in the same statesin 2018 showed that this figure had reduced to 44% [12].However, the same survey also found that 23% of peopleliving in households with a toilet still defecated in theopen. This figure contradicts findings from the NationalAnnual Rural Sanitation Survey, which found that 97%of households in India with a toilet actually use it [13],perhaps highlighting methodological challenges in meas-uring toilet use in large surveys.Hence, improving toilet use continues to be a public

health challenge in India. Current efforts to change toiletuse behaviour have often relied on traditional methods

of health education [5, 7, 14], often making use of socialpressure to effect behaviour change [12]. Creative, in-novative and modern approaches to behaviour change,for example using models such as the Risks, Attitudes,Norms, Abilities, and Self-regulation framework (RANAS) [15] or Behaviour Centred Design (BCD) [16] haveshown some promising results for handwashing [17, 18],food hygiene [19] and water treatment [15], but haverarely been applied to sanitation behaviour and have notbeen tested within a larger programme. Therefore, thepresent study aimed to test the effect of the 5-Star Toiletcampaign, a scalable intervention which was developedusing BCD, on the use of toilets in rural villages in theIndian state of Gujarat in the context of the ongoingSBM-Gramin programme. Toilet construction was notpart of this intervention. The study was part of a com-parative programme involving three other studies pursu-ing similar study designs, tools and behaviour changeobjectives. The other three trials were conducted byother groups in the Indian states of Odisha, Bihar andKarnataka, testing different behaviour change interven-tions to increase toilet use in households with an exist-ing toilet [20].

MethodsStudy design and settingThe study was a cluster-randomised trial with 47 inter-vention and 47 control clusters, conducted in the districtof Bhavnagar, Gujarat, India. Bhavnagar was chosen be-cause it was the last district in Gujarat to be declaredopen defecation free, and therefore deemed to have ahigh prevalence of non-use of toilets. The study designis depicted in Fig. 1. Overall, 94 villages were rando-mised to intervention or control groups. Outcomes (useof toilets by all household members and a range of sec-ondary outcomes) were assessed in a cross sectionalquestionnaire survey post-intervention.

Enrolment of villagesTo enrol villages we compiled a list of all Gram Pan-chayats (an administrative unit in India consisting of oneor more villages) in three blocks (Mahuva, Talaja andPalitana). There were a total of 335 villages from 325

Schmidt et al. BMC Public Health (2020) 20:1389 Page 2 of 16

Gram Panchayats. For our study, Gram Panchayats wereeligible for enrolment if, based on government records,they had at least 70% toilet coverage (n = 137). Whenthere were multiple villages within a Gram Panchayat,we selected the village with the higher reported cover-age. From the list of 137 villages, 110 villages were se-lected using probability proportional to size samplingusing data from the National Census of 2011. We con-ducted a household census in 106 of the 110 clusters (4were excluded due to logistics). We excluded 16 villageswith the lowest coverage, arriving at 94 villages selectedfor the study. Out of the 94 clusters, three clusters hadpopulations of more than 300 households. We thereforeused chunking to segment these larger villages into mul-tiple parts and then selected two segments of approxi-mately 150 households in each village which were bothenrolled as the same cluster. Toilet coverage in theresulting villages was lower than expected prior to thecensus (mean 60.0%, range 9%–100%). Randomisationwas done in 13 strata. We created 5 different strata oftoilet coverage (0–24%, 25%–44%, 45%–59%, 60%–74%,

75%+) and 3 different strata of household tap watercoverage at village level (0% to 49%, 50% to 74%, 75%+).We predicted that these two variables might correlatewith toilet use and the success of the intervention. Thecombination of these two strata resulted in 13 differentstrata (stratum size ranging from 2 to 20 villages). Aminimum 3 km distance was maintained between inter-vention and control clusters.

Enrolment of householdsWe carried out a household census, and from this listrandomly selected 40 eligible households per village forthe study. Households were eligible if they had receivedany assistance, either monetary or any other, under anygovernment programme, to construct a toilet and had afunctional toilet, defined as having; 1) a pan that is notbroken, and 2) a functional connection to a pit (single ortwin pits).From the list of 40 households per village we randomly

allocated 10 households to the baseline survey and 30 to

Fig. 1 Study flow diagram

Schmidt et al. BMC Public Health (2020) 20:1389 Page 3 of 16

the follow up survey (post-intervention). The baselinestudy was done to estimate the overall baseline preva-lence of the outcomes and the intra-class correlation co-efficient to confirm the sample size calculation. The 10baseline households per village were discarded from fur-ther study. In the other 30 households, toilet use andother detailed sanitation-related data were collected onlypost-intervention, not at baseline.

InterventionThe ‘5 Star Toilet’ campaign used the BCD frameworkand theory of change (ToC) to design the intervention[16] (Fig. 2). BCD’s theory of change involves five steps:Assess; (review what is known about the determinantsand context of the target behaviour); Build (gather datafrom the field to understand the determinants of behav-iour in-situ); Create (work with a creative team to iden-tify a novel, surprising insight about the targetbehaviour, incorporated into intervention components);Deliver (implement the components of the intervention)and Evaluate (determine the effectiveness of the imple-mentation vis a vis behaviour change). A detailed de-scription of the intervention development has beenpublished elsewhere [21]. The Assess step included a lit-erature review and a Framing Workshop. Mixedmethods formative research was conducted in villagesclose to the study villages that were not part of the ac-tual trial to identify the determinants of toilet use/non-use in the study population and to arrive at a designbrief (Build). Formative research methods includedstructured conversations (n = 40) with the help of a dis-cussion guide, and range of other research tools [21].We also carried out a survey in 200 households tounderstand toilet coverage and functionality. These

insights were then organised using the BCD checklistof potential factors in the environment, physical set-ting and informants’ minds and included water avail-ability, pit filling, knowledge about disease, manners,shame, dignity, safety, comfort, nurture, routine andhabit. The creative process (Create step) involvedbrainstorming and reflection to generate concepts andideas to address the determinants of toilet use. The ‘5Star Toilet’ concept was nested within the campaigntheme of ‘The World is Getting Smarter’ and a life-style is not ‘completely smart’ until people have a toi-let that matches the quality of their other ‘smart’belongings such as smart phones, laptops, gadgets etc.A general theme of the intervention was the attemptto position the toilet as a modern appliance, muchlike the mobile phones and bikes. This lead to theconcept of presenting the intervention in a “digitalcontext” by employing innovative methods such as“Virtual Reality” toilets, “Pi-Fi” (A local wifi whereone can download campaign films without mobiledata), and films shot in a village and showed backusing a casting device, or uploaded on YouTube.Components of the intervention were piloted separ-ately and as a package.The final iteration of the intervention was rolled out

by the Coastal Salinity Prevention Cell (a local NGO)from mid-September to December 2018. The interven-tion was delivered during 2 days in each village by twoteams comprising three trained facilitators and perform-ing artists (Table 1). Day 1 and Day 2 were separated byabout 4 weeks. Facilitators had at least an undergraduatedegree in social sciences or an equivalent qualificationand were trained over a 3 week period prior to the pilot-ing of the intervention.

Fig. 2 BCD concept, from [16]

Schmidt et al. BMC Public Health (2020) 20:1389 Page 4 of 16

Table 1 Campaign components

Activity Description Tools

Pre intervention delivery

Meetings to seeksupport

Meetings were held with village leaders to discuss the campaign and get supportto plan and organize day 1 event.

Facilitation script.

Recruitment ofvolunteers

1 volunteer identified from each village with help of local NGO partner/ Sarpanchwho would promote 5 Star Toilet concept, help the team of facilitators to deliverthe day 1 and day 2 intervention and follow up with community members for 5Star Toilet makeovers.

Training of volunteers.

3 Days before day 1event

Call/s were made to each village volunteer to ensure the WhatsApp broadcastgroups are formed to share information on the campaign with the communitymembers, mobile teasers have been passed around, leaders met with andlocations identified for event/s.

WhatsApp teasers, phone calls

DAY 1

Announcements A customized campaign vehicle to go around the village to make announcementsand also carry all the material for the events.

Vehicle design, announcement script,song recording, media player

Interaction withvolunteer

Facilitators, with the help pf volunteer, identify location for the evening event andcreate a route plan for the household visits and street events.

Interaction withchildren

Expose children to a virtual reality (VR) experience of a 5 star toilet design and theidea of 5 star toilets and teach them slogans around 5 star toilets.

VR App on 5 star toilet design, VRgoogles, phone

Household Visits Two teams of facilitators + Artists + Van + Children go from street to street,making household visits. Expose the idea of 5 star toilets, enquire if they wouldlike to know the rating for their toilets, rate their toilet and express appreciationfor what they already have. If they have 5 star: Award them with a 5 star stickerand paste it on their toilet and invite them to the evening event to receivecertification. Take photographs. If they don’t have 5 star: Explain what they needto do to get 5 star.

5 star toilet leaflet5 star toilet poster

Van in thecommunity

Park the van in the street and make announcements, play songs, display 3Dphotographs of different toilet innovations from around the world, display smalltoilet model, and VR experience of a 5 star toilet.

Music player, photographs, mobile,VR goggles, VR App

Preparations for theevening event

Set up the venue for the evening event: AV + seating arrangement for communitymembers, download photographs of the day’s activities from phone/camera andwrite certificates for 5 star toilet awardees.

Certificates, AV system, rug forseating arrangement

Enrolment Corner In parallel create an enrolment corner for households willing to improve theirexisting toilets into a 5 star toilet with a standee on 5 star toilets, a table toshowcase 5 star toilet model and a toilet chair on display for differently abledpeople.

Leaflets and documentation sheet,toilet chair, smart network Wi-Fi

Evening Event 1. Play the campaign song and interact with the children and makeannouncements2. Play Films – Saving Time and Saving Effort3. Skit performance4. World of Toilets (slide show)5. Toilet makeover films and toilet chair films6. Celebrate those with 5 star toilets/ 5 star+ by awarding certificates7. Introduce those who have enrolled – call them to the front and celebrate them8. Farewell – “All the best! We will come back in 2–3 weeks to celebrate again”.

AV equipment, films, artists,certificates

Follow up Volunteers promote 5 star toilet makeovers between day 1 and day 2 events. Takephotographs of families who have modified their existing toilets.

Home visits, follow up on phone

Share films and song Share films and campaign song in the village WhatsApp groups. WhatsApp, YouTube linkhttps://www.youtube.com/channel/UCqmL6DxtcDpAKeIU4Io33ig

DAY 2

Organize All the pre-post toilet makeover photographs from the village are compiled into apresentation – clearly marking the names of people.

Laptop/Tablet

Announcements Make announcements about the evening event. Van, audio system, announcementscript.

Testimonial Videos Record videos of families that undertook 5 star toilet makeover. Phone camera

Evening communityevent

Make preparations, identify site.

Schmidt et al. BMC Public Health (2020) 20:1389 Page 5 of 16

Outcomes and outcome assessmentThe primary outcome of toilet use was defined as theproportion of households where all members above theage of 5 years were reported to use the toilet the lasttime they defecated. This outcome was measured usinga questionnaire which also covered secondary study out-comes, i.e. exposure to the intervention, perceptionsaround toilet ownership in the community, agreementwith sanitation related statements, and directly observedtoilet characteristics. Apart from exposure to the inter-vention, the questions asked were the same at baselineand endline. The survey was addressed to one adult re-spondent per household, preferably the male or femalehousehold head. A gap of 6 weeks was maintained be-tween campaign roll out and endline data collection.The questionnaire was simultaneously administered inintervention and control clusters.Because of concerns regarding over-reporting of toilet

use, we developed an alternative tool to reduce sociallydesirable responses and responder bias. Specifically forthe purposes of this study, we designed a questionnaireabout physical activities relating to diet and incidentalactivities concerning the daily routine of respondents,with time spent for each physical activity measured inminutes (categorised as < 10min, 10-30 min, > 30 min,or NOT doing this activity). Among the questions, therewas one question about time spent to walk to the fieldfor defaecation, with “using toilet at home” offered as apossible response. This questionnaire was delivered by aseparate team from a different data collection companythat was kept blind to the real purpose of the survey.The tool was administered about 1 week before the end-line survey in each village to make it less likely that re-spondents would link the physical activity study to theintervention or the subsequent main endline survey. Thequestionnaire was completed by one or two people perhousehold (preferably one female and one male) aged 18and above who were present at the time of the visit and

who were only asked about their own physical activity. Ifmore than two eligible persons were found in a household,two individuals were selected at random. If only one eli-gible person was present in a household, the team visitedan additional household until the target number of 60 in-terviews per village was met. The complete physical activ-ity questionnaire is added as Supplementary material.Both teams (main questionnaire team and physical ac-

tivity team) were given the same list of about 30 house-holds per village selected for the endline survey atbaseline (see above). To account for non-availability ofhouseholds due to migration, refusal to take part in thesurvey and other factors, we randomly selected an add-itional 15 households per cluster, or fewer, dependingon availability. Both data collection teams selected fromthis additional list households to replace households thatwere not available, or (for the physical activity question-naire) if only one eligible person was found. As the twoteams for the physical activity tool and the endline toiletuse questionnaire worked independently, only 66% offinal household samples overlapped (Fig. 1).

Process evaluationThe process evaluation aimed to understand the reasonsfor the results of the 5 Star Toilet Campaign. Data col-lection methods and sources used to assess the processincluded; 1) Document review (reports, newspaper clip-pings, and government BCC strategy paper), 2) Field ob-servations (n = 6) and review of activity logs to assessintervention fidelity, and participation of community, 3)Semi-structured interviews (n = 14) with SBM officials,the design team, intervention delivery team, and partici-pants from intervention and non-recipients from controlclusters, 4) Focus Group Discussions (n = 5) withprogramme staff (n = 1) and participants (2 each withwomen and men, 8–10 participants per group) to ex-plore views on the campaign and the perspectives on toi-let use and non-use. In depth process evaluation was

Table 1 Campaign components (Continued)

Activity Description Tools

Guessing Contest PitFilling

Participants asked to guess how fast a pit fills up. This is done through a life sizepit standee. The facilitator explains the time it takes for a pit to fill and explainsthe process of composting.

Live sized pit standee

Guessing ContestCompost

Jars with normal soil and compost are kept on a table. Participants are invited toguess which jar contains compost.

Glass jars with soil and compost.

Films of pit fillingand testimonialvideos

Films of pit filling are showcased and videos of people who undertook toiletmakeovers are played.

Testimonial films, short films

Toilet Board Photographs of people who did toilet makeover are displayed on a board and theboard in placed in village centre or Panchayat Gahr.

Board, pictures, printer

Toilet Makeover Presentation of certificates to those who improved toilets. Invite participants tocome and share their experience with those in attendance.

Pre/post presentation.

Farewell People are thanked for their participation.

Schmidt et al. BMC Public Health (2020) 20:1389 Page 6 of 16

done in 2 intervention and 2 control clusters chosen atrandom (Fig. 1). Results have partially been published else-where [21] and will be reported in full in a future paper.

Sample sizeWe assumed that in 65% of households with a govern-ment supported toilet, all household members aged 5years or older would be using this latrine. This figurewas based on formative research in the study area thatfound that about 44% of households had members whogo for open defecation. We assumed that use would in-crease to 75% (by 10% points) compared to the controlarm, which was deemed an effect size of public healthinterest. Using a standard formula to calculate a samplesize for the comparison of two proportions resulted inrequiring 349 households per arm to detect this 10% dif-ference with 80% power and an alpha of 0.05, ignoringvillage level randomisation. We assumed an intra-classcorrelation coefficient (ICC) of 0.1 based on a sanitationtrial in Orissa [22]. We chose to enrol 30 households pervillage cluster, as enrolling more only marginally re-duced the number of required clusters. This resulted in45 villages per arm, and 30 households per village at adesign effect of 3.9. This figure was increased to 47 vil-lages per arm to account for loss to follow-up of house-holds and whole clusters.

Statistical analysisPrevalence differences across trial arms were calculatedusing linear regression (function: Gaussian, link: iden-tity). Clustering at village level was adjusted for by usinggeneralised estimating equations and robust standard er-rors. The primary endpoint analyses were done usingunadjusted models (intention-to-treat analysis). Due tosome imbalance in socio-economic characteristics, allanalyses were finally adjusted for an asset index (con-tinuous variable) and maximum male education level(dichotomised into primary schooling or less vs second-ary or higher). The asset index was constructed usingprincipal component analysis of 9 socio-economic vari-ables. The first predicted component explained 27.7% ofvariation and was chosen as asset index (KMO: 0.73).Sample size calculations and all analyses were done inSTATA 14 (StataCorp).The trial was prospectively registered on the RIDIE

registry (RIDIE-STUDY-ID-5b8568ac80c30, 27-8-2018)and retrospectively registered on clinicaltrials.gov(NCT04526171, 30-8-2020).

ResultsOf the 1384 intervention households selected for themain questionnaire, 351 (25.3%) could not be found ordid not in fact, have a latrine (were ineligible), and 26(1.9%) did not consent. Two hundred seventy-one

households were added from the list in random order,resulting in 1278 households (6679 individuals) enrolledfor the endline survey. Of the 1333 control householdsselected for the endline survey prior to the intervention,331 (24.8%) could not be found or did not in fact have alatrine (were ineligible), and 33 (2.5%) did not consent.Two hundred forty-five households were added from thelist in random order, resulting in 1214 households (6174individuals) enrolled for the endline survey. The physicalactivity questionnaire was done in 4741 individuals(2483 intervention and 2253 control participants), ofwhich 3114 (66%) were from households also includedin the main questionnaire.Table 2 shows the socio-economic characteristics of

control and intervention study populations by interven-tion arm from the endline survey. Good balance wasachieved with respect to household size, caste, religion,female education and house structure. Some imbalanceswere observed, with male education and graduate leveleducation being more common in the intervention arm.There was also some imbalance in the distribution ofthe asset index, with intervention households more com-monly found in higher asset quartiles.Table 3 shows the effect of the intervention on the pri-

mary study outcomes (three responses in the interven-tion arm and 6 responses in the control arm weremissing). At baseline reported toilet use by all householdmembers was 87.0% of households in the control and83.4% of households in the intervention arm. At followup reported use of the toilet by all household memberswas 83.8% of households in the control group and 90.0%in the intervention group, i.e. 6.3 percentage points(95%CI 1.1, 11.4) higher in the intervention compared tothe control arm. This effect size was slightly attenuatedafter adjusting for asset index (as continuous variable)and highest male education in a household (dichoto-mised into illiterate to primary vs secondary or higher)to + 5.0% (95% CI -0.1, 10.1). A similar effect size wasfound for the analysis of the prevalence of toilet use inindividual household members (not collapsed at house-hold level) (85.1% vs 91.2%, adjusted difference + 4.6%difference, 95% CI 0.5, 9.7). Overall, toilet use by allhousehold members in the control arm at baseline (87%)was similar to toilet use by all household members ob-served in the control arm at follow up (84%), suggestingan absence of a temporal trend from baseline to followup, or an absence of an effect of the trial procedures onthe reporting of behaviour.The physical activity questionnaire produced a 4.4%

point lower overall estimate of individual toilet use thanthe endline tool (84.5% vs 88.9%). No major effects ofthe intervention on toilet use were observed, with orwithout adjusting for asset index and male education(difference + 1.7%, 95%CI -3.2, 6.7). The estimates were

Schmidt et al. BMC Public Health (2020) 20:1389 Page 7 of 16

not greatly affected by restricting the analysis to house-holds also part of the main survey. For the adjusted ana-lysis, the test for interaction to explore the difference ineffect estimates between the main questionnaire tool (ef-fect + 5.0%) and the physical activity tool (effect + 1.7%)showed a p-value of 0.09, suggesting some evidence for adifference in effect estimates.

As shown in Table 4 the intervention had only a lim-ited effect on observed toilet characteristics. Minor ef-fects were found including a 6.4% point increase foravailability of a water container, slippers and cleaningmaterials, as well as in 4 of the 5 attributes promoted bythe 5 Star Toilet campaign (painted walls, cleanliness,light bulb, water available), but the confidence intervalswere wide, while the effect sizes were reduced afteradjusting for asset index and male education. Slightlymore toilets in the intervention arm than in the controlarm were found to be in apparent use.Compared to the control group, intervention house-

holds more often reported having heard of, or attended,community events on sanitation. They also reportedhigher exposure to most campaign-specific elements,such as pit filling demonstrations, using a chair forassisting the disabled in the toilet, or seeing a smallmodel of a 5 star toilet (Table 5). A higher proportion ofrespondents in the intervention arm reported makingchanges to their toilets. However, overall campaign ex-posure was low. Only about 14% of intervention house-holds had heard the term “5 Star Toilet” (3% in control).Four percent could show a certificate awarded by thecampaign team (almost nobody in the control armcould). Only 18% of households in the intervention armhad seen the skit (5% in control arm), and 13% had seenthe 5 Star Toilet model (2% in control arm). Exposure tomost other campaign items showed an intervention-control difference of less than 10% points.There was little difference between intervention and

control regarding agreement with statements reflectingimportant campaign messages such as “Toilets are notjust for women; men should use them too”, “A smartperson is one who uses a toilet”, or (phrased negatively)“Toilet pits fill quickly if too many people in the house-hold use them” (Table 6). Consistent with the findingthat slightly more intervention arm respondents re-ported improving their toilets, they more often reporteda perception that those around them were improvingtheir toilets.In the intervention arm, reported use of toilet by all

household members (main questionnaire tool) was96.1% in those reporting to have heard of the 5 starcampaign, and 89.1% in those that had not heard of it(prevalence difference + 7.0%, 95%CI 2.2% / 11.7%). Bycontrast, again focussing on the intervention arm, therewas no difference in reported toilet use measured by thephysical activity questionnaire between exposed and un-exposed participants (86.8% vs 85.9%, prevalence differ-ence 0.9%, 95%CI -3.7% / 5.5%). To explore thestatistical support for this difference in the effect of cam-paign exposure on reported toilet use between the twotools, we conducted a test for interaction between typeof questionnaire (main tool vs physical activity tool) and

Table 2 Socio-demographic and socio-economic characteristicsof households at endline enrolled for follow up study

Item Control Intervention Prevalencedifference,%

N % N %

Total 1214 1278

Household size

1–3 252 20.8 267 20.9 0.1

4–5 398 32.8 436 34.1 1.2

6–7 365 30.1 324 25.4 −4.5

8+ 199 16.4 251 19.6 3.1

Caste

SC/ST 56 4.6 39 3.1 −2.3

OBC 730 60.1 792 62.0 1.7

General 315 26.0 357 27.9 2.9

Prefer not to disclose 113 9.3 90 7.0 −1.8

Religion

Hindu 1200 98.9 1264 98.9 0.1

Muslim 14 1.2 14 1.1 −0.1

Highest female education level (n = 2467)

No formal schooling 277 23.0 259 20.4 −2.5

Primary 161 13.4 181 14.3 1.1

Secondary 682 56.6 716 56.4 −0.3

Diploma 9 0.8 10 0.8 0.1

graduate 77 6.4 104 8.2 1.7

Highest male education level (n = 2449)

No formal schooling 83 6.9 82 6.5 −0.3

Primary 156 13.0 160 12.7 0.1

Secondary 774 64.7 727 57.7 −7.0

Diploma 29 2.4 23 1.8 −0.5

graduate 156 13.0 269 21.3 8.6

Asset index quartile

Lowest 348 28.7 294 23.0 −5.6

Low intermediate 319 26.3 288 22.5 −4.1

High intermediate 295 24.3 340 26.6 2.6

Highest 252 20.8 356 27.9 7.4

House structure

Kutcha 158 13.0 165 12.9 −0.1

Semi-pukka 619 51.0 631 49.4 −1.5

Pukka 437 36.0 482 37.7 1.7

Schmidt et al. BMC Public Health (2020) 20:1389 Page 8 of 16

having heard of the campaign in the intervention arm whichshowed a p value of 0.08 (Fig. 3). This provides some supportfor the idea that in the intervention arm over-reporting ofthe outcome (reported toilet use) occurred especially in thoseexposed to the campaign, while the physical activity toolminimised over-reporting related to campaign exposure.

Process evaluationThe process evaluation revealed several challenges in thedelivery of the intervention. Most prominently, the scat-tered population made it difficult to reach all households

within the time and budget available. Often, due to hotweather, people did not want to step out of their homesin the afternoon to participate in the street events. On-going agricultural work or other livelihood activitiesmeant that many people either lived on the farmland ordid not return home until around late in the evening.Some communities did not encourage women to partici-pate in evening events. We found that the role of thevolunteers remained weak as the implementation teamcould not recruit volunteers in 10 clusters, and in someclusters their engagement could not be sustained beyond

Table 3 Effect of the intervention on study outcomes

Item Control Intervention PD,%

95%CI

APD% 95% CI ICC

N % N %

Baseline

Use of toilet by all household members (irrespective of apparent toilet use) 328 87.0 303 83.4 −4.9 –

Endline

Primary outcome

Use of toilet by all household members 1208 83.8 1275 90.0 6.3 1.1 /11.4

5.0 −0.1 /10.1

0.14

Secondary outcomes

Individually reported toilet use (reported use not collapsed at household level) 6174 85.1 6679 91.2 6.1 1.1 /11.2

4.6 −0.5 /9.7

0.17

Individually reported toilet use (physical activity tool) 2253 80.7 2483 82.2 1.5 −3.4 /6.4

– – 0.12

Individually reported toilet use (physical activity tool) restricted to householdsalso taking part in endline survey

1636 82.8 1736 85.9 3.3 −1.7 /8.3

1.7 −3.2 /6.7

0.11

PD prevalence difference, calculated using linear regression (function: Gaussian, link: identity). Clustering at village level was adjusted for by using generalisedestimating equations and robust standard errors. APD adjusted prevalence difference. PD was adjusted for asset index (continuous variable) and maximum maleeducation level (dichotomised into primary or less vs secondary or higher)

Table 4 Effect of intervention on observed toilet characteristics

Item Control Intervention PD,%

95% CI APD% 95% CI

N % N %

Latrine use for other purpose 1214 9.6 1278 6.3 −3.3 −6.4/− 0.2 −2.6 −5.6/ -0.4

Clogging of squatting pan 1214 15.0 1278 10.6 −4.2 −8.4/0.0 −3.2 −7.4/ 1.0

Availability of water container 1214 84.9 1278 89.1 4.2 −0.7/9.0 3.3 `-1.6/ 8.1

Availability of slippers 1214 19.8 1278 24.9 4.8 0.1/ 9.4 3.0 −1.8/ 7.7

Availability of cleaning materials 1214 77.6 1278 84.3 6.4 0.8/ 12.0 5.0 −0.7/ 10.6

Toilet is in apparent use 1214 86.1 1278 90.4 4.3 −0.6/ 9.2 3.1 −1.8/ 8.0

Made any changes in last 6 months 1214 6.3 1278 6.0 −0.2 −2.4/ 1.9 − 0.3 −2.5/ 1.9

Plan to make any changes 1214 27.6 1278 22.9 −4.7 −9.3/ 0 −3.6 −8.3/ 1.2

Five star items

Painted walls 1214 44.9 1278 52.9 8.1 1.9/14.2 5.3 −0.8/11.5

Clean 1214 68.5 1278 76.0 7.4 1.3/13.4 5.7 −0.4/11.7

Light bulb 1214 53.4 1278 62.4 9.3 1.8/16.8 6.9 −0.3/14.1

Ventilation 1214 18.0 1278 18.8 0.8 −3.0/4.7 0.4 −3.6/4.3

Water 1214 39.0 1278 47.6 8.9 2.2/15.6 5.3 −1.0/11.7

PD prevalence difference, calculated using linear regression (function: Gaussian, link: identity). Clustering at village level was adjusted for by using generalisedestimating equations and robust standard errors. APD adjusted prevalence difference. PD was adjusted for asset index (continuous variable) and maximum maleeducation level (dichotomised into primary or less vs secondary or higher)

Schmidt et al. BMC Public Health (2020) 20:1389 Page 9 of 16

Table 5 Exposure to Intervention

Item Control Intervention PD,%

95% CI APD% 95% CI

N % N %

Recently heard about toilets in any of these contexts (in last 6 months)

Conversation with others 1214 6.7 1278 9.9 3.1 0.6/5.6 2.8 0.3/5.4

Visits to neighbours 1214 3.1 1278 4.5 1.3 −0.2/2.8 1.2 −0.3/2.8

WhatsApp message 1214 2.1 1278 3.1 1.0 −0.7/2.7 0.4 −1.1/2.0

Village meeting 1214 14.3 1278 23.5 9.1 5.1/13.1 8.4 4.3/12.4

Event in community 1214 13.1 1278 30.0 16.7 11.4/22 16.3 11/21.6

Posters /stickers 1214 6.9 1278 13.2 6.5 3.6/9.5 6.2 3.2/9.2

Radio 1214 0.4 1278 0.6 0.1 −0.4/0.7 0.0 −0.1/0.1

TV 1214 21.9 1278 22.9 1.3 −3.3/5.8 0.0 −0.5/4.4

What did you hear

One should construct a toilet if a household doesn’t have one 1214 14.4 1278 19.9 5.6 2.0/9.2 5.2 1.6/8.9

One should improve one’s toilet if it is poor quality 1214 6.7 1278 12.1 5.4 3.1/7.7 5.2 2.8/7.5

One should use toilet for defecation instead of going out in the open 1214 18.5 1278 25.0 6.3 2.5/10.1 5.8 2.1/9.5

After hearing this did you make changes to your toilet or done anything as a consequence?

talked with someone 1214 15.0 1278 18.3 3.4 −0.1/7.5 3.1 −1.1/7.3

made changes to my toilet 1214 8.0 1278 12.8 4.6 1.3/7.8 4.0 0.8/7.2

saved money for a toilet 1214 3.5 1278 3.6 −0.1 −1.8/1.8 0.2 −1.6/2.0

Heard of any community event that talks about toilet in the past 6 months 1214 18.5 1278 39.1 20.7 15.4/26.0 19.7 14.3/25.1

Attended such an event 1214 8.3 1278 22.3 13.9 10.6/17.1 13.3 9.9/16.7

Promote toilet improvement 1214 6.5 1278 18.3 11.7 8.8/14.6 11.2 8.1/14.2

Commit to improve toilet 1214 22.8 1278 12.3 −12.3 −21.2/−3.5 −12.5 −21.4/−3.5

Heard the phrase ‘5 star toilet’ 1214 2.6 1278 13.9 11.3 8.9/13.8 10.9 8.5/13.4

Where did you hear it

TV 1214 0.7 1278 1.6 0.1 0.0/1.9 0.1 0.0/1.9

Village meeting 1214 1.2 1278 5.2 3.9 2.6/5.2 0.4 2.4/5.1

Community event 1214 1.8 1278 10.9 9.1 6.7/11.5 8.8 6.4/11.2

WhatsApp message 1214 0.3 1278 0.5 0.1 −0.4/0.6 0.1 −0.4/0.1

Posters/stickers 1214 0.7 1278 4.1 3.4 1.9/5.0 3.2 1.8/4.7

Virtual Reality film 1214 0.4 1278 0.9 0.5 −0.2/1.3 0.5 −0.3/1.3

Friend/relative 1214 0.3 1278 0.7 0.4 −0.3/1.1 0.3 −0.4/1.1

Certificate for a 5-star toilet 1214 0.4 1278 4.5 4.0 0.3/5.1 3.8 2.7/4.8

Picture of your family on the village ‘Toilet Board’ poster 1214 0.2 1278 4.8 4.5 3.3/5.7 4.3 3.2/5.5

Skit about toilet convenience 1214 4.9 1278 18.2 13.1 10.0/16.3 12.6 9.4/15.8

Seen small-sized 5-star toilet model 1214 1.9 1278 12.5 10.7 8.2/13.1 10.3 7.8/12.8

Certificate about your toilet, or know anyone who has 1214 1.5 1278 7.4 5.9 4.2/7.6 5.7 3.9/7.5

Seen a certificate give-away 1214 2.0 1278 11.2 9.2 0.7/11.4 9.0 6.7/11.3

Someone talking about or showing a movie about pit filling 1214 2.4 1278 10.1 7.6 5.5/9.7 7.4 5.2/9.7

Movie about using a chair in the toilet for disabled or elderly people 1214 2.9 1278 11.0 8.0 6.0/10.1 7.8 5.6/10.1

Use any of the following

Facebook 1214 18.2 1278 23.6 5.2 1.3/9.1 2.1 −1.6/5.7

WhatsApp 1214 24.0 1278 31.2 7.1 3.0/11.2 3.0 −0.6/6.7

Instagram 1214 6.0 1278 8.7 2.6 0.0/5.2 0.8 −1.5/3.1

YouTube 1214 19.2 1278 22.1 2.7 −1.4/6.8 −0.8 −4.5/2.9

Schmidt et al. BMC Public Health (2020) 20:1389 Page 10 of 16

the Day 1 event. During field observations (n = 6) and exitinterviews (n = 6) with participants after the Day 1 andDay 2 events, we found that participants had largelyunderstood the campaign messages. Not all participants inthe evening events were exposed to the campaign mate-rials such as the toilet model, world of toilets and virtualreality film showcased during the street events. In theevening event where certificates were awarded to thosewith 5 star toilets, participants in large numbers expressedthe desire to get their toilets five-star certified. The skitwas the most recalled event of Day 1. Participants’ recol-lections of Day 2 event activities included the process of

how faeces converts into compost, the mad scientist videoon reducing pit filling anxiety, and the board with picturesof families with a 5 star toilet.The digital elements of the intervention were on the

whole well received and generated a lot of interest, butwere hard to deliver at scale. For example, the VirtualReality to showcase the 5 star toilet attracted crowds,but was limited by how many can view it at any point.

DiscussionThe intervention was associated with a 5% point increasein households that reported toilet use by all members

Table 5 Exposure to Intervention (Continued)

Item Control Intervention PD,%

95% CI APD% 95% CI

N % N %

Ever got or sent a message on WhatsApp about toilets 1214 2.3 1278 2.8 0.6 −0.7/1.9 −0.1 −1.3/1.2

Heard about Swachh Sunder Shauchalay campaign 1214 40.6 1278 45.8 5.3 0.6/10.1 3.7 −0.8/8.2

Swachh Sunder Shauchalay campaign is about

Paint your toilet walls 1214 7.9 1278 8.8 0.9 −1.5/3.3 0.7 −1.8/3.1

Decorate your toilets 1214 28.1 1278 32.6 4.5 0.2/8.7 3.4 −0.8/7.6

PD prevalence difference, calculated using linear regression (function: Gaussian, link: identity). Clustering at village level was adjusted for by using generalisedestimating equations and robust standard errors. APD adjusted prevalence difference. Pd was adjusted for asset index (continuous variable) and maximum maleeducation level (dichotomised into primary or less vs secondary or higher)

Table 6 Agreement with sanitation related statements among respondents

Item Control Intervention PD,%

95% CI APD% 95% CI

N % N %

Most people around here use a toilet regularly. 1214 83.6 1278 89.5 5.2 1.0/9.4 4.6 0.5/8.7

Everyone in my household uses a toilet. 1214 87.0 1278 90.9 3.8 −1.0/ 8.6 2.8 − 1.9/7.5

Many people around here are improving their toilets. 1214 71.6 1278 77.2 5.5 1.1/ 9.9 5.6 1.2/10.1

Using a toilet saves time and effort compared to open defecation. 1214 97.9 1278 98.2 0.3 −1.0/1.6 0.3 −1.0/1.6

Using a toilet builds your reputation in the community. 1214 97.6 1278 97.9 0.2 −1.2/1.7 0.0 −1.5/1.5

A smart person is one who uses a toilet. 1214 53.0 1278 51.6 −1.2 −6.5/4.2 − 1.1 −6.4/4.3

It is possible to feel proud of one’s toilet. 1214 94.9 1278 96.6 1.7 −0.2/3.5 1.4 −0.4/3.2

Most people around here think it’s good to use a toilet. 1214 96.1 1278 96.9 0.7 −1.3/2.8 0.5 − 1.6/2.6

Using a latrine gives me a ‘packed’ (claustrophobic) feeling. 1214 6.7 1278 5.5 −1.3 −3.3/0.7 −0.8 −2.8/1.1

Toilets are not just for women; men should use them too. 1214 81.0 1278 79.4 −1.2 −6.3/4.0 − 1.0 −6.2/4.2

It is appropriate to have a toilet as good as your house. 1214 98.4 1278 98.6 0.1 −0.9/1.2 0.0 −1.0/1.1

It is ok for poor people to practice open defecation. 1214 21.6 1278 17.7 −4.0 −7.1/−1.0 −3.3 −6.2/−0.4

Toilet pits fill quickly if too many people in the household use them. 1214 66.1 1278 65.7 0.1 −4.7/5.0 0.5 −4.2/5.3

Most of the people I care about think I should use a toilet. 1214 96.0 1278 95.9 −0.1 −1.9/1.7 −0.2 −1.9/1.4

People around here think a household should have a good toilet. 1214 97.9 1278 98.0 0.1 −1.1/1.3 0.1 −1.1/1.3

Even if no one else around here had a good toilet, I would still make sure I had one. 1214 91.4 1278 94.1 2.7 −0.4/5.8 2.1 −0.9/5.2

During farming season, most people around here defecate in the field/open 1214 68.7 1278 62.5 −5.9 −11.0/0.8 −4.9 −9.8/0.1

Defecating in the field is more convenient than using a toilet 1214 18.4 1278 17.8 −0.5 −4.1/3.2 0.4 −3.3/4.1

Having a good toilet at home is a mark of better status in the village 1214 98.1 1278 98.1 −0.1 −1.3/1.2 −0.1 −1.3/1.1

PD prevalence difference, calculated using linear regression (function: Gaussian, link: identity). Clustering at village level was adjusted for by using generalisedestimating equations and robust standard errors. APD adjusted prevalence difference. PD was adjusted for asset index (continuous variable) and maximum maleeducation level (dichotomised into primary or less vs secondary or higher)

Schmidt et al. BMC Public Health (2020) 20:1389 Page 11 of 16

aged 5 years or above. In the control arm, about 85% ofhouseholds were consistent toilet users. In this light, a5% increase could be interpreted as a relevant effect giventhat only around 15% of the target households (house-holds with an existing toilet) may have been inconsistentor non-users prior to intervention. It could therefore beargued that a third of those households who were in aposition to improve toilet use behaviour, did so as a con-sequence of the campaign. However, our alternative toolto measure toilet use, the physical activity questionnaire,which we assume to be less likely to be influenced by re-sponder bias, showed no evidence for an effect. We believethe effect estimate of the primary outcome is likely to besubject to over-reporting of toilet use in the interventionarm. This is supported by the comparison of exposed andunexposed in the intervention arm, which resulted in amajor difference in the primary outcome measured by themain tool, but not in the physical activity tool (sum-marised in Fig. 3). The results therefore suggest that the 5Star Toilet campaign in this rural Indian setting with highpre-existing toilet coverage and probable high levels of usedid not further increase toilet use.The campaign was delivered by trained facilitators and

follow-up in the community was done through villagevolunteers. The word ‘smart’ was translated as saru(good), saras (nice) and sunder (beautiful) by the facilita-tors while delivering the campaign. In this manner thecampaign attempted to mainstream the 5 Star Toiletconcept by placing it in the context of other desirable,modern things in people’s lives.There are a number of possible reasons for the failure

of the intervention to achieve major changes in toilet

use behaviour. First the intervention may have been illconceived, second it may have failed to reach enough ofthe target audience with enough intensity to effect meas-urable change, third it may have been delivered to apopulation who were already convinced of the need touse toilets, leaving only a small number of potentialusers who could not be persuaded.Process evaluation data suggests that, for those who

participated, the programme was well received. Throughinterviews and focus group discussions with participantsexposed to the campaign and regional government rep-resentatives, we found that intervention componentssurprised the participants and were different from whatpeople may have experienced before, in government orNGO-led initiatives. Through discussions with partici-pants, the most commonly reported motives for toiletimprovement included comfort, convenience, affiliation,status and honour related to women’s safety. It thus ap-peared that our theory of change for how the interven-tion would lead to toilet use was supported, at least forthose who received the intervention.However, the exposure of the target population to

the intervention was low. Only about 10–15% of theintervention households showed evidence of exposureto the intervention. This low exposure was insuffi-cient to change the study population’s perceptionsaround toilet ownership and other relevant sanitation-related factors at village level. Small positive changesin toilet features and proxy markers of current usewere observed (see Table 4) but statistical support forthese changes was low and could have occurred bychance.

Fig. 3 Comparison of effect sizes resulting from the two questionnaire tools used in the study. The left side shows the effect size of theintervention on toilet use vs control (+ 5.0% for the main tool, + 1.7% for the physical activity tool). The right side shows the effect sizes on toiletuse among participants exposed to the intervention vs those not exposed in the intervention arm (+ 7.0% for the main tool, + 0.9% for thephysical activity tool). P values denote test for interaction

Schmidt et al. BMC Public Health (2020) 20:1389 Page 12 of 16

The low exposure to the 5 Star Toilet campaign mayhave been due to the fact that clusters in the study areawere geographically spread out and various socio-economic strata and caste groups/religious communitieslived in different segments of the village. This challengedthe campaign facilitators as within the short timeframeavailable for the day event, it was difficult to reach outto each and every household. The main occupation ofpeople in the study clusters was agricultural, managinglivestock and diamond polishing, and many had also mi-grated to the nearby cities of Surat and Ahmadabad. Theabsence of householders made it difficult for the cam-paign facilitators to identify eligible households and torecruit participants for the intervention. It is also plaus-ible that the non-users of toilets were a particularlyhard-to contact and hard-to-convince group, since manyof those around them had already adopted the practice.On the whole, intervention intensity was temporarily

high in the locations where activities occurred but geo-graphically too scattered and too infrequent to achieve ahigh exposure at population level. The digital elementsof the interventions were well received by the popula-tion, but were found to be no substitute for the sus-tained efforts on the ground likely to be required toachieve behaviour change.The study findings are in line with other water/sanita-

tion/hygiene-related behaviour change campaigns, such asthose targeting handwashing behaviour where small inter-ventions have shown success [17], while larger campaignsat scale have failed to produce major effects [23–25]. De-livering behaviour change interventions at scale remains achallenge. Three other trials conducted under the sameinitiative alongside the present study in other parts ofIndia also failed to achieve relevant changes in toilet use,even though these were carried out in different settingswith lower toilet coverage and possibly lower baseline toi-let use [20].Rates of usage of toilets, at 84–88%, by a variety of re-

ported measures, were higher in this study than we ini-tially expected, based on small scale surveys prior to theintervention. One solution to the problem of interveningin a population who in the majority did not need tochange behaviour might have been to find a way to tar-get only the approximately 15% of people or householdswho were not using their latrines consistently. From theprogramme perspective, intervention efficiency will bereduced if it mainly consists of activities performed atthe community level such as public events and roadshows. Intervention resources are then wasted on a major-ity of people attending such events who have no need tochange their behaviour. On the other hand, identifyingtarget households within a given community may not beeasy without in-depth knowledge from inside the commu-nity and serial household visits to increase intervention

exposure in those who could benefit from it most. Ap-proaches to identify households not using toilets need tobe conducted in a way that avoids stigmatising householdsbased on income, caste and other status-related character-istics. Thorough formative research taking into accountknowledge from earlier programmes will be needed toguide the decision on whether to favour a community-level or more targeted approach for a given interventionin a particular setting.Limitations of the present study include the use of

self-reported behaviour to measure the primary out-come, imbalances in some socio-economic variablesacross study arms, the low coverage of the interventionand the short time frame from randomisation to inter-vention delivery and outcome assessment.The study relied on self- or proxy-reported toilet use as

the primary outcome, which is likely to lead to over-reporting of socially desirable behaviours. This methodwas used across all four studies in this programme to en-sure standardised reporting of the primary outcome. Inthe setting of a randomised trial testing the effect of anintervention on socially desirable behaviours (here: toiletuse), there is an additional risk of differential reporting be-haviour between intervention and control arm. Study par-ticipants in the intervention arm who have just beenexposed to an intervention may be more prone to over-reporting toilet use than participants in the control arm,for whom the survey may simply appear as just anotherhousehold survey, unlinked to an intervention. Higherover-reporting of toilet use in the intervention wouldcause a spurious effect of the intervention on toilet use.We tried to explore the potential for differential over-

reporting influencing the study results by employing anewly developed tool to measure toilet use and opendefecation – the physical activity tool. Here, going foropen defecation was one of many questionnaire itemsrelated to different physical activities throughout theday, alongside other questions related to chronic non-communicable diseases including dietary pattern. Thistool found a 4.4% points lower toilet use among all studyparticipants (individual level), and there was no evidencefor an increase in toilet use among the interventionhouseholds. These findings are compatible with thepresence of over-reporting in the primary outcome, andsuggest that the observed effect of a 5% percentage pointincrease in toilet use may be due to differential over-reporting. This is supported by the conspicuous differ-ence in reported toilet use in the intervention armbetween those reporting to have heard of the campaignand those that had not. The effect of exposure on re-ported toilet use clearly depended on the questionnaire(Fig. 3). The main questionnaire showed higher reportedtoilet use among those in the intervention arm directlyexposed to the campaign compared to the unexposed.

Schmidt et al. BMC Public Health (2020) 20:1389 Page 13 of 16

By contrast, the physical activity tool showed no differ-ence in this comparison. In our view, this is evidence forcampaign exposure changing reporting behaviour butnot toilet use. The difference between the two tools inthe control arm was not great (84% vs 81%, Table 3),suggesting that in the absence of an intervention, and ifhouseholds are visited only once, over-reporting may belimited.Ahead of the study we suspected over-reporting to

occur even in the control arm if households are visitedrepeatedly before and after the intervention. We tried toreduce this potential for over-reporting by not repeatingquestions related to sanitation and toilet use in the samehouseholds at baseline and at follow up. Householdsundergoing these questions at baseline were discardedfrom further study. This strategy appears to have beensuccessful. Toilet use by all household members at base-line (overall 85%, 87% in the control arm, 83% in theintervention arm) was similar to toilet use by all house-hold members observed in the control arm at follow up(84%), suggesting that the trial procedures did not influ-ence reporting behaviour. These findings further suggestthat administration of the physical activity tool, whichwas done about 7 to 10 days before the endline tool, didnot influence responses of the endline tool, possibly bysuccessfully camouflaging the purpose of the physical ac-tivity survey as a health and lifestyle survey. The lack ofincrease in reported toilet use from baseline to follow upis in contrast to findings from the other three trialswhich were part of this initiative. These trials employedsimilar study and intervention designs, targeting thesame health behaviour (consistent use of already builttoilets by all household members). They all used a simi-lar tool to measure the primary outcome of reported toi-let use behaviour at individual level in each household.However, these trials revisited the same households atbaseline and follow up, whereas we removed all house-holds in the baseline toilet use study from further study.Unlike in our study, these trials found a marked increasein reported toilet use in the control arm (Fig. 4, data ex-tracted from [20]). One could argue that it may havebeen less straightforward for households in our study tolink the purpose of the questionnaire to the interventioncompared to studies repeating the same questionnaire inthe same people with an intervention in between. How-ever, the strong increase in reported toilet use found inthe control arm in the other three trials may also havebeen due to an increase in state- or district-level sanita-tion activities immediately following the baseline, thuscontaminating the trial sites [20].The lack of baseline toilet use data in the households in-

cluded in the endline survey meant that we could not ad-just the effect estimates for any imbalances in the primaryoutcome, or use such data for restricted randomisation to

achieve balance. Some endline imbalances were observedin variables associated with the primary outcome (assetindex and male education). Adjusting for these variablesattenuated the observed effect sizes, but did not funda-mentally change the interpretation of the results. On thewhole we believe that minimising over-reporting and biasby not revisiting the same households before and after anintervention is more important than achieving a high de-gree of baseline balance across arms with regard to thestudy outcome. Bias is difficult to address analyticallywhereas imbalances are due to a chance process whichcan be adjusted for (at least to some extent) and inter-preted in the light of confidence intervals and the resultsof other studies (in meta-analysis).Finally, for programmatic reasons, the trial had to be

conducted in a short time frame. Less than 12monthswere available for baseline study, randomisation, inter-vention delivery, outcome assessment and reporting ofresults. Due to budget constraints, intervention activitiesin each village had to be reduced to 2 days in total. Inretrospect, we doubt whether interventions requiringhouseholds to make changes to their toilets and to mo-tivate household members to change their toilet use be-haviours can result in success within such a shorttimeframe and with such limited resources.

ConclusionThe 5 star toilet intervention did not achieve a convin-cing increase in toilet use behaviour in this rural Indiansetting with high toilet coverage and high usage. Insuffi-cient geographic spread within clusters and low expos-ure among those most likely to benefit from thecampaign are likely to have contributed to the low im-pact of the intervention. Interventions only working atcommunity level without visits to individual households

Fig. 4 Comparison of baseline to endline change in reported toiletuse in control arm across all 4 trials participating in the programme,adapted from [20]. The columns for Gujarat describe the findingsfrom the present study

Schmidt et al. BMC Public Health (2020) 20:1389 Page 14 of 16

may be relatively cheap and scalable but may become in-efficient if only a minority of the population are poten-tial beneficiaries. Future research could be directed tohow to better target large scale sanitation interventionsto sub-populations at greatest need, without stigmatisingeconomically and socially disadvantaged groups. It fur-ther needs to be established how large scale sanitationcampaigns can be incorporated into the overall sanita-tion strategy at local, district, state and national level.Researchers need to develop better tools for assessing

toilet use that are not prone to over-reporting and, inparticular, differential over-reporting between an inter-vention and a control arm. Trials should not rely onself-reported toilet use as the only method for outcomeassessment if the questionnaire design makes the pur-pose of the study obvious to study participants.

Supplementary informationSupplementary information accompanies this paper at https://doi.org/10.1186/s12889-020-09501-y.

Additional file 1. Physical activity questionnaire.

AbbreviationsBCD: Behaviour Centred Design; CI: Confidence interval; ICC: Intra-classcorrelation coefficient; RANAS: Risks, Attitudes, Norms, Abilities, and Self-regulation framework; SBM-G: Swacch Bharat Mission Gramin; ToC: Theory ofChange

AcknowledgementsWe thanks all study participants for their support. We thank Bala Gopalanand Nipa Desai (Upward Spiral) for their contribution to intervention designand delivery. We thank staff members of the Coastal Salinity Prevention Cellfor leading on intervention delivery.

Authors’ contributionsAll authors (WPS, KC, RA, VP, PB, SY, DM, DS and VC) contributed to thedesign of the study, design of study tools, and reviewing the manuscript. KC,RA, and VC contributed to the formative research and interventiondevelopment. KC with DS, PB, VP and SY led on study implementation anddata collection. WPS, KC, PB, SY and VP did the data analysis. All authors(WPS, KC, RA, VP, PB, SY, DM, DS and VC) have read and approved themanuscript.

FundingThe Study was part of International Initiative for Impact Evaluation’s (3 IE)Promoting Latrine Use in Rural India Evidence Programme, funded by the Bill& Melinda Gates Foundation. 3iE had a role in study design, mainquestionnaire tool development and timing of trial procedures, but had norole in physical activity tool development, data collection, data analysis andwriting of the manuscript.

Availability of data and materialsThe datasets generated and/or analysed during the current study areavailable from the authors on request.

Ethics approval and consent to participateWe obtained ethical approval for conducting the study from the LSHTMResearch Ethics Committee (ID 14535) and the IIPHG Independent EthicsCommittee (ID 12/2017). Written informed consent was obtained fromparticipants in their native language (Gujarati).

Consent for publicationNot applicable.

Competing interestsNone declared.

Author details1Environmental Health Group, Department of Disease Control, LondonSchool of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT,UK. 2Indian Institute of Public Health Gandhinagar, Opp. Airforce HeadQuarters, Nr. Lekawada Bus Stop, Chiloda Road, Lekawada CRPF P.O,Gandhinagar, Gujarat 382042, India. 3Center for Development Research (ZEF),Bonn, Germany.

Received: 25 February 2020 Accepted: 4 September 2020

References1. Cumming O, Arnold BF, Ban R, Clasen T, Esteves Mills J, Freeman MC,

Gordon B, Guiteras R, Howard G, Hunter PR, et al. The implications of threemajor new trials for the effect of water, sanitation and hygiene onchildhood diarrhea and stunting: a consensus statement. BMC Med. 2019;17(1):173.

2. Schmidt WP. Seven trials, seven question marks. Lancet Glob Health. 2015;3(11):e659–60.

3. Sahoo KC, Hulland KR, Caruso BA, Swain R, Freeman MC, Panigrahi P,Dreibelbis R. Sanitation-related psychosocial stress: a grounded theory study ofwomen across the life-course in Odisha, India. Soc Sci Med. 2015;139:80–9.

4. WHO, UNICEF: Joint Monitoring Programme (JMP) for Water Supply andSanitation. 2014. http://www.wssinfo.org/.

5. Routray P, Torondel B, Jenkins MW, Clasen T, Schmidt WP. Processes andchallenges of community mobilisation for latrine promotion under NirmalBharat Abhiyan in rural Odisha, India. BMC Public Health. 2017;17(1):453.

6. Routray P, Schmidt WP, Boisson S, Clasen T, Jenkins MW. Socio-cultural andbehavioural factors constraining latrine adoption in rural coastal Odisha: anexploratory qualitative study. BMC Public Health. 2015;15:880.

7. WaterAid India. Feeling the Pulse. A Study of the Total Sanitation Campaignin Five States. New Delhi; 2008.

8. Sinha A, Nagel CL, Schmidt WP, Torondel B, Boisson S, Routray P, Clasen TF.Assessing patterns and determinants of latrine use in rural settings: alongitudinal study in Odisha, India. Int J Hyg Environ Health. 2017;220(5):906–15.

9. Swachh Bharat Mission (Gramin) Dashboard. Swachh Bharat Mission Portal.https://sbm.gov.in/sbmdashboard/. Accessed 10 Sept 2020.

10. Curtis V. Explaining the outcomes of the 'Clean India' campaign:institutional behaviour and sanitation transformation in India. BMJ GlobHealth. 2019;4(5):e001892.

11. Office N-NSS: SBM status Report. 2016.12. institute R: SQUAT survey. 2018. https://riceinstitute.org/data/2014-and-2018-

rural-sanitation-surveys/.13. NARSS: National Annual Rural Sanitation Survey. 2019. http://www.

indiaenvironmentportal.org.in/content/465583/national-annual-rural-sanitation-survey-narss-round-2-2018-2019-national-report/.

14. Pattanayak SK, Yang JC, Dickinson KL, Poulos C, Patil SR, Mallick RK, BlitsteinJL, Praharaj P. Shame or subsidy revisited: social mobilization for sanitationin Orissa, India. Bull World Health Organ. 2009;87(8):580–7.

15. Lilje J, Mosler HJ. Effects of a behavior change campaign on householddrinking water disinfection in the Lake Chad basin using the RANASapproach. Sci Total Environ. 2018;619-620:1599–607.

16. Aunger R, Curtis V. Behaviour Centred design: towards an applied science ofbehaviour change. Health Psychol Rev. 2016;10(4):425–46.

17. Biran A, Schmidt WP, Varadharajan KS, Rajaraman D, Kumar R, Greenland K,Gopalan B, Aunger R, Curtis V. Effect of a behaviour-change intervention onhandwashing with soap in India (SuperAmma): a cluster-randomised trial.Lancet Glob Health. 2014;2(3):e145–54.

18. Seimetz E, Kumar S, Mosler HJ. Effects of an awareness raising campaign onintention and behavioural determinants for handwashing. Health Educ Res.2016;31(2):109–20.

19. Gautam OP, Schmidt WP, Cairncross S, Cavill S, Curtis V. Trial of a novelintervention to improve multiple food hygiene behaviors in Nepal. Am JTrop Med Hyg. 2017;96(6):1415–26.

20. The RICE institute: Promoting-latrine-use-in rural-india. https://www.3ieimpact.org/our-work/water-sanitation-and-hygiene/promoting-latrine-use-rural-india-evidence-programme.

Schmidt et al. BMC Public Health (2020) 20:1389 Page 15 of 16

21. Chauhan K, Schmidt WP, Aunger R, Gopalan B, Saxena D, Yashobant S,Patwardhan V, Bhavsar P, Mavalankar D, Curtis V: 5 star toilet campaign-improving-toilet-use-rural India. 2019. https://www.3ieimpact.org/evidence-hub/publications/impact-evaluations/5-star-toilet-campaign-improving-toilet-use-rural.

22. Clasen T, Boisson S, Routray P, Torondel B, Bell M, Cumming O, Ensink J,Freeman M, Jenkins M, Odagiri M, et al. Effectiveness of a rural sanitationprogramme on diarrhoea, soil-transmitted helminth infection, and childmalnutrition in Odisha, India: a cluster-randomised trial. Lancet Glob Health.2014;2(11):e645–53.

23. Huda TM, Unicomb L, Johnston RB, Halder AK, Yushuf Sharker MA, Luby SP.Interim evaluation of a large scale sanitation, hygiene and waterimprovement programme on childhood diarrhea and respiratory disease inrural Bangladesh. Soc Sci Med. 2012;75(4):604–11.

24. Briceno B, Coville A, Gertler P, Martinez S. Are there synergies fromcombining hygiene and sanitation promotion campaigns: evidence from alarge-scale cluster-randomized trial in rural Tanzania. PLoS One. 2017;12(11):e0186228.

25. Lewis HE, Greenland K, Curtis V, Schmidt WP. Effect of a school-basedhygiene behavior change campaign on Handwashing with soap in Bihar,India: cluster-randomized trial. Am J Trop Med Hyg. 2018;99(4):924–33.

Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.

Schmidt et al. BMC Public Health (2020) 20:1389 Page 16 of 16