Rapid mobile phone based surveys (Scott Chaplowe, IFRC)
Transcript of Rapid mobile phone based surveys (Scott Chaplowe, IFRC)
www.ifrc.orgSaving lives, changing minds.
Rapid Mobile Phone-based Surveys (RAMP) for Evidence-based Emergency Response
ALNAP 28th Annual Meeting,5-7 March 2013, Washington, D.C.
Scott Chaplowe, Senior M&E Officer, IFRCRose Donna, Director, Datadyne.org
Jason Peat, Senior Officer Public Health, IFRCAmanda Mcclelland, Emergency Health Officer, IFRCJoel Selanikio, CEO DataDyne GroupMac Otten
www.ifrc.orgSaving lives, changing minds.
Presentation Overview Application of mobile technology (RAMP) to address specific challenges in data collection during emergency operations.
1) Introduce RAMP
2) How RAMP works
3) Emergency contexts
4) Key considerations
www.ifrc.orgSaving lives, changing minds.
What is RAMP?
RAMP (Rapid Mobile Phone-based Surveys) is a survey methodology utilizing mobile phones to help RCRC National Societies, governments, NGOs and other partners efficiently conduct quality surveys that:
Reduced time
Reduced cost
Improved quality assurance
Limited external technical assistance
www.ifrc.orgSaving lives, changing minds.
RAMP Background (www.ifrc.org/ramp)
1. Developed by IFRC in partnership with WHO, CDC, and other partners.
2. Initial focus = malaria program household surveys Four pilots in Africa 2011-2012 (Kenya, Namibia and Nigeria),
3. Refine and developed trio of user guides: 1. Designing a RAMP survey 2. Implementing a RAMP survey3. Training a RAMP survey team
4. Scale-up to other program areas – increase survey functionality – use of SMS
www.ifrc.orgSaving lives, changing minds.
RAMP takes advantage of 2 technologies1. Mobile phone to collect data(Low-cost, standard mobile phones, as well as Android, Symbian, Blackberry, SMS, and iPhone)
2. Web-based software application Enables mobile phones to become a data collection platform
www.ifrc.orgSaving lives, changing minds.
How does RAMP work?
5. Data Reports
2. Data collection on phone
1. Develop survey on website
3. Transmit data
4. Collate/analyze data on computer
www.ifrc.orgSaving lives, changing minds.
Connectivity
InternetRequired
• Create/edit surveys
• View/export data
• Create reports
InternetNot Required• Collect data
Can be cellular, wifi, cable
www.ifrc.orgSaving lives, changing minds.
Data monitoring and analysis
Preliminary analysis available before data collection is complete
www.ifrc.orgSaving lives, changing minds.
Survey bulletins/updates Full survey reports
Timely Reporting
www.ifrc.orgSaving lives, changing minds.
Digital Data Collection – Changing the way we work
Paper questionnaires filled out in the field
Data entered into a computer at a central location
Data analysis and reporting often takes months to complete
Local capacity is often under-utilized and there is a dependence on external experts
Mobile and internet-based technologies used to reduce time for data collection to reporting
Enables rapid reporting of results, decision making, and action
Empowers local ownership of evaluation and research
The “old” The “new”
www.ifrc.orgSaving lives, changing minds.
Vaccination coverage
Surveillance
Supply chain management
Household surveys
Clinic surveys
Supervisory checklists
Anything that can be put on a form
www.ifrc.orgSaving lives, changing minds.
RAMP Potential in Emergencies?
Beginning to explore the potential of RAMP in emergency context: Site assessment – needs, damage
Community assessment – needs, damage
Beneficiary registration
Distribution of emergency (and non-emergency) items
Baseline/endline data collection (monitoring and impact study)
Repeated surveys to track time trends for key indicators
Beneficiary communication – (broadcast Terra)
Beneficiary/community monitoring
Disaster preparedness – EWS monitoring
www.ifrc.orgSaving lives, changing minds.
SMS Disease Surveillance Systems Piloting in community based disease surveillance
Sierra Leone – 400 community volunteers distributing ORS.
Referred only 5% of cases of AWD they saw in community = only 5% of cases were potentially recorded in normal MoH system.
RAMP allows real time communication and data gathering suitable for this context.
Problems with integration and harmonization of data between community and MoH.
But SMS proved real time information to assist program prioritization in outbreak scenarios.
www.ifrc.orgSaving lives, changing minds.
SMS Considerations Simplified questions rather than full surveys
Coding syntax with 2 to 7 key variables as best practice
Quantity of messages handled depend on networks, and whether staggered or simultaneous reporting.
Paper form can be used to facilitate data entry to SMS
Quality assurance auto feedback
Reminder SMS to field person to report data at a set time
Thank you SMS to confirm receipt of data.
Ability to send airtime to the mobile account if someone reports from a common central account.
www.ifrc.orgSaving lives, changing minds.
Benefits?
www.ifrc.orgSaving lives, changing minds.
Benefits – decision making
Data rapidly available for decision-making
Maintain data control
Scalable for studies of varying sizes
Shared, electronic database to compare across contexts and with partners to build a body of evidence related to impact
www.ifrc.orgSaving lives, changing minds.
Benefits - management Cost effective
Do not have to reinvent the wheel – Adaptable RAMP toolkit
Consultants not required
No software licensing or subscriptions
Multiple languages (depending on program)
Export data for custom analysis using any statistical analysis package
Additional SMART phone features
www.ifrc.orgSaving lives, changing minds.
Benefits - management
Online library of survey forms
Collect and aggregate data form multiple areas and partners
Ease of creating and changing analyses/reports
Efficient reporting and dissemination
www.ifrc.orgSaving lives, changing minds.
Benefits - Fieldworkers
• Build local capacity for M&E
• Standard and familiar mobile phones
• No more paper to collect, transport or return
• Automated data submission (assuming network)
www.ifrc.orgSaving lives, changing minds.
Benefits - Quality Assurance
Remote QA: Enables monitoring of survey team work rate, productivity and quality Monitor times/location of data collection (time/date data stamps) Provide feedback remotely
Efficient data management reduces “paper” mistakes Easier to back-up forms/data Reduced error of repetitive data entry and re-entry Easier to change and update forms
Immediate QA: Real-time error analysis and field
correction Utilize skip patterns, custom logic and
validation
www.ifrc.orgSaving lives, changing minds.
Reality Check!
Not suitable for very long questionnaires
No “magic bullet” –work is still in the details!
Things to improve – i.e. offline form generation
Technology is a moving target – (hardware and software)
Challenges resource development/training
(But also means improvements and reduced costs)
www.ifrc.orgSaving lives, changing minds.
Questions to Consider
What applications do you see for mobile data collection in the humanitarian sector?
What has worked well?
What hasn’t worked well?
www.ifrc.orgSaving lives, changing minds.
www.ifrc.org/ramp
Package of field-friendly User Guides:1. Volume 1: Designing a RAMP survey: technical considerations2. Volume 2: Implementing a RAMP survey: practical field guide3. Volume 3: Training a RAMP survey team: guide for trainers
Living archive of additional resources: Example database and STATA files for data cleaning and analysis of a
sample malaria survey Latest up-to-date malaria questionnaires and STATA files for data
cleaning and analysis Country reports and results bulletins, information, useful links
www.ifrc.orgSaving lives, changing minds.
www.ifrc.orgSaving lives, changing minds.
The following slides are extra and can be referred to if needed, (but unlikely).
www.ifrc.orgSaving lives, changing minds.
Cost of a IFRC RAMP HH survey for Malaria programming (average)
Description Cost (US $)
Training (4 or 5 days) including two facilitators 10,623
Field survey, including transportation, daily allowances and accommodation 12,415
Mobile phones, accessories and air time 3,806
Survey administration 2,243
Total in-country expenditure (US $) US $ 29,087
www.ifrc.orgSaving lives, changing minds.
When might a RAMP survey be suitable? FlexibilityItems that can be adjusted Comments
Adjust precision ±10%, 5%, 3%, etc.
Adjust indicator type (denominator of indicator) including mixtures of indicator types
- Person all ages- Children <5 years old- Pregnant women- Households- Schools
Adjust number of domains - 1 domain with 30 clusters- 2 domains with 30 clusters to
compare statistically - 10 domains with 30 clusters each to
compareAdjust overall sample size - 200 to 5000 households
www.ifrc.orgSaving lives, changing minds.
How is the RAMP method different from MIS/DHS?MIS/DHS RAMP
Complex design, uses external consultants to design survey
Simple design, external consultants not needed
Listing of all households is done in all clusters; expensive, often taking several days in each cluster
Divides clusters into manageable-sized segments using standard survey methods; takes <1/2 day
Simple random sampling of households (from the cluster list)
Simple random sampling of households (from the final segment list)
Real-time data cleaning not possible Real-time data cleaning during the survey
Real-time data analysis not possible; results take several months
Real-time data analysis and results/draft report finished within 3 days of last interview
Data analysis done by third-party consultants
Organization performs analysis, building capacity and maintaining control of data
www.ifrc.orgSaving lives, changing minds.
Mobile application
Record field data, even without network coverage
www.ifrc.orgSaving lives, changing minds.
RAMP
Based on standard survey sampling methodology
Web-based platform for survey design, data storage, analysis, reporting and data export.
Field-based data entry through mobile phone application.
Questionnaires downloaded to standard mobile phones
Web-based dataset that can export “real-time” for rapid analysis and reporting