1

Surveys in Humanitarian Emergencies

2

Methods of Data Collection

Assessment Survey Surveillance

Objective Rapid appraisalMedium-term appraisal

Continuous appraisal

Data TypeQualitative/

Cross sectional snapshot

Quantitative/

Cross sectional

snapshot

Quantitative/ Longitudinal trends

MethodObservational /

Secondary source

Sample with survey instrument

Periodic, standardized data collection

3

Overview

When to do a survey

The need for a standardized tool and methodology

Key activities in designing and implementing health and nutrition surveys

Survey interpretation

4

Surveys in Emergencies

Why would you consider doing a survey?

When would you do a survey?

5

Why Do a Survey in an Emergency

Estimate prevalence (example acute malnutrition)

Determine mortality

Prioritize interventions

Collect baseline data for program planning

Evaluate program success

Advocate for intervention

6

When To Do a Survey in an Emergency?

After the initial emergency phase has passed

After basic needs for survival been met

When major population movement has stopped

When you are in a position to take action

After preparatory information has been collected

When security allows

7

Difference Between Emergency and Development Surveys

Emergency surveys• Must be done with action in mind• Must be implemented and analyzed quickly• Hard to find trained/trainable personnel• Difficult to come up with sampling frame• Multi sectoral, multi partner (UNICEF, UNFP,

etc.)• Security

8

The Problem: Non-standardization of Methods

Ethiopia 1999-2000

Among 67 nutrition and mortality surveys, only 6 (9%) met all 5 eligibility criteria to be valid and precise*

Probabilistic sampling

Sample size > 500 children aged 6-59 months

25 clusters

20 or more children per cluster

Weight for height used as anthropometric index

Source: Spiegel P et al, JAMA 2004

9

Survey design and implementation: key tasks

Determine objectives of the survey

Determine broad questions to be answered outcomes to be measured

Estimate Budget needed and Identify key partners

Define the sampling frame & sampling design

Design the questionnaire, translate, and test

Conduct training , plan the logistics, equipment, and survey team needed

Data entry, analysis, and Interpretation of results

Preparation and dissemination of results

Take action

10

What Might Be Included in an Emergency Survey?

Demographics Household

Characteristics Child Health Child Nutrition Reproductive health Mortality

Food Sources and Consumption

Food Aid deliveries & usage

Health interventions Water sources Sanitation

11

Defining The Sampling Frame

What population do you want to extrapolate your results to?

What population is or will be the target of the program?

Is the population homogeneous? Are there geography and access issues? Security conditions?

12

Sampling Frame for the Darfur Survey

13

Sampling Frame for the Ethiopia Survey

Population not homogeneous

• Highlands versus lowlands However, intervention targeted by Woreda

14

Decisions Concerning Sampling Design

Probability sampling: Simple random sampling Systematic random

sampling Cluster sampling

Non-probability sampling: Key informants Convenience sampling Purposive sampling

Sampling methodology: types

15

Each individual or sampling unit in the population has the same chance or probability of being selected

The selection of one individual should be independent of the selection of another

Representative Sampling

16

Simple Random Sampling

Most basic type of sampling Selection of units independent and random Steps:

• Number each sampling unit

• Choose new random number for each selection

17

Systematic Random Sampling

Similar to simple random sampling First sampling unit chosen randomly Systematic selection of subsequent sampling units Steps:

• Compute sampling interval (SI) (Number in population / Sample size)

• Select random start between 1 and SI

18

Simple and systematic random sampling

What is required for both simple and systematic random sampling?

Both require a complete list of all basic sampling units.

19

Calculating the sample size For A Random Sample or Systematic Sample

The sample size is calculated using the following formula:

n = { t2 x p x q }

d2

Where: n = sample size

t = the risk of error (1.96 or 5% error)

p = expected prevalence (fraction of 1)

q = 1- p (expected non-prevalence)

d = level of precision (fraction of 1)

Sample size should by multiplied by design effect in cluster surveys

20

An example of calculating the sample size

The sample size is calculated using the following formula:

81 = { 1.962 x 0.3 x 0.7 }

0.12

Where: n = sample size

t = the risk of error (1.96 or 5% error)

p =expected prevalence (fraction of 1)

q = 1- p (expected non-prevalence)

d = level of precision (fraction of 1)

21

Cluster sampling

Objective: To choose smaller geographic areas in which simple or systematic random sampling can be done

22

Simple Random Sampling

Simple randomSampling: (300 households required)

Samplinguniverse

Selected households

Non-selected households

23

Cluster samplingCluster sampling : 30 Clusters of 10 households each

1

2

3

But, because we are using cluster sampling, we must increase the sample size because we are not choosing randomly, but households are clustered together. May end up choosing 30 clusters of 15 households.

24

The sample size is calculated using the following formula:

n = { t2 x p x q } x design effect

d2

Calculating the sample size in cluster surveys

25

Which sampling method here?

26

Which sampling method here?

27

Common Problems With Sampling

• For surveys:- Was a random or probability sample taken using a recognized

method?– Did everyone have an known probability of selection?

Who was left out? (selection bias)– What was the geographical coverage?

• Does it relate to catchment area of program?– Were case definitions clear and standardized and piloted or

validated?– How was age determined?– What was the sample size?– Are confidence limits reported?– Watch out for denominator of rates

28

Sampling - Goal

Regardless of the sampling method, if the sample is not representative, you can’t generalize the findings to the whole population.

Remember this point!

29

Sampling – Sample size

So how do we decide on the needed precision? One-time results for advocacy alone does not

need much precision (0.10 good enough) Results that you will need to compare against in

the future need greater precision (0.05 if program will have large impact)

Results you will monitor frequently (e.g. year by year) need even greater precision (0.02)

30

Adjustments to Sample Size

Source of information about number of analysis units (e.g. adult males) per samping unit (household)• Calculate from census statistics• Use previous surveys to calculate

Finally, need to add margin for non-response• Look at previous surveys• May differ by region of country• May be higher for some measures (e.g. blood)

31

Training & data analysis

Training needs to include: what data to collect, how to select households, data collection method, consent form, interview method

Data entry: advise to begin data entry simultaneously as data collection

Data analysis: select the analysis tool (Epi-info, SPSS), identify key personnel

32

Interpreting ResultsMeasles coverage: Measles immunization coverage 59.4% [CI:54.6-

64.1]

Major morbidity in last two weeks: Watery diarrhea 17.2% [CI:13.8-21.2] Mortality rates: (over 7 month period) Crude mortality rate: 3.2/10,000/day <5 mortality rate: 9.8/10,000/day

33

Interpretation: Other Factors

Trends and change Confidence Intervals (CI) Seasonality Aggravating factors or risks Baseline or ‘normal’ prevalence Prevalence of other types of malnutrition e.g.

chronic malnutrition Mortality levels

34

Mortality Trends in Selected Refugee Camps

0

2

4

6

8

10

1 2 3 4 5 6 7 8 9 10

Months

CM

R: D

eath

s/10

,000

/day

Thailand: 1979-1980 Somalia: 1980-1981

35

Assessing the Quality Of Survey Implementation

Did the survey answer the question it was designed to? Were objectives clear measurable and measured? Was the sampling frame adequate? Questionnaire

• Clear?

• In local language?

• Translated, back translated? Sampling design

• Representative?

• Sample size adequate?

36

Assessing the Quality Of Survey Implementation

Logistics, equipment, and survey team

• Equipment standardized?

• Training adequate?

• Personnel? What checks for data quality have been done? Do you agree with the interpretation of results? Have results been disseminated to all partners What action is planned?

37

One Way to Evaluate MethodsCharacteristic Assessment Points

Sampling frame Unadjusted

Adjusted

1

2

Sampling method Convenience

Random

1

2

Sample size Small 1

Good (justified) 2

Geographical coverage Small

Appropriate

1

2

Case definitions Not defined Standardized

1

2

Confidence limits Not reported 1

Appropriate 2

38

Distribution of ht/age z-scores, Country X, 2000

02

468

10

121416

1820

-6.00to -5.0

-4.99to -4.5

-4.49to -4.0

-3.99to -3.5

-3.49to -3.0

-2.99to -2.5

-2.49to -2.0

-1.99to -1.5

-1.49to -1.0

-0.99to -0.5

-0.49to 0.0

0.01to 0.5

0.51to 1.0

1.01to 1.5

1.51to 2.0

2.01to 2.5

2.51to 3.0

3.01to 3.5

3.51to 6.0

Ht/Age z-score

Per

cent

of

child

ren Survey

NCHS reference

Standard deviationof sample = 1.75

Assessing The Quality of the Survey

39

Take Action

Interpret and understand findings Review and revise program objectives Advocate for resources e.g. food pipeline or access to

clean water Address underlying causes of poor health or nutrition Increase coverage of programs

• Vaccination Close emergency nutrition programs Use findings as part of wider country information system Use findings as baseline data