1

1st PartMaking Sense of Data in Emergencies

• The impact of emergencies on information• Lack of information• Information and decision-making• Features of information• Sources of information flaws• Making sense of poor data

S.C. October 2003

2

How do emergencies impact on information?

• Routine information systems collapse:– Poor communication,– Reduced access limited coverage

• Population movements lack of reliable denominators

• Collection & analysis of data clash with other priority activities

• Increased background noise

3

Lack of information?

• Lack of information as the strongest indicator of a crisis

• Lack of information as the best piece of information for crisis decision-making

but

Lack of information is not = to lack of data

4

From facts to decisions in an IDEAL world

Judgement / decisions / actions assessing options /

Decision-making attributing values

Facts/events deaths collection / recordingData nr. of deaths countingIndicators mortality rates in sorting, selecting

a pop.,area, period calculatingInformation rates by sex, age, analysis

provenience, etcKnowledge / trend analysis, interpretation

Understanding comparison withother areas, benchmarks,

contextual information

Steps example intelligence at work

5

From facts to decisions in the REAL world

• Decision-making involves political and personal values, especially in complex emergencies:

• “priority setting is ultimately concerned with power relations and value judgements” (Green & Barker, 1988)

6

(Non)-information and (non)-decision-making

• Informational exclusions

• Non-decision making as a frequent strategy in difficult contexts

• The extreme case of non-decision making:“paralysis by analysis”

• Decision-making without information

7

Features of information in emergencies

• Timeliness • Validity • Relevance• Precision:

– “precise quantification of incidence and prevalence rates rarely influences the choice of programmes offered by the health services” (Sandiford, 1992)

• Cost

8

The opportunity cost of information in emergencies

Cost

: ca

pacities

& o

ppor

tunities

9

Validity and Precision

C. Precision only

A. Both validity and precision B. Validity only

D. Neither validity nor precision

Jekel, 1996

10

Bias versus Lack of Precision

• Assuming that in the area X in August 2002 the “real” (unknown) CMR is 1.6/10,000/day

– Valid (unbiased) & precise estimate: 1.5 (1.4 - 1.7)

– Valid & imprecise estimate: 1.4 (1.0 – 2.2)

– biased estimate: 0.6 (0.3 - 0.8)

– imprecise estimate: 1.2 (0.8 - 1.9)

– biased & imprecise estimate: 0.6 (0.1 - 1.2)

11

Common sources of information flaws

• Inconsistent denominators• Biases • Inconsistent levels of aggregation • “Brand”, bad indicators• Incomplete reporting• Over-reliance on quantitative techniques

and/or presentation tools, applied to poor data

12

Making sense of poor data

• Linking indicators related to the same areas and produced by different sources,

• comparing them for reliability,• checking suspect data at their sources,• comparing them with other standards,

triangulating them with other sources,• inquiring why certain data are not reported,• discarding irrelevant and/or clearly unreliable

data,• validating data, information and interpretation

with knowledgeable people

13

Statistics, epidemiology and intelligence

• “For scientists, linked to respectable statistical methods, complexities can be an unmanageable nightmare: for, if they simplify them until they are measurable, they destroy the complexities ..” (Chambers & Toulmin, 1989)

• Statistical and epidemiological techniques can only rarely be applied in the messy context of an emergency,

• however, they are crucial, together with field experience, for developing hunches for figures and guiding the analyst in gradually making sense of poor data

14

2nd part:Health Needs Assessment

in Emergencies

• NA as a function of health intelligence• NA: meaning, subject matter, & scope• Definition and classification• Key questions that NA address in an emergency• Preparing for a NA• An example of template• tips

15

Main functions of Health Intelligence

Assessment on-off past, present &future

Evaluation on-off past (present)

Monitoring continuous past, present &future

Function Periodicity Time scale

16

Needs Assessment in Health

What are we talking about?• Needs? • Risks / capacity?

What is the subject matter?• Health status of the affected population?

• The determinants of ill-health?

• The performance of health services?

17

Needs Assessment in Health (cont.d)

Context:• NA in emergency?• NA in recovery / reconstruction?

Scope:• area-based?• Country / sector-wide (CAP, reconstruction)?

18

What a RHA is and for what is used?

“collection of subjective and objective information in order to measure damage and identify those basic needs of the affected population that require immediate response”

From: RHA protocols for emergencies, WHO, 1999

19

Health assessment and information systems

• Routine:– Surveillance Systems– Health (Unit-managed) Information Systems– Civil registration (vital statistics)

• Non-routine:- Rapid Health Assessments (RHAs)- Surveys

20

Non-routine data collection methods in emergencies

Type When What How

Rapid reconnaissance

Immediately after a disaster

A quick, preliminary inspection of the disaster area

• Satellite imagery • Overflights • Mapping • Drive/walk through

RHA As soon as it is

possible to go to the area

A quick collection of information to confirm the emergency, measure the impact, identify health needs and guide response

• Visual inspection • Analysis of records • Interview of key

informants • Rapid surveys

(MUAC, etc)

Surveys When the situation stabilises and response has been activated

A detailed study in which information is systemati-cally collected in a sample of population (morbidity, mortality, nutrition, KAP)

• Probability sampling • Non-probability s.

21

HypotheticalTimeframe

Alert Decision 1

• Whathappened?

• Where?• RHA?• When?

Pre-assessmentsitrep

??

Preparation

• To do what?(ToRs)

• Which team?• Where?• How?

( methods)• How? (logistics)

few hours/days

RHADecision 2

• Emergency?Y/N

• What needs?• What

constraints?• What local

resources?• Response?• Which one

(which resources)?

• What evolution?

1-4 days

Emergency

0

Analysis

• Action?• If yes,

which one,whichresourcesrequired?

possiblyduring RHA

Referencevalues

Sitrep

• Presentsmainfindings

• Givesrecommen-dations:

• What to do?• Why?• By whom?

1 day

ActionDecision 3

• Who doeswhat?

• When?• How?• With whom?

??

Political/financialconsiderations

Needs assessment and decisions: the time sequence

22

Which are the key questions in a RHA?

• Is there an emergency or not?

• What is the main health problem?

• What is the existing response capacity?

• What decisions need to be made?

• What information is needed to make these decisions?

23

Which information?• The population:

– numbers, characteristics, & trends– morbidity and mortality

• The vital needs:– security– food– water– shelter & sanitation– clothes and blankets– domestic utensils and fuel– health care

• The support systems:– information– logistics– coordination– resource flow

• Other relevant contextual issues

24

Main steps of a RHA

• Set the assessment priorities

• Collect the data:

– reviewing existing information

– inspecting the affected area

– interviewing key people

– carrying out a rapid survey

• Analyse and interpret the findings

• Present results and conclusions

25

Preparing for a RHA

• What should I know before going to the field?

• What methods are appropriate, considering:– the given emergency context, and

– the security, time, logistic, technical constraints?

• What composition of the NA team?

• Which logistics, communication & transport?

26

What is available in the EHA webpage?

• Short introduction with objectives• SitrepTemplate• Instructions• Reference Values

27

28

29

Advantages of using a standard template

• Quicker and comparable analysis

• Ensures all important items of information are included

• Consolidates information from different sources into a single document

• But, remember, the standard template is just a guide and can be adapted to your needs

30http://www.who.int/disasters/sitrep.cfm

EHA first sitrep

31

Reference Values

32

RHA: a few tips (1)• Concentrate on your sector, but don’t lose sight of

the context

• Concentrate on the NOW, but look at the past(WHY?) and think of the future (...WHAT IF...?)

• use local knowledge:– “a good rapid assessment is one that accesses as broad a

cross-section of local knowledge as possible” (Collins, 2001)

• Don’t create excessive expectations! A NA, as a rule, should be followed by response

• Share with your team, report to your HQ, but leave something for who remains in the field

33

RHA: a few tips (2)

• Don’t be too ambitious: time is short

• Being roughly right is generally better than being precisely wrong or precisely late

• Beware: wrong conclusions from the RHA can do more harm than not taking any action

34

What is needed for a RHA?• Clear lines of authority and reporting• Partnerships• Division of responsibilities and agreed procedures• Maps• Transport• Radio or mobile/sat phones• Tent, food?• Security clearance• Qualified personnel• Interpreters (if no local collaborators are part of the

team)• Data collection forms, containers for specimen, other

equipment• Guarantee of follow-up (response, other assessments)

35

CMR: reference values

• “normal” in developed countries: 0.2 per 10,000 per day• “normal” in developing countries: 0.5 “ “ “• crisis (under control): < 1.0 “ “ “• very serious: > 1.0 “ “ “• out of control: > 2.0 “ “ “• famine, major epidemic,

catastrophic: > 5.0 “ “ “• Ajiep, South Sudan, in 1998: 26.0 “ “ “• Rwanda, 1994: 19.4-30.9 “ “

36

Defining humanitarian needs

• basic human needs (“food is a basic need”)

• lack of - or deficit in - the above(“these people need food”) -norm / capacity, not risk

• need for relief assistance (“these people need food aid”) -response

Adapted from Darcy & Hofman, ODI, 2003

37

Defining needs• Need for a health care intervention, an intermediate

state (a product of derived demand in economics)• Need as the capacity to benefit from health care

(economist’s view concerned about the change in state), implying that an effective intervention must be available

• Need for a desiderable end state (e.g. health)• Normative need: I.e. defined by professionals (an

example of a socially constructed need)• Felt need/wants: individually experienced• Demand: expressed need• Comparative need: against a reference group• Need as a deficit: the difference between a norm -

2,100 Kcl- and the reality (the actual intake of calories)

38

Defining risk

• Hazard (threat) x vulnerability• it is a mix of probability and consequences:

• risk of an earthquake is the probability of its occurrence

• risk from an earthquake refers to its consequences (death, injury, etc)

From R.Doran, 2002

39

3rd part:The practice of health needs assessments.

Lessons from Southern Iraq

• The environment for NA• The outbreak of Nas• Assessing for what?• Recurrent flaws in Nas• Lessons learned• Conclusions

40

The environment in Iraq in the aftermath of the war

• Limited humanitarian space• No major humanitarian crisis under way • High-visible crisis, whose political overtones

sidelined health-related considerations • Limited knowledge of the health sector by most

participants• Unique features of the crisis and of the health

sector, not anticipated by newcomers

41

An outbreak of needs assessments?

• Many agencies embarked in needs assessments, which became their many and often only activity

• 1,600 health facilities (75% of the entire network) were inspected between April and June

• Coverage was very uneven. Many facilities were assessed several times, whereas ¼ of the network remained inaccessible

• A new geography of accessibility: little attention to systemic issues and administrative division of the country

• The supposed scope of the needs assessments shifted towards lying the grounds for the formulation of reconstruction plans

42

Assessing for what?

• Different instruments and criteria were applied, generating a wealth of data impossible to aggregate

• Compiling these data was compared to ‘herding cats’(and generated an analogous outcome)

• The yield of this impressive activity was remarkably modest in terms of understanding the overall situation of the health sector.

• The discussion about reconstruction, started in June, relied on approximate estimates based on heroic assumptions, formulated in the absence of hard data, despite the previous enormous fieldwork investment

43

Some recurrent flaws in needs assessmentsmay be recognised across crises:

– To apply a micro approach to macro issues– To opt for rapid assessment techniques because of

their apparent simplicity or ease of application– To mistake rapid needs assessments with the data

collecting tools designed to contribute to them– To apply a short-term, survival-oriented perspective

to problems calling for long-term, recovery-oriented solutions

– To jump to assess end-results of long chains of events

– To assess needs with already a response in mind– To limit only to those needs that are "easy" to

assess

44

Lessons learned from the Iraq crisis

• Tap local knowledge from the beginning of the crisis• Approach issues systemically rather than piecemeal• Concentrate on a few selected critical areas calling

for urgent intervention• Work inside the system to the maximum extent• Verify in the field whether the identified priorities

are truly such• Co-opt NGOs to support this strategy• Advocate for restraining from hurried-up initiatives

or for measures not called for by true needs• Start long-term processes, such as human resource

development, as soon as possible

45

Conclusions

– In many complex situations, the primary source of intelligence is the unused information, scattered around in the health environment and waiting to be tapped

– No formalised, ‘technical’ approach can substitute common sense, a solid cultural background, field experience and familiarity with the issues at stakes