Post on 17-Jul-2015
Errors and biases in
decision making process
Implications for public health
Aymery Constant
Maître de Conférences
Psychologie de la santé
Comparison of Risk in Health Care
With Other Industries
MODERATE RISK
MINIMAL RISK (<1/100,000)
HEALTH CARE
Bungee jumping
Driving
Chemical Manufacturing
Commercial Aviation
Nuclear Power
HIGH RISK (>1/1000)
Number of Encounters
Liv
es L
ost/
Year
Modified from R. Amalberti and L. Leape
Medical decision • Diagnoses
• Choosing a treatment?
• Initiate a new treatment? Keep the status quo?
• Death or disability?
• What’s the priority?
• Life? Well be? Freedom? Duty? Compassion?
Decision making
1) Logical and rational process
2) the Accumulation of knowledge improves the decision
3) Past experience improve judgment
4) collective decisions are more reliable
= MYTHS
Bounded Rationality
Case study
Ms. M., 62, visits her GP because she feels tired all the time and is
struggling to manage her daily activities.
Her GP ordered blood tests, that showed elevated levels of
transaminases (liver enymes)
He declared that she drinks too much alcohol and recommended
total abstinence. He talked about possible treatments, an hospitalization to
perform complementary medical exams (endoscopy, examination of the liver...),
and the possibility to join self-help groups.
Case study
= > Hepatitis C, she was infected during a sclerosis surgery ten years ago
HCV testing has been practiced several weeks after the initial
diagnosis, because Ms. M. finally convinced her GP to review his
diagnosis. Because she drinks very moderately
She was addressed to the Department of hepatology of the
University Hospital to start an antiviral treatment
No consequences (except the stress of being labeled as
alcoholic)
Where does the error comes from?
Lack of knowledge? Skills?
= > It is a graduated GP
Medical misinformation?
= > All medical data were valid
Lack of experience?
= > He works as a GP for decades
Lack of knowledge about the patient?
= > He was his family doctor for years
Hypotheses
High transaminase = alcohol abuse is the most readily
available diagnosis in memory?
May be that a large number of patients has a drinking
problem? And he made the connection with Ms. M.
Maybe he thought that the problem originated from
internal cause? Possible external causes were dismissed
He did not comprehensively seek all the relevant
information such as an invasive medical intervention 10 years
ago
He focused his attention on information confirming his
initial belief (alcoholism), ignoring those that refute it (no
clinical signs) or those who were in favor of others
hypotheses (chronic fatigue)?
Heuristic
Simplified cognitive strategy, used to save time,
allowing decision-maker to make acceptable
deductions
Evidenced by Tversky & Kahneman, in the 70s
Tversky, A. & Kahneman, D. (1974). Science, 185, 1124–1130.
Kahneman, D (1972). Judgment under uncertainty: Heuristics and biases.
Cambridge: Cambridge University Press.
Three of these heuristics are particularly influential:
- Availability
- Representativeness
- Anchoring
Availability
= > Estimating the likelihood of an event according to
its availability in memory.
The first impression or the most present explanation in mind is
used in priority to explain an event or estimate a probability..
According to a survey, 50% of Americans were affraid
of dying in a terrorist attack in 2006 (Gallup poll). The
probability of such an event was estimated at one
chance over 20 million (calculated on the basis of the
available figures)
Tversky, A; Kahneman (1973). "Availability: A heuristic for judging frequency and probability". Cognitive Psychology 5
(1): 207–233
A group of 100 persons, including 30 lawyers
and 70 engineers were interviewed. These
interviews have led to the drafting of brief
fact sheets on each of these people.
One of these cards was randomly chosen:
"John is 39 year-old. He is married and has two kids. He is involved in
local politics. His favorite hobby is the collection of rare books. He loves
competition, discussion and is an articulate speaker.
Representativeness
What is the probability that John is a lawyer?
Representativeness
Good answer: 30% (30 lawyers out of 100 persons)
But participants overestimated this proportion
because we tend to:
- Neglect probability and estimates. Individual or
descriptive information is preferred, in opposition to
mathematical logic.
- Focus on the supposed similarities between the
information (speaks well...) and our initial belief (he’s a
lawyer)
Anchoring
Participants were asked to estimate the percentage of African
countries members of the United Nations
Participants’ answer : 25% on average
Participants’ answer : 45% on average
• In the other group: "Is this more or less than 65%?"
• In a group: "Is this more or less than 10%?"
Anchoring
All the reasoning process is influenced by a
single initial data, which is difficult to ignore
afterwards
Explains the influence of first impressions, or those
of early learnings on decision making or situational
analysis
Cognitive Bias
Tendency to acquire and process information by filtering it
through one's own likes, dislikes, and experiences.
Relevant
Informations
Relevant
responses
Non relevant
informations
Non relevant
responses
Selection
bias
Reasoning
bias
Heuristics
Acad. Med. 2003;78:775–780
33 biases and heuristics reportedly active in medical practice
Information-gathering problems
Unpacking – failure to elicit all relevant information
Availability – recent exposure influences diagnosis
Anchoring – holding onto a diagnosis after receiving
contradictory information
Premature closure – accepting a diagnosis before it is
fully verified
Systems contributions
Diagnosis momentum – early diagnosis by
another provider is accepted as definite
Feedback sanction – final diagnosis does not
return to initial decision-maker
Triage cueing – location cues management (seen
through the lens of the first provider)
Probability Pitfalls
Aggregate bias – aggregate data do not apply to my patients
Base-rate neglect – ignoring the true prevalence
Gender bias – gender inappropriately colors probability
Gambler’s fallacy – sequence of same diagnoses will not continue
Posterior probability – sequence of same diagnoses will continue
Hypotheses High transaminase = alcohol abuse: most readily available
diagnosis in memory? Availability
May be that a large number of patients has a drinking problem?
And he makes the connection with Ms. M. Posterior Probability
Maybe he think it the problem originate from internal cause? And
not external? Attribution error
May he did not seek comprehensively all the relevant information
(invasive medical intervention it 10 years ago) Anchoring
=>Unpacking
Maybe he focused his attention on information confirming his initial
belief (alcoholism), ignoring those that refute it (no clinical signs) or
who were in favour of an others hypothesis (chronic fatigue)?
Interpretative bias
Emotions can jeopardize the rationality of
any decision maker
three types of affects:
- transitional: surrounding factors, fatigue...
- Due to personality: trait anxiety, emotional
adaptation
- Related to clinical situations: children
involved…
Affective bias
The best scientific evidence can be overlooked because of
affects, emotions….
“Our affective reactions to patients are often
our very first reactions, occurring automatically
and subsequently guiding information
processing, judgment, and decision making…”
Zajonc, American Psychologist, 1980
Two competing reasoning modes:
The first is fast, intuitive, automatic (or even
unconscious) often involves an affective or emotional
component, and requires few resources
=> satisfying responses
The second is analytic, slow, deliberate, affect-free
and costly in resources
=> Optimal responses
Most vulnerable
to biases
Bounded rationality:
Implications for public health
Some scientific information benefit from a large
consensus in the scientific community, the public
option, the media and health professionals
Strong Epidemiological evidence
high acceptance by the public and the media
health professionals practices
Public Health
Tobacco, alcohol abuse; unhealthy diet; drink
driving ….
Making relevant decisions regarding the health of
people suppose to be able to collect relevant
information
In ambiguous, new, changing unknowns, or
emotionally situations, the collection and
interpretation of the information will be influenced
by:
Information seeking
1) The way the information is presented
2) decison-makers’ initial beliefs
You are a top decision maker at WHO and you want take
preventive actions to reduce the number of deaths and
injuries from animal bites. On which animal will you
focus your intervention?
In Australia?
A) Dogs, B) Sharks, ou C) Crocodiles
In Romania?
A) Dogs, B) Sharks, ou C) Crocodiles
Public health
In South Africa?
A) Dogs, B) Sharks, ou C) Crocodiles
Framing The entire reasoning process is governed by the
way the information is presented
Chosen by 72% of participants
Chosen by 28% of participants
We expect that an epidemic will kill 600 people.
Participants are asked to choose between two
possible actions
Option A: can save the lives of 200 people
Option B: 33% chance of saving 600 people
Framing
Same scenario, different layout
Option A: 400 people will die
Option B: 33% chance that there is no death
Chosen by 22% of participants
Chosen by 78% of participants
The expected value is the same in both cases, but the statements
mentioning death are rejected, while those mentioning saved lives are
preferred
Framing
Tendency to Loss aversion: the negative value of a loss is
considered to be superior to an equivalent gain (capacity, life,
death…)
Outcome feeling
Gain of X Positive (+)
Loss of X Extremely negative
(---)
Framing
The framing allows to influence decision using
our tendency to loss aversion
If you are favorable to the LAMBDA treatment, you
will say to the patient:
"with the LAMBDA treatment, you have 90% chance of
survival.
If you prefer an alternative solution, you will say:
"with the LAMBDA treatment, you have 10% chance of
death”
Confirmation bias
Is organic food healthier? Can video games make kids violent? Is WI - FI carcinogenic?
=> Might cast a doubt, even in the absence of
scientific evidence
When faced with a closed (yes/no) question, we
naturally tend to seek confirming information to
answer “yes”
and then, ignore the alternative hypothesis
Confirmation bias
Exemple: Does Behaviour change during the full moon?
We will immediately remember some cases when we
were stressed, anxious, insomniacs or so…during full
moon nights…
….and forget the hundreds of full moon nights when
we slept normally
EHESP 35
Change the question
Open-ended Question : what are the risk factors for cancer?
- Tabacco - Alcohol - Unhealthy diet (sugar, fat…) - Sedentary behavior - Diesel - Sun exposure - ….
Changing mind?
...not an easy task
Interpretation bias
University of Santford (USA): Two groups of students are formed according to their opinions on death penalty: A group in favor of death penalty, and a group against the death penalty
Then two studies were presented: one demonstrating
the usefulness of death penalty, and one showing DP is
counter-productive
…And they are asked to assess the
methodological quality of these two studies
“Biased assimilation and attitude polarization: The
effects of prior theories on subsequently considered
evidence”
Conclusions regarding death penaly
« efficient » « counter-productive »
In favor of death
penalty Quality ++ Quality --
Conclusions regarding death penaly
« efficient » « counter-productive »
In favor of death
penalty Quality ++ Quality --
Against death
penalty Quality ++ Quality --
The studies are considered valid if they are in
line with personal opinions, and discredited
otherwise
“Biased assimilation and attitude polarization: The
effects of prior theories on subsequently considered
evidence”
Other effects
Media covering a political campaign will be considered hostile by both sides Lobbies will select studies which suit their own interest (bicycle helmet; adverse effect of a food or a drug benefit...) and ignore others studies Scientific work will be judged from political or ideological perspectives, regardless of actual scientific value (especially in social sciences…)
Case study from France
Rapport INSERM 2005
INSERM is the French institute for Health and Medecine Relying upon data from over a thousand international scientific studies, INSERM experts advocate in a 364-pages document (available online) to best inform parents and educators on conduct disorders in children and to detect it as soon as possible (3 years old), to ensure the prevention.
Conduct disorders (esp ADHD) are a major cause of unintentional injuries in children
http://www.ipubli.inserm.fr/bitstream/handle/10608/142/expcol_2005_trouble_synthese_eng.p
df?sequence=1
Recommandations from the report
Improving information on conduct disorder in France,
through epidemiological studies among children and
adolescents in the general population, as well as in high risk
populations
• Identify risk: family factors, pre - and perinatal, genetic.
• Develop specific prevention programs, train teachers and
educators, intervene with families at risk, track as soon as
possible warning signs, i.e. at the age of 3, and record
. Perform regular, accurate and multidisciplinary assessments
for subjects with symptoms.
Assess the effects of various psychotherapies. Further
pharmacological research.
Lobbying from French physicians
The very next day, the national Union of physicians of maternal and child welfare (SNMPMI), protested. Its leaders created a petition (available online) signed by nearly 200 000 Web users, along dozens of associations , psychologists, doctors, educators…. The first signatories include child psychiatrists, clinical psychologists, doctors, lecturers, childhood professionals,...... Many belonging to a lobby which pushed health minister to censor a report from INSERM in 2004 (namely
psychoanalysts…)
Online Petition
• "detected children would be subjected to a battery of tests
developed on the basis of the theories of behavioral
Neuropsychology" “According to the criteria of the Anglo-
Saxon studies"
• Nurseries and schools transformed into “hunting places',
• The potential offender , too small to defend himself, will
be transformed into a junkie
• "the slightest gesture, the first child nonsense may be
interpreted as the expression of a pathological personality
at the mercy of a "robotic health care thinking.
“http://www.pasde0deconduite.org/appel/
Framing: changing terms
Terms used in the
Report
Terms used in the online
petition
International studies « Anglo-saxons » studies
screening Hunting down
Identify risk factors and
prevent disorders
Relentless linearity principle
and deterministic approach
Early prevention Early hunt
Pharmacology as a second
line
Drug use
Psychotherapeutic support Behavioral conditioning
Interventions towards
families at risk
stigmatization
Children: high emotional bias
political context , cultural lobbies and ideological opinions
beliefs concerning the "Anglo-Saxon" (behaviorists)
use of international recommendations to change local
practices (esp those deriving from psycho-analysis)
Pétition: framing and loss
aversion
Discredit :
INSERM
"Anglos-Saxon"
Renowned experts
international studies
scientific recommendations
Achieved Result: statut quo
context
Change in practices
=
Fascism
Early screening and detection is
advocated for aged people
Past experience improve decison-making ?
Not sure….
Question # 1: "describe the exact content of the midday meal you took 15 days ago.
Memory bias
Question # 2: "describe the situation in which you were when you heard about the 9/11 attacks “
Recall is facilitated by emotions related to the
event: their presence (yes/no); their intensity
(high/low) and their nature (positive/negative)
“Dissociation of recall and fragment completion”
Journal of Abnormal Psychology, 101, 575–580
Memory bias
In the absence of mental health disorder, recalling pleasant memories is easier than recalling unpleasant ones
increased availability producing a memory bias
Our past experiences seem to be much more positive than they actually were
Memory bias Aims:
- mechanism of protection against depressed mood
("positive illusions"),
- justify aftermath decisions made during lifetime
("choice-supportive bias")
But: Any subjective reference to a past situation may
not be relevant to assess a present or predict a future
situation
And: Tend to reinforce resistance to change, anchoring
bias, perseverance of false beliefs, interpretation bias…
Is it possible to question ourselves ?
Fundamental attribution
error
Highlighted by Jones and Harris (1967) Participants were asked to judge essays written by students and defending ideological positions imposed by the teacher. Half of the participants knew that the ideological position was imposed, the other half thinking that students expressed their opinions
« The attribution of attitudes », 1967
Even participants who knew that the position was imposed considered that it reflected the true opinion of the writer Writers’ attitudes were estimated in line with the position they were forced to defend Overestimation of internal characteristics at the expense of possible external causes
Fundamental attribution error
Self serving bias
Following a misdiagnosed cancer in 1980, Mr. Paul Mongerson decided to create a company programming diagnosis software to help physicians In 2006, he asked his GP on the reasons why he did not use diagnosis software . - "it takes time and I do less than 1% error“ - “the literature shows that the error rate is 5-10%"
- "it is because of others physicians.“
The American Journal of Medicine (2008) Vol 121 (5A), fvii
Self serving bias
Gosling (1992) asked teachers on the cause of their students’ success and failure The results showed that: - when students succeed, teachers thought it was due to the quality of their teaching
- when students fail, they thought it was related to external causes (family context)
Qui est responsable de l’échec scolaire ? (1992 ) PUF, Paris
Reinforcing false beliefs:
Self fulfilling prophecies
September June
IQ test IQ test
Normal children intellectual bloomers
“Teacher Expectation for the
Disadvantaged”
Changes
Changes over a school year
All pupils labelled as “intellectual bloomers” improved their results significantly Better relationships were observed between these students and teachers Most important roles were attributed to spurters (keep the class, manage activities, etc.) Errors committed were minimized by teachers
Rosenthal effect
Children were labeled as “intellectual bloomers"or "normal" randomly
Rosenthal and Jacobson induced, for each child, either a “positive expectation” or “no expectation” among teachers
These expectations influnced teachers’ behaviors
towards each child, and academic outcomes….
…and confirmed the (falsified) IQ test !
A student was asked to call someone on the phone for some reason. A picture of the interlocutor was shown to him/her
Normal shape Overweight
(obese)
Sterotype : « obese persons are bad-tempered »
“Why does behavioral confirmation occur? A
functional perspective on the role of the perceiver”
Self-fulfilling prophecies
Those who called someone they believed to be overweight behaved in a most unpleasant way that those who called a person they believed to have a normal shape in return the person on the phone was also unpleasant with the caller => confirming the initial belief (again)
Snyder M, Haugen JA. 1994. Why does behavioral confirmation occur? A functional perspective on the role of the
perceiver. J. Exp. Soc. Psychol. 30:218–46
Prospective study on adolescent alcohol use
“Self-Fulfilling Prophecies : The Synergistic
Accumulative Effect of Parents' Beliefs on Children's
Drinking Behavior”
Adolescent alcohol use
Estimation by parents
Time 2
Adolescent alcohol use
12 months
Time 1
• Parents’ estimates similar to actual alcohol use
• overestimation
• underestimation
Higher alcohol
use
A study conducted in Britain, showed that the number of
emergency visits following a road accident was 52% higher on
Friday 13 as compared to Friday 6th, even though there were
fewer road traffic. The authors recommend to stay at home that
day
Scanlon, Luben, Scanlon, & Singleton, 1993
Is Friday the 13th bad for your health?
This student is an “intellectual bloomer”
obese persons are bad-tempered
my child drink a lot of alcohol
Friday the 13th is an unlucky day
Decisions, behaviors
expectations
Beliefs, sterotypes
Reactions and
consequences in line
with the expectations
Confirming
initial beliefs
« Placebo effect» Persistence of obsolete practices and false beliefs
Collective decision?
Beware! : Conformism
The ASCH Effect A group of students is asked to perform a simple vision test:
which line among the comparison line is like the standard one?
The ASCH effect
The group is composed of "accomplices", except one
student, who is the real subject of the experience
When accomplices unanimously support a wrong
answer, 37% of the subjects prefer to comply with
this false response rather than opposing the group
when the social pressure disappears, the correct response
rate reach 100 %
=>Normative social influence
=> Conformism
The ASCH effect
Fear to look ridiculous or silly, to be rejected (fear of
social disapproval)
= > Normative influence (conformism)
Not sure of its own response, lack of self confidence.
= > Informational influence
Informational influence is particularly important when someone
is faced with a task he does not control.
Normative influence is important when belonging to the group
is important
Obedience to authority figures
people obey orders from authority
figures: doctors, experts, scholar, intellectual,
etc. without asking neither the origin nor the
relevance of their statements
Bounded rationality
Why all these biases exist in the first place?
Because they give satisfactory results in familiars,
commons, routine situations and require few effort
=> adapted to the every day life situations
BUT: Adverse consequences for public health in
ambiguous situations: the discrediting of valid
knowledge, the persistence of false beliefs and obsolete
practices, status quo maintenance (resistance to change)
Counter measures:
From Bounded rationality to
critical rationalism
- Raise awareness of cognitive biases by information and
education, using concrete examples and detailed
descriptions of their potential effects
- Consider alternative hypothesis in any question.
- Take the step back from the immediate problem and
analyze it.
- Do not rely (too much) on memory
- - develop decision algorithms adapted to new situations
Critical rationalism
- Focus on information from several independent sources using rigorous and transparent methods to collect and analysis data (review of literature;)
- Make distinction between knowledge from science and knowledge from ideology, politics, religion (not evidence based)..
- The level of evidence contradicting an hypothesis must be at least equal to those confirming it, in order to cast a legitimate doubt
- As a decision maker, emotion is not your friend
EHESP 81
Nullius in Verba « words are nothing »
Motto of the Royal Society (London)