RISK COMMUNICATION

John Monahan, Ph.D.

Communication of the risk of terrorism

Communication of medical risks

Communication of violence risk in forensic psychiatry and psychology

Communication of the Risk of Terrorism

Problems with the Homeland Security Advisory System

• False positives: “Crying wolf”

Problems with the Homeland Security Advisory System

• False positives: “Crying wolf”• Vagueness

– No substantive content: risk of what?– No spatial detail: where is the risk taking place? – No temporal detail: when is the risk taking

place?• Politicized rather than transparent• Not prescriptive: what is a citizen supposed to do

to reduce his or her risk?

Effective April 2011

Lessons from the Communication of Terrorism Risk for Forensic Psychiatry and Psychology

• False positives are a pervasive problem: “accountability ping-pong” (Tetlock and Mellers, in press)

• Be as specific as possible in communicating risk (e.g., type of violence, victim, time-frame)

• Be as prescriptive as possible: risk management.

Communication of Medical Risks

Better Doctors, Better Patients, Better Decisions: Envisioning Health Care 2020

Gigerenzer & Gray, Eds (2011)

• Relative risk reduction, e.g.: “Lipitor reduces the chance of stroke by 48%”

• Absolute risk reduction, e.g.: “Lipitor reduces the chance of stroke from 28 in 1,000 to 15 in 1,000 (i.e., by 13 in 1,000 people, or 1.3%)”

Misleading Communication of RiskGigerenzer, Wegwarth, & Feufel, 341 BMJ 791 (2010)

Conveying relative risks [e.g., a 50% reduction] without baseline risk [e.g., from 10 in 1,000 to 5 in 1,000] is the first “sin” against transparent reporting.

The second is mismatched framing—reporting benefits, such as relative risk reductions, in big numbers and harms, such as absolute risk increases, in small numbers.

If we use the example of a treatment that reduces the probability of getting disease A from 10 to five in 1,000, whereas it increases the risk of disease B from five to 10 in 1,000, authors who use mismatched framing would report the benefit as a 50% risk reduction and the harm as an increase of five in 1000; that is, 0.5%.

Medical journals permit mismatched framing. One in three articles in the BMJ, JAMA, and Lancet [during 2009]... used mismatched framing when both benefits and harms were reported. Moreover, absolute risks [e.g., 5 in 1,000]… were more often reported for harms than for benefits.

Steps can be taken to improve the transparency of risk communication.

Firstly, editors should enforce transparent reporting in journal abstracts: no mismatched framing, no relative risks without baseline risks, and always give… absolute numbers.

Secondly, institutions that subscribe to medical journals could give journal publishers two years to implement the first measure and, if publishers do not comply, cancel their subscriptions.

Cochrane Review: Using Alternative Statistical Formats for Presenting Risks and Risk Reductions (2011)

35 studies; 83 comparisons. Participants: Physicians and patients

•Participants (health professionals and consumers) understood… frequencies better than probabilities. •[However,] they perceived interventions to be more effective, and were more persuaded to prescribe or accept an intervention, when exposed to relative risk reduction [e.g., a 50% reduction] compared to absolute risk reduction [e.g., from 10 in 1,000 to 5 in 1,000]. •Overall there were no differences between health professionals and consumers… This would be of concern because health professionals play a key role in medical decision making. It also suggests that the formal education and training of health professionals apparently has no effect on their handling of statistical information.

Lessons from the Communication of Medical Risks for Forensic Psychiatry and Psychology

• Use frequencies rather than probabilities to communicate risk

• Express the likely effects of treatment using absolute risks rather than relative risks

• Increase the “numeracy”—the ability to understand numbers—of forensic psychiatrists and psychologists, and of judges.

Communication of Violence Risk in Forensic Psychiatry and Psychology

COVR Classification Tree

Total Sample19% Violent

Prior ArrestsNone

9% Violent

Prior ArrestsNon-Violent20% Violent

Prior ArrestsViolent

36% Violent

ImpulsivenessLow

7% Violent

ImpulsivenessHigh

21% Violent

Father Used Drugs

No17% Violent

Father Used Drugs

Yes39% Violent

Violent Fantasies

No27% Violent

Violent Fantasies

Yes53% Violent

Risk Classes

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5

Classes

P

Probability of Violence 95% Confidence Interval

Psychiatry Research (2011), 188, 161-165.

COVR Default Categories

● Category 1: Very low risk [1%/1 of 100]

● Category 2: Low risk [8%/8 of 100]

● Category 3: Average risk [26%/26 of 100]

● Category 4: High risk [56%/56 of 100]

● Category 5: Very high risk [76%/76 of 100]

Illustrative User-Defined Categories

● Category 1: Discharge without further evaluation.● Category 2: Discharge if a routine evaluation is

negative for violence risk● Category 3: Do not discharge now; re-evaluate in 72

hours● Category 4: Do not discharge without a full

evaluation by Dr. Smith● Category 5: Consider warning potential victim

and/or petitioning for outpatient commitment.

Violence risk assessment and risk communication

Slovic, Monahan, & MacGregor, Law & Human Behavior (2000)

Case Vignette

A patient – Mr. James Jones – has been evaluated for discharge from an acute civil mental health facility where he has been treated for the past several weeks. A psychologist whose professional opinion you respect has done a state-of-the-art assessment of Mr. Jones. Among the conclusions reached in the psychologist’s assessment is the following:

RANDOMLY, EITHER

Patients similar to Mr. Jones are estimated to have a 20% probability of committing an act of violence to others during the first several months after discharge.

OR

Of every 100 patients similar to Mr. Jones, 20 are estimated to commit an act of violence to others during the first several months after discharge.

QUESTION If you were working as a supervisor at this mental health facility and received the psychologist’s report, would you recommend that Mr. Jones be discharged from the hospital at the present time?

Results

Risk Communication

20% probability

20 of every 100

Do Not Discharge (%)

21

41

Why?: “Imaging the Numerator”

Representations of risk in the form of probabilities (10% of 100 patients) led to relatively benign images of one person, whereas the “equivalent” frequentistic representations (10 out of 100 patients) created frightening images of violent patients (e.g., “Some guy going crazy and killing someone.”)

Subsequent Research• Forensic evaluators communicate risk primarily in

categories (high, medium, low), and use categories most often if risk is perceived as high (Grann & Pallvik, 2002)

• Forensic evaluators prefer categories that are prescriptive (Heilbrun et al, 2004)

• There is much disagreement among forensic evaluators as to the quantitative content of the categories used (high, medium, low) (Hilton et al, 2008)

“Numeracy”

Peters, Västfjäll, Slovic, Mertz, Mazzocco & Dickert, Numeracy and Decision Making, Psychological Science

Numeracy Scale—11 items Lipkus et al (2001)

• The chance of getting a viral infection is .0005. Out of 10,000 people, about how many of them are expected to get infected?

• In the Acme Publishing Sweepstakes, the chance of winning a car is 1 in 1,000. What percent of tickets in the Acme Publishing Sweepstakes win a car?

• Imagine that we roll a fair, six-sided die 1,000 times. Out of 1,000 rolls, how many times do you think the die would come up even (2, 4, or 6)?

Ss given a case vignette and a risk estimate that was randomly EITHER“Of every 100 patients similar to Mr. Jones, 10% are estimated to commit an act of violence to others during the first several months after discharge.” OR“Of every 100 patients similar to Mr. Jones, 10 are estimated to commit an act of violence to others during the first several months after discharge.”QUESTION: What is the risk that Mr. Jones will harm someone?(Scale: 1=low risk; 6=high risk)

Fig. 2. Numeracy and percentage (10% of 100) versus frequentistic (10 out of 100) representations of risk.

Peters E et al. Psychological Science 2006;17:407-413

Copyright © by Association for Psychological Science

Why?: Less Numerate People Can’t “Image the Numerator” in the Probability Format

Less numerate people may be influenced greatly by the affective imagery elicited by the frequency format, and do not have access to this affective imagery when given the probability format.

Communicating the Risk of Violence by People with Mental Illness

“The Treatment Advocacy Center estimates people with untreated schizophrenia and bipolar disorder commit about 1,000 of the 20,000 homicides per year nationwide” (NJ Star-Ledger, July 26, 2011).

OR

The annual probability of an American (N= 312,000,000) being killed by a person with untreated schizophrenia or bipolar disorder is .000003.

Legal Uses of Risk Communication Research

Civil Commitment Standard in Virginia

Pre-2008: “The person presents an imminent danger to himself or others as a result ofmental illness.”

2008: “There is a substantial likelihood that, as a result of mental illness, the person will, in the near future, cause serious physical harm to himself or others, as evidenced by recent behavior causing, attempting, or threatening harm.”

Civil Commitment Standard in Virginia

Pre-2008: “The person presents an imminent danger to himself or others as a result ofmental illness.”

2008: “There is a substantial likelihood that, as a result of mental illness, the person will, in the near future, cause serious physical harm to himself or others, as evidenced by recent behavior causing, attempting, or threatening harm.”

Risk Classes

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5

Classes

P

Probability of Violence 95% Confidence Interval

Question to Judges

What is the lowest likelihood of violence to others that you would accept… for authorizing short-term civil commitment?

Risk Classes

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5

Classes

P

Probability of Violence 95% Confidence Interval

Official Training Material, VA Department of Behavioral Health

Cohen, Bonnie, & Monahan, 2009

“A ‘one-in-four’ estimated risk of serious harm in the near future is sufficient [for civil commitment], A ‘substantial likelihood’ is not meant to mean ‘more likely than not’ (51%).”

Conclusions• Violence risk communication is a vibrant area of research

in forensic psychiatry and psychology• Forensic evaluators prefer to use categories, but rarely

define those categories• Probabilities are very easily misunderstood and

mischaracterized—they should be avoided• The lack of “numeracy” is a serious, underresearched

problem• The law can benefit from more transparent violence risk

communication.

TACK!

jmonahan@virginia.edu