We’re all going to die!We’re all going to die!

Trevor ButterworthTrevor Butterworth

Statistical Assessment Service (STATS)Statistical Assessment Service (STATS)George Mason University, VirginiaGeorge Mason University, Virginia

www.stats.orgwww.stats.org

““A journalist is a lookout on the A journalist is a lookout on the

bridge of the bridge of the ship of state” ship of state”

— — Joseph PulitzerJoseph Pulitzer

Sounding the alarm at Sounding the alarm at StarbucksStarbucks

An everyday story of risk…An everyday story of risk…

And the serious health And the serious health problems are:problems are:

“ “Pesticides in food have been Pesticides in food have been linked to ADHD and other linked to ADHD and other childhood health problems. And childhood health problems. And it's not just on the food, either. it's not just on the food, either. Scientists have found that Scientists have found that systemic pesticides are taken up systemic pesticides are taken up inside of the plants that we eat. inside of the plants that we eat. Gross!”Gross!”

It’s a bug’s life…It’s a bug’s life…

“ “ Many bug killers work Many bug killers work because they screw with a because they screw with a bug's nerves. But those same bug's nerves. But those same chemical controls could be chemical controls could be messing with children's messing with children's health, too.” health, too.”

Killer plastic…Killer plastic…

“ “Sure, we've known for a few Sure, we've known for a few years now that Nos. 3 and 6 years now that Nos. 3 and 6 plastic and BPA are particularly plastic and BPA are particularly bad for you. But a recent study bad for you. But a recent study published in the journal of published in the journal of Environmental Health Environmental Health Perspectives found that all Perspectives found that all plastic is bad for you.”plastic is bad for you.”

And so on…And so on…

Toxic toysToxic toys Unsafe soapUnsafe soap ““Harsh” Harsh”

disinfectantsdisinfectants Scented candlesScented candles Air freshenersAir fresheners

But…But…

374374 22 1,282,6351,282,635

Top substances in pediatric exposures (age Top substances in pediatric exposures (age 0-5)0-5)

   1 Cosmetics / personal care products 1 Cosmetics / personal care products

174,073 174,073 2 Analgesics 2 Analgesics 130,213 130,213 3 Cleaning substances (household) 3 Cleaning substances (household) 125,394 125,394 4 Foreign bodies / toys/ misc. 4 Foreign bodies / toys/ misc. 93,574 93,574 5 Topical preparations 5 Topical preparations 91,127 91,127

InnumerateInnumerate

“ “Generally I will try not to let two Generally I will try not to let two paragraphs with numbers bump paragraphs with numbers bump against each other — ever. against each other — ever. Because I think numbers are Because I think numbers are absolutely deadly.”absolutely deadly.”

— — Best Newspaper Writing, Best Newspaper Writing, 19821982

“ “Most journalists seem unable to judge Most journalists seem unable to judge whether numbers are really meaningful whether numbers are really meaningful or accurate. Consequently, they either or accurate. Consequently, they either trust all figures or they trust none; and trust all figures or they trust none; and they tend to focus exclusively on a they tend to focus exclusively on a report writer's conclusions, while report writer's conclusions, while ignoring specific numbers and data ignoring specific numbers and data collection techniques.”collection techniques.”

— — Tankard and Ryan Tankard and Ryan

A history of inaccuracyA history of inaccuracy

8.8% science stories judged error 8.8% science stories judged error free by scientistsfree by scientists

40-59% for other subjects40-59% for other subjects 42 “types” of error42 “types” of error Lack of methodological detail worst Lack of methodological detail worst

errorerror Lack of context – earlier research on Lack of context – earlier research on

issue ignoredissue ignored

—— Tankard and Ryan, 1974Tankard and Ryan, 1974

“…“…a lot of scientists distrust the a lot of scientists distrust the press. They have good cause for this. press. They have good cause for this. Some of the things the press has Some of the things the press has done to science are horrible to done to science are horrible to contemplate.” contemplate.”

— — Ralph Coghlan, St. Louis Post Ralph Coghlan, St. Louis Post Dispatch, Dispatch, writing in writing in The Scientific The Scientific Monthly Monthly on “The on “The need for science need for science writing in the press.” 1946writing in the press.” 1946

Have things improved or Have things improved or worsened?worsened?

How SOT Members Rate the Accuracy of the Media's Reporting on Certain Scientific Issues and Principles

Percentage Rating Poorly/Well

8%3% 3% 3% 2% 2% 2%

90%95% 96% 96% 97% 97% 96%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

In providingdiverse views &

balance

Explainingrisk/benefit

tradeoffs

Explaining that"dose makesthe poison"

Distinguishingcorrelation from

causation

Distinguishingbetween

absolute &relative risk

Distinguishinggood & bad

studies

Explaining oddsratios

POORLY WELLSTATS at George Mason University

14%

6%

74%

12%

9%

73%

10%

13%

68%

28%

36%

27%

48%

21%

18%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Individualstudies relative

to overallevidence

Individualscientists

relative toscientific

community

Studies byenvironmental

groupscientists

Studies bygovernment

scientists

Studies byprivate sector

scientists

How SOT Members Rate the Weight the Media Gives to Various Studies

TOO LITTLE RIGHT TOO MUCHSTATS at George Mason University

Consequences?Consequences?

So what happens when you So what happens when you take an innumerate take an innumerate watchdog that’s always on watchdog that’s always on the lookout for new threats, the lookout for new threats, and subject it to a relentless and subject it to a relentless assault of novel, alarming assault of novel, alarming studies?studies?

Always the deadliest Always the deadliest context?context?

Two years later…Two years later… “ “For farmed salmon, the cardiovascular For farmed salmon, the cardiovascular

benefits are greater than the cancer benefits are greater than the cancer risks by a factor of at least 300:1. With risks by a factor of at least 300:1. With the exception of some locally caught the exception of some locally caught sport fish from contaminated inland sport fish from contaminated inland waters, the levels of PCBs and dioxins waters, the levels of PCBs and dioxins in fish should not influence decisions in fish should not influence decisions about fish intake.”about fish intake.”

— — Dariush Mozaffarian, Dept of Dariush Mozaffarian, Dept of Epidemiology, Epidemiology,

Harvard School of Public HealthHarvard School of Public Health

“ “ Unfortunately, the media and Unfortunately, the media and others may have contributed to this others may have contributed to this confusion by greatly exaggerating confusion by greatly exaggerating the unsubstantiated claim of a the unsubstantiated claim of a health risk from fish.”health risk from fish.”

—— Erich Rimm, Dept of Erich Rimm, Dept of Epidemiology, Epidemiology,

Nutrition, Harvard School of Public Nutrition, Harvard School of Public HealthHealth

Barking madBarking mad

“ “ Journalists are, in the very nature Journalists are, in the very nature of their calling, alarmists; and this of their calling, alarmists; and this is their way if giving interest to is their way if giving interest to what they write. Herein they are what they write. Herein they are like little dogs, if anything stirs, like little dogs, if anything stirs, they immediately set up a shrill they immediately set up a shrill bark. ”bark. ”

— — Arthur Arthur SchopenhauerSchopenhauer

Is the public credulous?Is the public credulous?

29% believe news organizations get 29% believe news organizations get their facts straighttheir facts straight

63% say stories are inaccurate (up 63% say stories are inaccurate (up from 34% in 1985)from 34% in 1985)

70% believe news orgs try to cover 70% believe news orgs try to cover up mistakesup mistakes

- Pew Research Center for - Pew Research Center for the the

People and the , 2009People and the , 2009

Is Is schmexpertise schmexpertise driving driving expertise out of expertise out of

circulation?circulation?

Who do Who do youyou trust? trust?

Gisele?Gisele?

““Are you going to give chemical food Are you going to give chemical food to to

your child, when they are so little?”your child, when they are so little?”

““I cannot put this poison on my skin. I cannot put this poison on my skin.

I do not use anything synthetic.”I do not use anything synthetic.”

Bruce?Bruce?

““We have estimated that 99.9% of We have estimated that 99.9% of the chemicals humans are exposed the chemicals humans are exposed to are natural, and we find that they to are natural, and we find that they are as frequently positive in rodent are as frequently positive in rodent cancer tests as synthetic chemicals… cancer tests as synthetic chemicals… Many ordinary foods would not pass Many ordinary foods would not pass the health criteria that have been the health criteria that have been used to regulate human exposures to used to regulate human exposures to synthetic chemicals based on results synthetic chemicals based on results of animal cancer tests.”of animal cancer tests.”

Not a fair fight…Not a fair fight… Bruce Ames may be one of the most cited Bruce Ames may be one of the most cited

scientists in the world, but his views on scientists in the world, but his views on chemophobiachemophobia have rarely made the have rarely made the media – perhaps 10-15 citations in past media – perhaps 10-15 citations in past decade.decade.

Gisele Bundchen has approximately 300 Gisele Bundchen has approximately 300 news citations on chemical issues in past news citations on chemical issues in past two years (not all favorable, mind you). two years (not all favorable, mind you).

5,790,000 web citations according to 5,790,000 web citations according to GoogleGoogle

Visible risks, invisible risk Visible risks, invisible risk assessmentassessment

49:149:1

Error ratesError rates

What is the likelihood that one new What is the likelihood that one new study with a novel or alarming study with a novel or alarming conclusion can overturn the weight conclusion can overturn the weight of evidence?of evidence?

For scientists, the likelihood is For scientists, the likelihood is directly in relation to study method directly in relation to study method and replication, which is why they and replication, which is why they are, generally, suspicious of noveltyare, generally, suspicious of novelty

Journalism’s Journalism’s precautionary principleprecautionary principle

For journalists, the risk of For journalists, the risk of being being wrongwrong about about something going something going wrongwrong is is vastly greater than the vastly greater than the risk of being risk of being wrongwrong about about something that is something that is rightright..

Which is a truer picture Which is a truer picture of reality?of reality?

ConclusionsConclusions

Journalism is intellectually and Journalism is intellectually and emotionally invested in reporting emotionally invested in reporting risk.risk.

Journalism about risk gives the Journalism about risk gives the audience an emotional experience, audience an emotional experience, generally using selective data for generally using selective data for intellectual support.intellectual support.

Risk assessment gives people Risk assessment gives people an intellectual experience but an intellectual experience but little emotional connection to little emotional connection to the weight of evidence.the weight of evidence.

The emotional experience is The emotional experience is much more likely to move much more likely to move people towards action unless people towards action unless that action is so engrained in that action is so engrained in normal life the risk becomes normal life the risk becomes acceptable.acceptable.

It all comes down to It all comes down to storytellingstorytelling

“ “If you can succeed in telling a If you can succeed in telling a story that makes your audience story that makes your audience feel as if this heart is beating for feel as if this heart is beating for them… they will buy your product them… they will buy your product in order to own the story.”in order to own the story.”

—— Lynda Resnick (Fiji water etc) to Lynda Resnick (Fiji water etc) to

Peter Guber, in Peter Guber, in Tell to WinTell to Win..

A call to action?A call to action?

“ “The life of academic quietism is The life of academic quietism is over. The man of learning, over. The man of learning, however well equipped he may however well equipped he may be, must learn to become a man be, must learn to become a man of action, a politician, a man of of action, a politician, a man of the people, speaking for people, the people, speaking for people, leading people…” leading people…”

— — Joseph Brandt, Joseph Brandt, HarpersHarpers, , March 1946March 1946