Expecting the Unexpected

1

John Milanesemilanj3@rpi.edu

Regulatory Policy? Rationality?

We generate new technologies at a dizzying pace. Are they all for the best?

Solution: regulatory policy. Demand thorough testing Problems: expensive; miss out on benefits.

2

“Inventions are good”

Only recently has “the natural” or wilderness been perceived as beautiful or good. Pre 1900’s wilderness associated with evil

spirits and the devil.

1900’s – technology as something to fear. A-bomb obvious example. Better example is DDT

3

DDT as a “wonder chemical”

DDT – organochlorine pesticide, also endocrine disruptor. Created in 1870s, not until resynthesized in

1940 by Paul Muller; was it used (lice/mosquitoes) to stop spread of typhus and malaria.

1948 Muller wins Nobel Prize for Physiology and Medicine.

Everyone thought DDT was safe (See Oreskes). DDT was in perfect timing for military needs.

Military credited it with saving many lives. Less immediately toxic than arsenic-based

pesticides. Spraying from airplanes less expensive than

older means of eradicating disease (draining swamps, etc).

State and local governments used it, even household use too.

4

Carson’s Silent Spring

Rachel Carson’s Silent Spring; enormously influential – a springboard for the environmental movement in the U.S. Uncertainty about science: hidden dangers of

tech, caused/discovered/cured all by scientific research.

Imagery from the mid-20th century speaks volumes about Post WW2 naïve optimism about science…

5

6

7

8

9

10

11

12

13

How to approach regulation

Typical approach in U.S.: economic cost-benefit analysis. Monetize costs and benefits that are relevant Multiply each costs/benefits by their

probabilities to get expected value for each, then do addition.

Works great when you have reliable data about benefits/dangers and probabilities…. But we don’t always have these.

New approach is emerging to supplement:

14

“Precautionary Principles”

Precautionary principles advise caution when we are faced with uncertain but significant dangers of new technology or activity. Better safe than sorry; look before you leap; err on

side of caution.

“Uncertainty” vs. “Risk” – risk means we know probabilities.

“Significant” vs. ?

Precautionary principles have recently emerged as common in regulatory policy in the EU (not in U.S. as much). EU ban on GMO fruits and vegetables.

Here are some example principles:15

1992 Rio Declaration of the UN Conference on Environment and

Development (Principle 15)

“In order to protect the environment, the precautionary approach shall be widely applied by States according to their capabilities. Where there are threats of serious or irreversible damage, lack of full scientific certainty shall not be used as a reason for postponing cost-effective measures to prevent environmental degradation.”

16

1998 Wingspread Consensus Statement:

“When an activity raises threats of harm to human health or the environment, precautionary measures should be taken even if some cause and effect relationships are not fully established scientifically. In this context the proponent of the activity, rather than the public, should bear the burden of proof.”

17

charges against PP:

PP are unscientific

PP are poorly defined – use vague terms

PP are absolutist

self-defeating

18

EU’s ruling on antibiotics in agro

EU banned the use of the antibiotic virginiamycin in animal feed. The ban was challenged, and the court ruled

against the challenge, arguing that the antibiotic could be banned “ ‘even when no risk assessment had been conducted, when there was limited or no evidence of such bacterial resistance arising, [and] when there was no present need for the use of such antibiotics in human medicine...’ The Court insisted that conclusive evidence of the reality of the risk should not be required”

Problems?19

Worries from the EU ruling…

(1) PP is insensitive to magnitude of the probability of the danger.

(2) Ignores benefits. No weighing of the magnitude of the benefits of this use against the severity of the possible danger.

20

1st problems with the “absolutist objection”

PP doesn’t trigger under any perceived danger, but plausible & significant ones.

No authoritative formulation or interpretation of the principle that supports such an extreme requirement.

21

Big background worry about PP

PP plays on people’s fears / values / irrationalities Too political Need scientific / rigorous cost-benefit analysis

to avoid irrationality. Ex: Kahneman & Tversky (1981) study about framing heuristics…

22

Imagine that the U.S. is preparing for the outbreak of an unusual Asian disease, which is expected to kill 600 people. Two alternative programs to combat the disease have been proposed. Assume that the exact scientific estimate of the consequences of the programs are as follows:

If Program A is adopted, 200 people will be saved.

If Program B is adopted, there is 1/3 probability that 600 people will be saved, and 2/3 probability that no people will be saved.

Which of the two programs would you favor?

72% for A ; 28 % for B23

Probability Neglect

When people discuss a low-probability risk, their expressed concern rises even if the discussion consists mostly of apparently trustworthy assurances that the likelihood of harm really is tiny.

People are willing to pay for more flight insurance to cover losses resulting from “terrorism” than they are willing to pay for flight insurance to cover losses resulting from “all causes.”

24

Different Strategy for PP

PP supplements CBA: 2 ways this might go:

When we are uncertain about the probability estimates for events that are on our pro and con list.

New approach: When we are uncertain about how exhaustive our list of relevant pros and cons is. (we don’t consider everything). this strategy is just being taken up now (See

Grant and Quiggin). 25

The problem…

How can we know our level of unawareness? Know about “unknown unknowns?”

Idea: identify causes of our failing to consider relevant potential outcomes. My conjecture: causes at a social level, and

psychological level. Identifying our cognitive blind spots:

psychological heuristics. (example: Selmer’s lecture about moral robots).

26

1. Availability Heuristic

Events that are already on our minds, or easily retrieved from memory, or easily or vividly imaginable, are judged to be more probable.

Hyposcenia-B study is a good example. More vivid symptoms of an imaginary disease judged to be more likely (see Sherman).

When considering the likelihood of X, we often judge this on the basis of how quickly or vividly X comes to mind. People think tornadoes cause more harm to people than

asthma attacks (off by a factor or 20) natural disaster insurance depends on recent experience.

27

Slovic’s affect heuristic

judgments we form about the benefits or risks of a technology are shaped by how much we like the technology rather than what we regard to be true of the technology.

“Persuasive messages” study: we associate “beneficial” with “overall good” and overall good with “low risk.” (see Finucane).

28

Unconsidered Consequences Precautionary Principle

(UCPP) When we have good reason for thinking that

social or psychological causes of unconsidered consequences, or psychological distortions of risk, are significantly influencing our judgments about the range of potential consequences associated with a new technology or activity, additional resources or regulatory measures should be devoted for the purpose of uncovering and appropriately weighing unconsidered dangers.

Let’s look at some examples for implementation…

29

“Sketch” of how an application of UCPP might

go Take the availability heuristic. Let’s think about

3d printing.

Suppose a panel of 3DP experts arrive at these dangers: increased ease of copyright infringement. disenfranchising domestic manufacturing. increased production and exposure to potentially

harmful chemicals. Applications for terrorist uses.

Q: how much of our thinking is dictated by concerns we are already worried about?

Q: if we were able to ignore current problems, would we generate a different list?

30

3D printing and unconsidered dangers

Every pro/con list should have space at the bottom for a final row: “something not considered.”

If we have reason to believe that we are overestimating the cognitively available dangers, we have reason to nudge these down, thereby increasing the probability of the “something else” hypothesis.

Those 4 problem listed are potential concerns. Maybe outweighed by benefits, of course. 31

Media showcases the miraculous benefits of computer-brain interface technology. helped neuroscientist Scott Mackler live a fuller

life while suffering from the effects of ALS. enabled stroke victim Cathy Hutchinson, who

retained full intellectual capacity but was completely paralyzed, to communicate and operate a mechanical arm.

Point: If perception is dominated by miracle stories, then the affect heuristic research suggests that we will not appropriately attend to potential risks. DDT may be an historical example of the affect

heuristic at work…. 32

Social Unawareness

Medical research in the 1960s: too homogenous.

Homogenous inquiry: different thinking brings more out for consideration. Most dramatically: different kinds of thinking

beings. “Mycin,” early expert system for assisting

diagnosis of infections.

33

References

34

[1] Oreskes, N; Conway, E. M. Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming. Bloomsbury Publishing. 2010.[2] (Rio Statement) : http://www.un.org/documents/ga/conf151/aconf15126-1annex1.htm [3] (Wingspread statement)http://www.sehn.org/wing.html[4](virginiamyacin) Case T-13/99, Pfizer Animal Health S.A. vs. Council, 2002 WL 31337, 2002 ECJ CELEX LEXIS 3613 (European Court of First Instance, Sept. 11, 2002).[5] Grant, Simon; Quiggin, John. “Bounded Awareness, Heuristics and the Precau- tionary Principle”. Journal of Economic Behavior and Organization, 93 (2013):17– 31.[6] Sherman, Steven J; Cialdini, Robert B; Schwartzman, Donna F; Reynolds, Kim D. “Imagining Can Heighten or Lower the Perceived Likelihood of Contracting a Disease: The Mediating Effect of Ease of Imagery”. Personality and Social Psychology Bulletin, 11 (1985):118:127.[7]Finucane, Melissa L; Alhakami, Ali; Slovic, Paul; Johnson, Stephen M. “The Affect Heuristic in Judgments of Risks and Benefits”. Journal of Behavioral Decision Making, 13(2000):1–17.