1

Science and statistics in psychology Lecture 4

PredictionsPredictions

Dr Caleb Owens

Consultation: Wednesdays 9-10am

calebo@psych.usyd.edu.au

2

Conceive of experimental hypothesis

Build up knowledge about current theories and evidence

Design experiment

Test null hypothesis

Decide if findings are statistically significant

Decide if findings are practically significant

Draw conclusions

3

Conceive of experimental hypothesis

Build up knowledge about current theories and evidence

Design experiment

Test null hypothesis

Decide if findings are statistically significant

Decide if findings are practically significant

Draw conclusions

Lecture #1

Lecture #2&3

Lecture #4

Lecture #5

Lecture #6

4

• Urbain Le Verrier

– discovered Neptune in 1846

– “discovered” Vulcan in 1858

5

• Sylvia Browne, Psychic vs Sago Mine Disaster, Jan 3rd, 2006 – http://en.wikipedia.org/wiki/Sylvia_Browne#Sago_Mine_controversy

– When 12 of the thirteen miners were believed to have been found alive

– Noory: Had you been on the program today, and had they not been found, would you have felt as if, because they had heard no sounds, that this was a very gloomy moment, and they might have all died?

– Browne: No, I knew they were going to be found. Uh, you know, I hate people who say something after the fact. It's just like I knew when the Pope was dead and I said it on, thank God I was on Montel's show, and I said, according to the time, it was 9-something and whatever Rome time was, and I said he's gone, and he was.

– When it was revealed that all but one was dead– Noory: Sylvia, with the accuracy rate that you have, and it is very high,

of the few that you get wrong, do they sometimes become right, maybe later on?

– Browne: Yeah, but see I never... I didn't believe that they were alive– Noory: What's that? The miners?– Browne: Uhuh.– Noory: Oh, okay.– Browne: No, I think that they're... and see I've been on the show with

you, so I don't know but I don't think that there's any that are gonna make it.

– Noory: Well, that's not a good situation.– Browne: No.– Noory: Thirteen were in there, they said one had died, 12 were alive.– Browne: Well, there's 12 gone

6

• After four months of fruitless searching, Shawn's parents, Pam and Craig Akers, appeared on an episode of the Montel Williams show with Sylvia Browne on February 26th, 2003.

• Browne told Pam and Craig Akers their son "is no longer with us" but she had the impression his body was in a wooded area about 20 miles southwest of Richwoods. She said it would be near two large, jagged boulders that seem out of place in that area.

• Browne told the Akers that it was her vision that Shawn was taken by a "dark-skinned man, he wasn't black -- more like hispanic." She said he had long, black hair that he wore in dreadlocks and was "really tall.“

7

Looking for confirmation

• While the need for falsifiability might seem tedious, consider the alternative– i.e. What if we only ever looked for evidence which

confirmed our beliefs?• Since it is never clear what predictions do confirm a theory,

evidence which is unclear can be re-interpreted to fit the theory

• We might never realize our beliefs could never ever be disproven

• Progress toward understanding would never occur

• Confirmation bias is a well known reasoning error (http://www.skepdic.com/confirmbias.html)

8

John Edward

• John Edward: "I'm sensing the letter B. Is the letter B important to you in some way?"

• Liddy: "No."• John Edward: "Yes it is. The spirits do not lie.

Do you know a Bob, Billy, Bart, Ben, or Bonnie? Did your father know anyone by that name?"

• Liddy: (visibly shaken) Our mailman is named Bill! My father knew him!"

• John Edward: "That's it. I am sensing at one point, Billy must have delivered an important package to your father."

• Liddy: (tears flowing freely) "Yes! That did happen once!"

9Source: http://www.skeptics.org.nz/download/mediumchecklist.pdf

10

Propose a hypothesis(experimental hypothesis)

• Say what you expect to happen on the basis of a theory that makes scientific sense (i.e. derive the hypothesis)

– Say what result would support the theory

– Say what result would disprove the theory

11

Cognitive Dissonance

• We come to believe in what we suffer for• If our overt behaviour and our inner thoughts do not

correspond, we change out thoughts to better match our behaviour.

• Method (Festinger & Carlsmith, 1959): People perform a very boring experiment, and are then asked to describe the experiment to a second person, and are then asked what they actually thought of the experiment.

• Prediction: If people are paid $20 to convince another person that a boring experiment is interesting; they are less convinced themselves than if they are only paid $1– (and if the theory is wrong, then the amount of money should be

irrelevant)

12

Cognitive Dissonance

• Findings: People paid only $1 were more likely to regard the experiment as more interesting, and participate in a similar experiment, than people paid $20

• If the data had shown no difference, the theory of cognitive dissonance would have disappeared, and psychologists would have moved on to other explanations of behaviour.

Source: http://psychclassics.yorku.ca/F

estinger/index.htm

13

Falsifiability

• In science, hypotheses must be falsifiable

• If there is no way contrary evidence can disprove a theory, then the theory is not scientific

• “There are fairies at the bottom of my garden”

14

The null hypothesis• The null hypothesis is a statement of what would be the case if

nothing is happening– E.g. you wish to test the difference between two groups, you hope

there is a difference, you hypothesize there is a difference; your null hypothesis is therefore

• Group1(mean) = Group2(mean) ; (m1 = m2); (µ1=µ2)– Then if you can disprove that null hypothesis; if you can reject that

null hypothesis, then you have found something of scientific merit.– If however you do not find evidence inconsistent with the null

hypothesis you must retain it.

• Cognitive dissonance example– We hope that there is a difference between the $1 and $20 groups.

We start by stating what we would find if there is no difference, and attempt to reject that situation. Therefore our null is:

• Reactions by group receiving $1 = Reactions by group receiving $20– The null is useful because:

• It is precise, and because it is precise it can be disproven• We don’t know (or care) just how much of an effect $1/$20 will have, we

just want to find any effect!

15

The null hypothesis• Rejecting the null hypothesis – finding evidence that it

is wrong - is the aim of the scientist. – Rejecting the hypothesis of no difference means there is a

difference!

• What if the null hypothesis is retained?

• Cognitive dissonance example– No difference is found between the $1 and $20 groups– Does this mean we have found evidence that there is no

difference? (no)– We have not found evidence that there is a difference. This

does not constitute evidence that there is not a difference.

• Absence of evidence is not evidence of absence– We retain a null hypothesis we do not accept a null

hypothesis– The measuring equipment or construction of the experiment

might be flawed

16

Retaining the null hypothesis

• Dating example

• Macs and proper computers

• The attitude of scientists

17

The coin toss

• We are about to use a coin for a game and become concerned that it may be ‘loaded’; that is, we expect that if we toss a “fair” coin there is an equal (50/50) chance that it will turn up heads or tails, but the one we have might not be fair?

• We can take a ‘sample’ of the coins behaviour, by tossing it 100 times– Null hypothesis of no effect: the coin will be heads 50

times and tails 50 times– Experimental hypothesis (two tailed): the coin will be

heads more than 50 times or less than 50 times

18

Decision time• If we perform just one experiment, and toss a coin 100

times to determine if it is fair, and it turns up heads X times, would we say:– The coin is probably a fair coin– The coin is probably not a fair coin

• X Number of heads:50525560708090

100

19

The coin toss• If the coin lands Heads 52 times and Tails

48 times, and we conclude : “The coin is probably not a fair coin”.– This is an ERROR (the error of being too

sensitive; or too liberal)

• If the coin lands 80 Heads 20 Tails, and the null is not rejected (“The coin is probably a fair coin”)– This is an ERROR as well (the error of being

too conservative)• So at which point should we decide we have

enough evidence that the coin is not fair?

20

Main points• Science needs to produce theories which can be falsified

– If evidence cannot be produced which can ever modify a theory, how can the theory evolve and progress toward a closer approximation of the truth (surely no one is saying the theory is perfect to begin with?)

• Scientific predictions are either verified or falsified– Le Verrier – Neptune (verified)– Le Verrier – Vulcan (falsified)– Bertrand Russell – teapot between Earth and Mars (unfalsifiable)

• The “experimental hypothesis” is what a researcher expects to happen or hopes will happen, or a theory predicts will happen

• The “null hypothesis” is a statement of no effect, which needs to be disproved if evidence of a difference is to be found

• When interpreting outcomes, being too liberal or too conservative are both errors– We need an understanding of statistics to find the right point