Last Time Binomial Distribution –Excel Computation Political Polls –Strength of evidence...

Last Time

• Binomial Distribution– Excel Computation

• Political Polls– Strength of evidence

• Hypothesis Testing– Yes – No Questions

Administrative Matter

• Midterm I, coming Tuesday, Feb. 24

(will say more later)

Reading In Textbook

Approximate Reading for Today’s Material:

Pages 488-491, 317-318

Approximate Reading for Next Class:

Pages 261-262, 9-14, 270-276, 30-34

Haircut?

Why?

Website:

http://www.time.com/time/health/article/0,8599,1733719,00.html

Haircut?

Hypothesis Testing

Example: Suppose surgery cures (a certain

type of) cancer 60% of time

Q: is eating apricot pits a more effective cure?

Hypothesis Testing

E.g. Pits vs. Surgery

Let p be “cure rate” of pits

(i.e. proportion of people cured)

Hypothesis Testing



(H0 & H1? New method needs to

“prove it’s worth”

so put burden of proof on it)

Hypothesis Testing



H0: p < 0.6 vs. H1: p ≥ 0.6

Recall cure rate of surgery

(competing treatment)

Hypothesis Testing



H0: p < 0.6 vs. H1: p ≥ 0.6

(OK to be sure of “at least as good”,

since pits nicer than surgery)

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6

Now suppose observe X = 11, out of 15 were

cured by pits

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


cured by pits

I.e.: “best guess about p” is:

733.0ˆ1511 n

Xp

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


cured by pits


6.0733.0ˆ1511 n

Xp

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


cured by pits


6.0733.0ˆ1511 n

Xp

Looks Better?

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


cured by pits


But is it conclusive?

6.0733.0ˆ1511 n

Xp

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


cured by pits


But is it conclusive?

6.0733.0ˆ1511 n

Xp

Or just due to sampling variation?

Hypothesis Testing

Approach: Define

“p-value” =

Hypothesis Testing

Approach: Define

“p-value” = “observed significance level”

Hypothesis Testing

Approach: Define


= “significance probability”

Hypothesis Testing

Approach: Define


= “significance probability”

= P[seeing something as

unusual as 11 | H0 is

true]

Hypothesis Testing




true]

Hypothesis Testing




true]

Note: for

Hypothesis Testing




true]

Note: for could use “X/n = 0.733”

Hypothesis Testing




true]

Note: for could use “X/n = 0.733”,

but this depends too much on n

Hypothesis Testing



unusual as 11 | H0 is true]

Note: for could use “X/n = 0.733”,

but this depends too much on n

(look at example illustrating this)

Class Example 4

For X ~ Bi(n,0.6):n P(X/n = 0.6) P(X/n >= 0.6)

5 0.346 0.31710 0.251 0.36730 0.147 0.422

100 0.081 0.457300 0.047 0.475

1000 0.026 0.4863000 0.015 0.492

10000 0.008 0.496

Class Example 4

For X ~ Bi(n,0.6):

Computed using

Excel:http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg4.xls

n P(X/n = 0.6) P(X/n >= 0.6)

5 0.346 0.31710 0.251 0.36730 0.147 0.422

100 0.081 0.457300 0.047 0.475

1000 0.026 0.4863000 0.015 0.492

10000 0.008 0.496

http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg4.xls

Class Example 4

For X ~ Bi(n,0.6):

Note: these go to 0,

even at “most likely

value”

n P(X/n = 0.6) P(X/n >= 0.6)

5 0.346 0.31710 0.251 0.36730 0.147 0.422

100 0.081 0.457300 0.047 0.475

1000 0.026 0.4863000 0.015 0.492

10000 0.008 0.496

Class Example 4

For X ~ Bi(n,0.6):

Note: these go to 0,

even at “most likely

value”

So “small” is

not conclusive

n P(X/n = 0.6) P(X/n >= 0.6)

5 0.346 0.31710 0.251 0.36730 0.147 0.422

100 0.081 0.457300 0.047 0.475

1000 0.026 0.4863000 0.015 0.492

10000 0.008 0.496

Class Example 4

For X ~ Bi(n,0.6):

But for these

“small”

is conclusive

n P(X/n = 0.6) P(X/n >= 0.6)

5 0.346 0.31710 0.251 0.36730 0.147 0.422

100 0.081 0.457300 0.047 0.475

1000 0.026 0.4863000 0.015 0.492

10000 0.008 0.496

Class Example 4

For X ~ Bi(n,0.6):

But for these

“small”

is conclusive

(so use range,

not value)

n P(X/n = 0.6) P(X/n >= 0.6)

5 0.346 0.31710 0.251 0.36730 0.147 0.422

100 0.081 0.457300 0.047 0.475

1000 0.026 0.4863000 0.015 0.492

10000 0.008 0.496

Hypothesis Testing


= P[seeing 11 or more

unusual | H0 is true]

Hypothesis Testing


= P[seeing 11 or more

unusual | H0 is true]

So use:

= P[X ≥ 11 | H0 is true]

Hypothesis Testing

“p-value” = P[X ≥ 11 | H0 is true]

Hypothesis Testing


What to use here?

Hypothesis Testing


What to use here?

Recall: H0: p < 0.6

Hypothesis Testing


What to use here?

Recall: H0: p < 0.6

How does P[X ≥ 11 | p] depend on p?

Hypothesis Testing


Hypothesis Testing


Calculated in Class EG 4b:


p P(X >= 11|p)

0.2 0.000

0.3 0.001

0.4 0.009

0.5 0.059

0.6 0.217

0.7 0.515

0.8 0.836


Hypothesis Testing


Bigger assumed p

goes with

Bigger Probability

i.e. less conclusive

p P(X >= 11|p)

0.2 0.000

0.3 0.001

0.4 0.009

0.5 0.059

0.6 0.217

0.7 0.515

0.8 0.836

Hypothesis Testing

“p-value” = P[X ≥ 11 | H0 is true] =

= P[X ≥ 11 | p < 0.6]

Hypothesis Testing


= P[X ≥ 11 | p < 0.6]

So, to be “sure” of conclusion, use largest

available value of P[X ≥ 11 | p]

Hypothesis Testing


= P[X ≥ 11 | p < 0.6]



Thus, define:

“p-value” = P[X ≥ 11 | p = 0.6]

Hypothesis Testing


= P[X ≥ 11 | p < 0.6]



Thus, define:

“p-value” = P[X ≥ 11 | p = 0.6]

(since “=” gives safest result)

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6]

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6]

Generally: use


unusual as X = 11 | H0 is

true]

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6]

Generally: use


unusual as X = 11 | H0 is

true]

Here use boundary between H0 & H1

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6]

Generally: use


unusual as X = 11 | H0 is true]

Here use boundary between H0 & H1

(above e.g. p = 0.6)

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6]

Now calculate numerical value

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6]


(already done above,

Class EG 4)

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6] = 0.217



Class EG 4)

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6] = 0.217



Class EG 4)

How to interpret?

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6] = 0.217

Intuition: p-value reflects chance of error

when H0 is rejected

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6] = 0.217


when H0 is rejected

(i.e. when conclusion is made)

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6] = 0.217


when H0 is rejected


(based on available evidence)

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6] = 0.217


when H0 is rejected


(based on available evidence)

When p-value is small, it is safe to make a

firm conclusion

Hypothesis Testing

For small p-value, safe to make firm conclusion

Hypothesis Testing


How small?

Hypothesis Testing


How small?

Approach 1: Traditional (& legal) cutoff

Hypothesis Testing


How small?


Called here “Yes-No”:

Hypothesis Testing


How small?



Reject H0 when p-value < 0.05

Hypothesis Testing


How small?




(just an agreed upon value,

but very widely used)

Hypothesis Testing


How small?




(but sometimes want different values,

e.g. your airplane is safe to fly)

Hypothesis Testing

Approach 1: “Yes-No”


Hypothesis Testing



Terminology: say results are “statistically

significant”, when this happens

Hypothesis Testing





Sometimes specify a value α

Greek letter “alpha”

Hypothesis Testing





Sometimes specify a value α

as the cutoff (different from 0.05)

Hypothesis Testing

Approach 2: “Gray Level”

Idea: allow “shades of conclusion”

Hypothesis Testing



e.g. Do p-val = 0.049 and p-val = 0.051

represent very different levels of evidence?

Hypothesis Testing



Use words describing strength of evidence:

0.1 < p-val: no evidence

0.01 < p-val < 0.1 marginal evidence

p-val < 0.01 very strong

evidence

Hypothesis Testing






evidence

Hypothesis Testing






evidence

stronger when closer to 0.01

Hypothesis Testing





p-val < 0.01 very strong evidence

stronger when closer to 0.01

weaker when closer to 0.1

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6] = 0.217

Bottom Line:

Yes-No: can not reject H0, since

0.217 > 0.05

i.e. no firm evidence pits better than

surgery

Gray level: not much indicated

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6] = 0.217

No firm evidence pits better than

surgery


Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6] = 0.217


surgery


Practical Issue: since 73% = observed rate for

pits > 60% (surgery),

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6] = 0.217


surgery



pits > 60% (surgery), may want to gather

more data

Hypothesis Testing

“p-value” = P[X ≥ 11 | p = 6] = 0.217


surgery



pits > 60% (surgery), may want to gather

more data, might show value of pits

Research Corner

Medical Imaging – Another Fun ExampleMedical Imaging – Another Fun Example

Cornea DataCornea Data

Research Corner



Cornea = Outer surface Cornea = Outer surface

of eyeof eye

Research Corner




of eyeof eye

““Curvature” important toCurvature” important to

visionvision

Research Corner




of eyeof eye

““Curvature” important toCurvature” important to

visionvision

Study Study heat map heat map showingshowing

curvaturecurvature

Research Corner


Heat map Heat map shows curvatureshows curvature

Each image is one personEach image is one person

Research Corner




Understand “populationUnderstand “population

variation”?variation”?

Research Corner




Understand “populationUnderstand “population

variation”?variation”?

(too messy for brain(too messy for brain

to summarize)to summarize)

Research Corner


Approach: PrincipalApproach: Principal

Component AnalysisComponent Analysis

Research Corner




Idea: follow “direction” inIdea: follow “direction” in

image space, image space,

Research Corner




Idea: follow “direction” inIdea: follow “direction” in

image space, that highlightsimage space, that highlights

population featurespopulation features

Research Corner


Population featuresPopulation features

Research Corner



• Overall curvatureOverall curvature

(hot – cold)(hot – cold)

Research Corner





• With the rule astigmatismWith the rule astigmatism

(figure 8 pattern)(figure 8 pattern)

Research Corner





• With the rule astigmatismWith the rule astigmatism

(figure 8 pattern)(figure 8 pattern)

• CorrelationCorrelation

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6

Now suppose X had been 13 out of 15

(cured by pits)

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


(cured by pits)

(recall above saw 11 / 25 not conclusive,

so now suppose stronger evidence)

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


So 1513ˆ n

Xp

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


So %7.86ˆ1513 n

Xp

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


So

(more conclusive than before)

%60%7.86ˆ1513 n

Xp

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


So

(more conclusive than before)

(how much stronger is the evidence?)

%60%7.86ˆ1513 n

Xp

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


So

p-value = P[ X ≥ 13 | p = 0.6]

%7.86ˆ1513 n

Xp

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


So

p-value = P[ X ≥ 13 | p = 0.6] = 0.027

%7.86ˆ1513 n

Xp

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


So

p-value = P[ X ≥ 13 | p = 0.6] = 0.027

Calculated similar to above:http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg4.xls

%7.86ˆ1513 n

Xp


Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


p-value = P[ X ≥ 13 | p = 0.6] = 0.027

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


p-value = P[ X ≥ 13 | p = 0.6] = 0.027

Conclusions:

Yes-No: 0.027 < 0.05, so can reject H0 and

make firm conclusion pits are better

Hypothesis Testing

H0: p < 0.6 vs. H1: p ≥ 0.6


p-value = P[ X ≥ 13 | p = 0.6] = 0.027

Conclusions:

Yes-No: 0.027 < 0.05, so can reject H0 and

make firm conclusion pits are better

Gray Level: Strong case, nearly very strong that

pits are better

Hypothesis Testing

In General:

p-value = P[what was seen,

or more conclusive | at

boundary between

H0 & H1]

Hypothesis Testing

In General:



boundary between

H0 & H1]

(will use this throughout the course,

well beyond Binomial distributions)

Hypothesis Testing

HW C14: Answer from both gray-level and

yes-no viewpoints:

(a) A TV ad claims that less than 40% of

people prefer Brand X. Suppose 7 out of

10 randomly selected people prefer Brand

X. Should we dispute the claim? (p-value

= 0.055)

Hypothesis Testing

HW C14: Answer from both gray-level and

yes-no viewpoints:

(b) 80% of the sheet metal we buy from

supplier A meets our specs. Supplier B

sends us 12 shipments, and 11 meet our

specs. Is it safe to say the quality of B is

higher? (p-value = 0.275)

Warning

Avoid the “Excel Twiddle Trap”

Warning

Avoid the “Excel Twiddle Trap”, E.g. C14(a)

Warning


Find what Excel needs:

Warning


Find what Excel needs:

Number_s: 7

Trials: 10

Probability_s: 0.4

Cumulative: true

(plug in)

Warning


Check given answer

(0.055)

Warning


Check given answer

(0.055)

Way off!

Warning


Check given answer

(0.055)

Way off! Try “1 -”

i.e. target (0.945)

Warning


Check given answer

(0.055)


i.e. target (0.945)

Still off, how about

the “> vs. ≥” issue?

Warning


Check given answer

(0.055)


i.e. target (0.945)



try replacing 7 by 6?

Warning


Check given answer

(0.055)


i.e. target (0.945)



try replacing 7 by 6? Yes!

Warning

Avoid the “Excel Twiddle Trap”:

• Can solve HW OK

Warning


• Can solve HW OK

• But not on exam

– No numerical answer given

– No interaction with Excel

Warning


• Can solve HW OK

• But not on exam

– No numerical answer given

– No interaction with Excel

• Real Goal: Understanding Principles

And now for something completely different

Lateral Thinking: What is the phrase?


Lateral Thinking: What is the phrase?

Card

Shark


Lateral Thinking:What is the phrase?



Knight Mare



Gator Aide

Hypothesis Testing

In General:



boundary between

H0 & H1]

Hypothesis Testing

In General:



boundary between

H0 & H1]

Caution: more conclusive requires careful

interpretation

Hypothesis Testing


interpretation

Hypothesis Testing


interpretation

Reason: Need to decide between

1 - sided Hypotheses

Hypothesis Testing


interpretation


1 - sided Hypotheses, like

H0 : p < vs. H1: p ≥

some given numerical value

Hypothesis Testing


interpretation



H0 : p < vs. H1: p ≥

And 2 - sided Hypotheses

Hypothesis Testing


interpretation



H0 : p < vs. H1: p ≥

And 2 - sided Hypotheses, like

H0 : p = vs. H1: p ≠

Hypothesis Testing


H0 : p = vs. H1: p ≠

Note: Can never have H1: p =

Hypothesis Testing


H0 : p = vs. H1: p ≠

Note: Can never have H1: p = ,

since can’t tell for sure between

and + 0.000001

Hypothesis Testing


H0 : p = vs. H1: p ≠

Note: Can never have H1: p = ,

since can’t tell for sure between

and + 0.000001

(Recall: H1 has burden of proof)

Hypothesis Testing


interpretation

1 - sided Hypotheses & 2 - sided

Hypotheses

Hypothesis Testing


interpretation


Hypotheses

(important choice will need to make a lot)

Hypothesis Testing


interpretation


Hypotheses

Useful Rule: set up 2-sided when problem

uses words like “equal” or “different”

Hypothesis Testing

e.g. a slot machine

• Gambling device

Hypothesis Testing

e.g. a slot machine

• Gambling device

• Players put money in

Hypothesis Testing

e.g. a slot machine

• Gambling device

• Players put money in

• With (small) probability, win a “jackpot”

(of quite a lot more money)

Hypothesis Testing

e.g. a slot machine bears a sign which says

“Win 30% of the time”

Hypothesis Testing



(in real life, focus is on “return rate”)

Hypothesis Testing




(since people enjoy fewer, but bigger jackpots)

Hypothesis Testing




(since people enjoy fewer, but bigger jackpots)

(but usually no signs,

since return rate is < 0)

Hypothesis Testing



In 10 plays, I don’t win any.

Hypothesis Testing




Can I conclude sign is false?

Hypothesis Testing




Can I conclude sign is false?

(& thus have grounds for complaint,

or is this a reasonable occurrence?)

Hypothesis Testing



In 10 plays, I don’t win any. Conclude false?

Let p = P[win]

Hypothesis Testing




Let p = P[win]

(usual approach: give unknowns a

name, so can work with)

Hypothesis Testing




Let p = P[win], let X = # wins in 10 plays

Hypothesis Testing





Model: X ~ Bi(10, p)

Hypothesis Testing






(set up as H0, the point want to disprove)

Hypothesis Testing






Test: H0: p = 0.3 vs. H1: p ≠ 0.3

Hypothesis Testing

e.g. a slot machine bears a sign which says “Win

30% of the time”




Test: H0: p = 0.3 vs. H1: p ≠ 0.3

(“false” means don’t win 30% of time,

so go 2-sided)

Hypothesis Testing

Aside (similar to above):

• Can never set up H0: p ≠ 0.3

Hypothesis Testing



• And then prove that p = 0.3

Hypothesis Testing




• Since can’t handle gray area of hypo test

Hypothesis Testing





• E.g. can’t distinguish from p = 0.30001

Hypothesis Testing





• E.g. can’t distinguish from p = 0.30001

• Could always be “off a little bit”

Hypothesis Testing






Test: H0: p = 0.3 vs. H1: p ≠ 0.3

Hypothesis Testing






Test: H0: p = 0.3 vs. H1: p ≠ 0.3

(now test & see how weird X = 0 is, for p = 0.3)

Hypothesis Testing






Test: H0: p = 0.3 vs. H1: p ≠ 0.3

p-value = P[X = 0 or more conclusive | p = 0.3]

Hypothesis Testing

Test: H0: p = 0.3 vs. H1: p ≠ 0.3


Hypothesis Testing

Test: H0: p = 0.3 vs. H1: p ≠ 0.3


(understand this by visualizing # line)

Hypothesis Testing

Test: H0: p = 0.3 vs. H1: p ≠ 0.3


0 1 2 3 4 5 6

Hypothesis Testing

Test: H0: p = 0.3 vs. H1: p ≠ 0.3


0 1 2 3 4 5 6

30% of 10, most likely when p = 0.3

i.e. least conclusive

Hypothesis Testing

Test: H0: p = 0.3 vs. H1: p ≠ 0.3


0 1 2 3 4 5 6

so more conclusive includes

Hypothesis Testing

Test: H0: p = 0.3 vs. H1: p ≠ 0.3


0 1 2 3 4 5 6

so more conclusive includes

but since 2-sided, also include

Hypothesis Testing

Generally how to calculate?

0 1 2 3 4 5 6

Hypothesis Testing


Observed Value

0 1 2 3 4 5 6

Hypothesis Testing


Observed Value

Most Likely Value

0 1 2 3 4 5 6

Hypothesis Testing


Observed Value

Most Likely Value

0 1 2 3 4 5 6

# spaces = 3

Hypothesis Testing


Observed Value

Most Likely Value

0 1 2 3 4 5 6

# spaces = 3

so go 3 spaces in other

direct’n

Hypothesis Testing

Result: More conclusive means

X ≤ 0 or X ≥ 6

0 1 2 3 4 5 6

Hypothesis Testing


X ≤ 0 or X ≥ 6


Hypothesis Testing


X ≤ 0 or X ≥ 6


= P[X ≤ 0 or X ≥ 6 | p = 0.3]

Hypothesis Testing


X ≤ 0 or X ≥ 6


= P[X ≤ 0 or X ≥ 6 | p = 0.3]

= P[X ≤ 0] + (1 – P[X ≤ 5])

Hypothesis Testing


X ≤ 0 or X ≥ 6


= P[X ≤ 0 or X ≥ 6 | p = 0.3]

= P[X ≤ 0] + (1 – P[X ≤ 5])

= 0.076

Hypothesis Testing


X ≤ 0 or X ≥ 6


= P[X ≤ 0 or X ≥ 6 | p = 0.3]

= P[X ≤ 0] + (1 – P[X ≤ 5])

= 0.076

Excel result from:http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg4.xls


Hypothesis Testing

Test: H0: p = 0.3 vs. H1: p ≠ 0.3

p-value = 0.076

Hypothesis Testing

Test: H0: p = 0.3 vs. H1: p ≠ 0.3

p-value = 0.076

Yes-No Conclusion: 0.076 > 0.05,

so not safe to conclude “P[win] = 0.3”

sign

is wrong, at level 0.05

Hypothesis Testing

Test: H0: p = 0.3 vs. H1: p ≠ 0.3

p-value = 0.076



sign


(10 straight losses is reasonably likely)

Hypothesis Testing

Test: H0: p = 0.3 vs. H1: p ≠ 0.3

p-value = 0.076



sign


Gray Level Conclusion: in “fuzzy zone”,

some evidence, but not too strong

Last Time Binomial Distribution –Excel Computation Political Polls –Strength of evidence...

Documents

Transcript of Last Time Binomial Distribution –Excel Computation Political Polls –Strength of evidence...