Stascal Thinking at the Introductory Level Stascs and ... › ~nolan › talks › beijing ›...
Transcript of Stascal Thinking at the Introductory Level Stascs and ... › ~nolan › talks › beijing ›...
7/21/12
1
Sta)s)calThinkingattheIntroductoryLevel
DeborahNolanUniversityofCalifornia,Berkeley
Sta)s)csandMathema)cs
AdaptedfromCobbandMoore,Mathema)cs,Sta)s)cs,andTeaching
Sta)s)csisa
• Mathema)calscience• Datascience• Computa)onalscience
• Sta)s)csisnotasubfieldofmathema)cs
• Sta)s)csmakesessen)aluseofmathema)cs
Aphorisms:
GeorgeBox:• Allmodelsarewrong,butsomeareuseful
GeorgeCobb:
• Inmathema)cs,contextobscuresstructure.Indataanalysis,contextprovidesmeaning
7/21/12
2
Aphorisms:
DavidMoore:• Mathema)caltheoremsaretrue:sta)s)calmethodsaresome)meseffec)vewhenusedwithskill
Variability
• Needofsta)s)csarisesfromtheomnipresenceofvariability
• Repeatedmeasurementsonthesameindividualvary.
• Some)meswewanttofindunusualindividuals
• Other)meswefocusonthevaria)onofmeasurements.
• Other)meswewanttodetectsystema)ceffectsagainstthebackgroundnoiseofindividualvaria)on.
Theroleofcontextinsta)s)cs
Context
• Sta)s)csrequiresadifferentkindofthinking• Dataarenotjustnumbers
• Dataarenumberswithacontext
7/21/12
3
Mathema)csandContext
• Contextisusedformo)va)on• Contextisasourceofproblems
• Ul)mately:– Contextistheirrelevantdetailthatweignoretorevealthehiddenpurestructure.
– Contextobscuresstructure.– FocusisonabstractpaVerns
DataAnalysisandContext
• FocusisalsoonstructureandpaVerns• Ul)mately,– ThepaVernshavemeaning,– Thestructurehasvalue,– IfthepaVernsmakesenseinthecomplementarythreadsofthestoryline
• Contextprovidesmeaning
Implica)onsforteaching
Implica)onsforteaching
• Needmorethanmathema)caltheory• Needtounderstandthenon‐mathema)caltheoryofsta)s)cs
AND
• Needrealillustra)ons• Needtouseillustra)onstodevelopcri)caljudgment
7/21/12
4
Essen)alpiecesofsta)s)calanalysis
• Designfordataproduc)on• Explora)onforpaVernsandstructure• Formula)onofmodels
• Applica)onofmethods
• Summarizeresults
• Interpreta)onofresults
ASA/MAARecommenda)ons
• Almostanysta)s)cscoursecanbeimprovedonby– Moreemphasisondataanalysis
– Moreemphasisonconcepts– Fewerrecipes– Lesstheory
ASA/MAARecommenda)ons
• Mainfocusofanintroductorycourseshouldbeonsta)s)calthinking.
• Sta)s)calThinkingincludes:– Theneedfordata– Theimportanceofdataproduc)on,– Theomnipresenceofvariability,– Thequan)fica)onandexplana)onofvariability.
Sta)s)cstaughtasmagic
• Studentisthesorcerer’sappren)ce• Incanta)onhasautoma)ceffec)veness,e.g.rendersastudypublishable
• Appren)ceisnotmeanttounderstandhowtheincanta)onworks
• Followtherecipeexactly,beVeryet–useso]ware
7/21/12
5
CounterSta)s)csasmagic
• Retrea)ngtomathema)csdoesnotsolvetheproblem
CounterSta)s)csasmagic
• Presentanintellectualframeworkthat– Makessenseofthecollec)onoftoolsthatsta)s)ciansuse
– Encouragesflexibleapplica)onoftoolstosolveproblems.
– Reasonsfromuncertainempiricaldata
TopicsTodayandTomorrow
Workshoptopics
1) Examplesofrealillustra)onstodevelopastudent’scri)caljudgment
2) Compu)ngtechnologyhascompletelychangedtheprac)ceofsta)s)csandisanecessarytool
3) BoxModel–Ateachingstrategyforlearninghowrandomnessindataproduc)onleadstoinference
7/21/12
6
Workshoptopics
4) Engagingingraphicsearlycanestablishgoodhabits,preparefordesignandinference,provideexperiencewithdatadistribu)ons,introduceconcepts
5) Physicalexamplescangiveapictoralgraspofasta)s)calconceptandbeeffec)veinconveyingideas
Sta)s)csCoursesatBerkeley
IntroductoryCourses
• Quan)ta)veReasoningrequirementforArtsandHumani)esmajors
• Sta)s)csforBusinessmajors
• Sta)s)csforstudentswithcalculusbackground
IntroductorySta)s)csCourse
• 120students• Firstandsecondyearstudents• Undeclaredmajors,interestedineconomicsandbiologicalandphysicalscience
• Calculusprerequisiteforthecourse• 3hoursof“lecture”aweek• 2hoursoflab–25studentstothelab
7/21/12
7
• Threeversionsoftheintroductorycourse• Allfocusonsta%s%calthinking• PhilosophyofSta%s%cs,Freedman,Pisani,andPurves
• Thisphilosophyappearsthroughoutthecurriculum–– inthetheore)calcoursesforthesta)s)csmajorand
– inthePhDlevelcourses
Threestories:Realillustra)onstodevelopastudent’scri)caljudgment
RandomizedControlledExperiments
TheHIPTrialadaptedfromSta)s)calModels:TheoryandPrac)ce,Freedman
Background
• Breastcancercommonmalignancyamongwomen
• IfDetectedearly,thenchanceofsuccessfultreatmentbeVer
• Mammography–screeningbyX‐ray
7/21/12
8
Doesmammographyspeedupdetec)onenoughtomaVer?
HealthInsurancePlaninNewYork
• HIP–groupmedicalprac)cewith700,000membersin1960s
• Subjects:62,000women– Aged40‐64– MembersofHIP
• Splitatrandomintotwogroups
Treatments
• “Treatment”:invita)onto4roundsofannualscreening– Clinicalexam
– Mammography
• Control:receivedusualhealthcare
Results
GroupSize
BreastCancerNumber
Rate
Treatment 31,000 39 1.3
Control 31,000 63 2.0
Arethesetherightnumberstocompare?Notallwomeninthetreatmentgroupacceptedtreatment
7/21/12
9
Results
GroupSize
BreastCancerNumber
Rate
Treatment
Screened 20,200 23 1.1
Refused 10,800 16 1.5
Total 31,000 39 1.3
Control 31,000 63 2.0
Adifferentcomparison:thosewhoacceptedscreeningtothosewhorefused.
Results
GroupSize
BreastCancerNumber
Rate
Treatment
Screened 20,200 23 1.1
Refused 10,800 16 1.5
Total 31,000 39 1.3
Control 31,000 63 2.0
Anothercomparison:thosewhoacceptedscreeningtothoseincontrolgroup.
Whichcomparisonmakessense?
Considera)ons
• Inves)gatorschose(atrandom)thosetoreceivetreatment
• Subjectsthemselvesdecidedwhetherornottoaccepttreatment
• ComparingthosewhoaccepttothosewhorefuseisanObserva)onalComparison
7/21/12
10
Differencesbetweengroups
• RicherandbeVer‐educatedsubjectsmorelikelytoacceptinvita)onthanthosewhowerepoorerandlesswell‐educated
• Richerwomenarelessvulnerabletomostdiseases,butbreastcancerhitsrichharder
• Socialstatusisaconfoundingfactor:afactorassociatedwiththeoutcomeandwiththedecisiontoacceptscreening
Whichcomparison?
• Thecomparisonofthosewhoaccepttreatmenttothosewhorefuseisbiasedagainstscreening
• Thecomparisonofthosewhoaccepttreatmenttothoseinthecontrolgroupisalsoproblema)cbecausethecontrolgroupincludeswomenwhowouldhaverefusedscreening
Results
GroupSize
BreastCancerNumber
Rate
AllOtherNumber
Rate
Treatment
Screened 20,200 23 1.1 428 21
Refused 10,800 16 1.5 409 38
Total 31,000 39 1.3 837 27
Control 31,000 63 2.0 879 28
• Experimentalcomparison:betweenwholetreatmentgroupandwholecontrolgroup
• Inten)on‐to‐treatanalysis• Effectoftheinvita)on:– 63–39=24livessaved– Inrela)veterms39/63=62%
7/21/12
11
SurveySample
TheHiteReport,adaptedfromSampling:Designand
Analysis,byLohr
Publishedin1987
Surveyof4,500women
Hite’ssample&USpopula)on
Income Hite’sSampleU.S.popula)on
under$2,000 19.0% 18.3%$2,000‐$4,000 12.0% 13.2% $4,000‐$6,000 12.5% 12.2%$6,000‐$8,000 10.0% 9.7%$8,000‐$10,000 7.0% 7.4%$10,000‐$12,500 8.0% 8.8%$12,500‐$15,000 5.0% 6.2%$15,000‐$20,000 10.0% 9.8%$20,000‐$25,000 8.0% 6.4%$25,000andover8.5% 8.2%
Hite’ssampleandUSpopula)onTypeofarea Hite’sSample U.S.popula)on
Largecity/urban 60% 62%Rural 27% 26%Smalltown 13% 12%Race Hite’sSample U.S.popula)on
White 82.5% 83.0%Black 13.0% 12.0%Hispanic 1.8% 1.5%Asian 1.8% 2.0%
7/21/12
12
AccordingtotheWomenSurveyed:
• 84%ofwomenare“notsa)sfiedemo)onallywiththeirrela)onships”.
• 70%ofallwomen“marriedfiveormoreyearsarehavingsexoutsideoftheirmarriages”.
• 98%wanttomakebasicchangesintheirrela)onships.
• 84%ofwomenreportformsofcondescensionfromthemenintheirloverela)onships.
Thissurveyappearedtobeground‐breaking?
Or,didsomethinggowrong?
OtherStudiesandExperts:
• Harrispoll(1987)89%saytheirrela)onshipwiththeirpartnerissa)sfying.
• "Anyques)onyouaskedthatgot98%iseitherawrongques)onorwronglyphrased"saysTomSmithoftheNa)onalOpinionResearchCenter.
• Severalotherpollsfound25‐30%whoaremarriedhavehadorarehavinganextramaritalaffair.
Ques)onnaire
• Vaguewordingofques)ons,e.g.“inlove”• Leadingques)ons:Doesyourhusband/loverseeyouasanequal?Orarethere%meswhenheseemstotreatyouasaninferior?Leaveyououtofthedecisions?Actsuperior?
• Longsurveywith127essayques)ons,manywithseveralparts
7/21/12
13
Selec)onBias
• Ques)onnairesmailedtowomen’sgroupsincludingprofessionalwomen’sorganiza)ons,counselingcenters,churchsocie)es,…Non‐responsehigh:100,000ques)onnairesmailed,4.5%returned
• Self‐selectedsample
=USWomen
Women’sGroups
Notincluded=WomennotbelongingtoaWomen’sGroup
Belongtogroupbut:Didn’treceiveques)onnaireChosenottofillout
Womenwhobelongtoagroupandtookthe)metocompletethesurvey
=Girls&Meninthegroups
HypothesisTes)ng
UnitedStatesv.KristenGilbert
(adaptedfromCobbandGehlbach,“Sta)s)csintheCourtroom”,inSta%s%cs:AGuidetotheUnknown)
KristenGilbert
• Bornin1967inFallRiver,MA• Graduatedhighschoolat16• GraduatedfromGreenfieldCC,andreceivedcer)fica)onasaregisterednursein1988.
• In1989,shejoinedtheVAMedicalClinicinNorthampton,MA.
7/21/12
14
VAMedicalClinic
Whenapa)entwentintocardiacarrest,shewould:– Soundthecodebluealarm
– Staycalm– Administerashotofepinephrinetorestarttheheart
Gilbertestablishedareputa)onofbeingpar)cularlygoodincrisis
Suspicions
• Bythemid1990’s,othernurseshadbecomesuspiciousofGilbert.
• Itseemedthereweretoomanycodebluecalls,toomanycriseswhenGilbertwasontheward.
• Anini)alVAreportfoundthatthenumberofdeathswereconsistentwithpaVernsatotherVAhospitals.
• Thesuspicionsofthestaffremained.
Suspicions
• Thestaffbroughttheirconcernsagaintotheadministra)onoftheVAClinic.
• Theyhiredasta)s)cianasaconsultanttolookintothesitua)on(Gehlbach)
• Hisfindingsagreedwiththestaffconcerns.
AssistantU.S.AVorneyWelchconvenedagrandjuryin1998toheartheevidenceagainstGilbert.
7/21/12
15
GrandJury
• Agrandjurydetermineswhetherthereisenoughevidenceforatrial.
• Thegrandjuryexaminesevidenceandissuesanindictment,aformalaccusa)onthatapersonhascommiVedacrime.
EvidenceConsidered
• MoJvaJon–Gilbertlikedthethrillofacrisis,neededtherecogni)on,andwantedtoimpressherboyfriendwhoalsoworkedattheVAClinic.
• TesJmonyofco‐workersaboutaccessGilberthadtoepinephrine.
• TesJmonyofaphysicianaboutthesymptomsofthemen(healthy,middle‐aged,nottypicalcandidatesforcardiacarrest).
Convincing?
• NoonehadseenGilbertgivefatalinjec)ons.• AmajorpartoftheevidencewasstaJsJcal.
QUESTION:WeretheresomanyexcessdeathswhenGilbertwaspresentastobesuspiciousintheeyesofscience?
7/21/12
16
Gelbach’sTes)mony
• PaVernofdeaths,byshi]andbyyearonthemedicalwardwhereGilbertworked
• Variabilityinachanceprocess
• Sta)s)caltestforwhetherthepaVernlinkingtheexcessdeathstoGilbert’spresenceonthewardwastooextremetoberegardedasordinary,expectedvariability
PaVerninDeaths
1988 1990 1992 1994 1996
010
20
30
40
year
Deaths
Night
Day
Evening
PaVerninDeaths
• ThereisaclearpaVernassocia)ngGilbert’spresencewithexcessdeaths
• However,thepaVernmightbenothingmorethantheresultofordinary,expectablevariaJon.
Sta)s)calTest
• Considerthe18monthsleadinguptoFeb1996,whenGilbertwentonmedicalleave.
• Therewere547daysinthis18monthperiodand3shi]saday,foratotalof1641shi]s
• Foreachshi],wehavewhetherornotGilbertworkedtheshi]andwhetherornottherewasadeathontheshi].
7/21/12
17
ASta)s)calTest
!"#$%&'(&)%*+$,&
-*./"0$&10")"($,& & 234& & (5& & $5678&
234& & & & & 9:& & &&&;<=& & &&&;>=&
(5& & & & & ?9& & <?>:& & <?@9&
$5678& & & & & =9& & <>A=& & <A9<&
&
• Thetablesummarizesrecords.• Inthe1641shi]stherewere74shi]sforwhichtherewasatleastonedeath.• On40ofthe74shi]s,Gilbertwasworking.Isthatmorethanyouwouldexpect?
P(atleast40)<1inatrillion
0 20 40 60
0.00
0.04
0.08
0.12
Shifts with at least one death
Chance
Howlikelyisittogetatleast40deathsinthe257shi]s?
GrandJury
• Thegrandjuryfoundthesta)s)calevidencepersuasive
• Gilbertwasindicted• TheVAhospitalislegalpropertyofthefederalgovernmentsoitwasafederalindictment.
• Trialwouldbeinfederaldistrictcourt,andthedeathpenaltywouldbeapossiblesentence,iffoundguilty.
TheTrial
• ThepeJtjury(ortrialjury)hearstheevidenceinatrialaspresentedbyboththeplain)ffandthedefendant.
• A]erhearingtheevidence,thegroupre)resfordelibera)on,toconsideraverdict.
• Themajorityrequiredforaguiltyverdictwasasimplemajority(7outof12).Aunanimousverdictforthesentencewasneededforthedeathpenalty.
7/21/12
18
ExpertWitness
Thecourtsystemallowsexperttes)monywhentheevidenceinvolvesspecializedtechnicalorscien)ficissuesthatgobeyondwhatjurorswouldordinarilybefamiliarwith.
TheexpertshelpthejuryunderstandtheevidencebeVer.
DuelingExpertWitnesses
• SupremeCourthassetguidelinesatmakingsureunscien)fictes)monyisnotadmiVed.
• Ifthereisexperttes)monyononeside,aVorneysfortheothersidesome)meshireanotherexpertwhowilldisagreeand“cancelout”theotherexpert.
QUESTION:Shouldthetrialjurybeallowedtohearthesta)s)calevidence?
ReporttotheJudge
• Wasthesta)s)calanalysisdonecorrectly?• Whatdoestheprobabilitycalcula)onNOTtellyou?
• Associa)onvsCausa)on:ConclusionsdrawnfromanObserva)onalStudyvsaDesignedExperiment
• Prosecutor’sFallacy:Probabilitycomputedunderassump)onsofinnocence
7/21/12
19
Wasthesta)s)calanalysisdonecorrectly?
• TheDefenseSta)s)cian(Cobb)agreedwiththeanalysisperformedbytheProsecu)onSta)s)cianforthegrandjury.
• Thenumberofexcessdeathswasanextremelyunlikelyoutcomeduetochancevaria)on.
• ThepaVernofdeathsjus)fiedtheindictment.
Associa)onvsCausa)on
• Inawell‐designedexperiment,e.g.theSalkFieldtrials,theonlydifferenceisinthetreatment,allotherpossiblecausesofaneffecthavebeeneliminated.
• The)nyprobabilityrulesoutchancevaria)on,andtheconclusionisthatthedifferenceis“real”.
• Inadesignedexperiment,wecanconcludethattheexplana)onfortheobserveddifferenceisthetreatment.
Associa)onvsCausa)on
• ThiswasNOTarandomizedcontrolledexperiment.(Gilbert’spresenceonthewardwouldhavehadtobeassignedusingachancedevice).
• Wecanconcludethatthedifferenceisnotduetochancevaria)on,butthe)nyprobabilitydoesnotprovideanexplana)onforwhathappened.
• Therecouldbeotherpossibleexplana)ons.
Prosecutor’sFallacy
• Theprobabilityof1inatrillionwascomputedassumingthataresultisduetochancevaria)on.
• Wecomputethechanceofgexngaresultasextremeastheoneobserved.
• Ifitisveryrare()nyprobability),weconcludethatitisnotreasonabletothinkthatrandomvaria)onisthecause.
• Thislogicsaysnothingaboutothercauses
7/21/12
20
Prosecutor’sFallacy–slipperylogic
• SupposeGilbertisinnocentandthedeathsbehaveinachance‐likeway.Theprobabilityislessthan1inatrillionthatyouwouldseesomanyexcessdeathsonGilbert’sshi]s.
• IfGilbertisinnocent,thenitwouldbealmostimpossibletogetsomanyexcessdeaths.
• Withthismanyexcessdeaths,thechanceislessthanoneinatrillionthatGilbertisinnocent.FALSELOGIC
Conclusion
• Itisveryeasytobetemptedbythefalselogic(thattheprobabilityisthechanceofinnocence).
• Judgeruledthatsta)s)calevidenceshouldnotbeallowedattrial.
• JuryfoundGilbertguiltyon3countsoffirst‐degreemurderand2countsofaVemptedmurder.
• Juryvoted8‐4foradeathpenalty.• Gilbertwasgivenlifeinprisonwithoutpossibilityofparole.
Exercisestotry
Considera)ons
• Provideananswerin“plainEnglish”or“plainChinese”
• Iden)fythecoresta)s)calthinkingconcept• Howisthemathinsufficientforansweringtheques)on?
7/21/12
21
Discussion
• Whatarethechallengestoteachingthisway?
• Hardtoteachtheseconcepts• Hardtogradestudentwork• Contextcandependonculturalbackground• O]entheexamplesareveryreduc)onist
• Thiscanleadtoanover‐cri)calapproach
Discussion
• Whatarethebenefitstoteachingthisway?
• Seethatthereismoretosta)s)csthanmanipula)onofformula
• Gainprac)ceinsta)s)calthinking• Seeingmanyexampleswillhelpwhenconfrontnewproblems
Sta)s)csshouldbetaughtassta)s)cs
Resources
• Sta%s%cs,Freedman,Pisani,Purves
• Sta%s%csaGuidetotheUnknown,Mostelleretal,edi)on
• Sta%s%csaGuidetotheUnknown,Pecketall,edi)on
• StatLabs:TheorythoughApplica%ons,Speed&Nolan