Sensitivity and Specificity
description
Transcript of Sensitivity and Specificity
![Page 1: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/1.jpg)
Sensitivity and SpecificityPart II – Computations and Examples
![Page 2: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/2.jpg)
This video is designed to accompany
pages 81-94in
Making Sense of UncertaintyActivities for Teaching Statistical
ReasoningVan-Griner Publishing Company
![Page 3: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/3.jpg)
ImPACT Example
Concern about concussions in young athletes is at an all-time high. A variety of screening tests are available for assessing concussion status, some are to be used on the field; others take more time. One such test is the “ImPACT” test, which stands for “Immediate Post-Concussion Assessment and Cognitive Testing.”
How well does ImPACT perform as a screening test?
![Page 4: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/4.jpg)
ImPACT Example
Rule: Beyond a certain level on the ImPACT scale you are said to be
concussed.Actual Status
ImPACT Prediction Athlete was, in
reality, not concussed
Athlete was, in reality,
concussed TotalsImPACT said “not
concussed” 59 13 72
ImPACT said “concussed” 7 59 66
Totals 66 72 138
“Sensitivity and specificity of the ImPACT Test Battery for concussion in athletes.” Archives of Clinical Neuropsychology, 2005
FPR = 7/66 = 0.11 or 11%FNR = 13/72 = 0.18 or 18%
Specificity = 89%Sensitivity = 82%
![Page 5: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/5.jpg)
Beck Inventory Example
The original BDI, first published in 1961, consisted of twenty-one questions about how the subject has been feeling in the last week. A typical question is below.
Choose the most appropriate response (0) I do not feel sad. (1) I feel sad. (2) I am sad all the time and I can't snap out of it. (3) I am so sad or unhappy that I can't stand it.
How well does Beck perform as a screening test?
![Page 6: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/6.jpg)
Beck Inventory Example
Rule: If your total score on the Beck Inventory is 10 or greater then you
will be categorized as clinically depressed.
Actual Status as determined by
Diagnostic and Statistical Manual of Mental Disorders IV
Beck Inventory Indication
Patient was, in reality, not depressed
Patient was, in reality,
depressed TotalsBeck said “not
depressed” 66 5 71
Beck said “depressed” 12 12 24
Totals 78 17 95
“Sensitivity and Specificity of Depression Questionnaires in a College-Age Sample.” Journal of Genetic Psychology, 2008, 169(3), 281–288
FPR = 12/78 = 0.14 or 14%FNR = 5/17 = 0.29 or 29%
Specificity = 85%Sensitivity = 71%
![Page 7: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/7.jpg)
Field Sobriety Testing
There are three common testing procedures implemented during an FST: horizontal gaze nystagmus (HGN), one-leg stand (OLS), and walk and turn (WAT).
Data from 296 subjects who participated in the NHTSA’s 1998 San Diego field sobriety test validation study, ordered by total scores on these three FST procedures, are available .
Let’s set a cutoff for the field rule and assess how well the rule is working as a screening test for intoxication.
![Page 8: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/8.jpg)
Rules
For The Test: assume that a 4 or above on the FST will tag a participant as drunk.
Concerning The Truth: assume that a BAC of 0.04% or greater means a participant is legally drunk. There were 267 study participants with BACs of 0.04% or greater.
![Page 9: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/9.jpg)
Task at Hand
Using Total FST >=4 to Designate “Drunk”
FST Decision
Actual BAC Totals< 0.04
(Sober)>= 0.04 (Drunk)
SoberDrunkTotals 29 267 296
Fill out this table!
![Page 10: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/10.jpg)
Field Sobriety TestingCase HGN OLS WAT Total FSTActual BAC
229 0 0 1 1 0254 0 0 1 1 0.0266 1 1 0.067
142 2 0 0 2 0.005217 2 0 0 2 0.03191 2 2 0.034182 2 0 0 2 0.038109 2 2 0.04259 2 0 0 2 0.04199 0 1 1 2 0.048113 2 0 0 2 0.0522 2 2 0.0667 2 1 3 0.022
145 0 3 0 3 0.0353 2 0 1 3 0.03215 2 0 1 3 0.04
287 2 0 1 3 0.0489 2 1 0 3 0.05
123 0 1 2 3 0.05258 2 1 0 3 0.05335 2 0 2 4 011 2 1 1 4 0.01
247 4 4 0.0166 2 0 2 4 0.02
294 2 0 2 4 0.02231 2 0 2 4 0.0374 2 1 1 4 0.04
214 2 1 1 4 0.0458 4 4 0.0514 2 1 1 4 0.05834 4 0 0 4 0.05812 2 0 2 4 0.06
211 2 1 1 4 0.06232 4 0 0 4 0.06293 2 2 0 4 0.06130 2 1 1 4 0.07297 4 4 0.08271 2 0 2 4 0.1119 2 2 4 0.121
Using Total FST >=4 to Designate “Drunk”
FST Decision
Actual BACTotals
< 0.04 >= 0.04
Sober
Drunk
Totals 29 267 296
209 11
276
20 256
FPR = 20/29 or 69%Specificity = 9/29 or 31%FNR = 11/267 or 4%Sensitivity = 256/267 or 96%
![Page 11: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/11.jpg)
What If?
Change the rule: assume that a 2 or above on the FST will tag a participant as drunk.
Changes that occur: FPR will surely go up and FNR will surely go down.
So the point is: how well a screening test performs is directly related to the cutoff it uses to identify a “positive.”
![Page 12: Sensitivity and Specificity](https://reader030.fdocuments.in/reader030/viewer/2022033022/568166a9550346895ddaa064/html5/thumbnails/12.jpg)
One-Sentence Reflection
Simple fractions are used to compute sensitivity and specificity in many situations where both test results and the truth are arrayed in 2x2 tables.