Core Conference Series: Numerical Concepts in Clinical Study Interpretation Type I and Type II error in Clinical Trials
“Truth” Clinical Benefit No Benefit
Trial Result
“Positive” “Correct” Type I error
“Negative” Type II error “Correct” The “p-value” = probability of type I error—the probability of finding benefit where there is no benefit. “α” The power = 1 - probability of type II error—the probability of finding no benefit when there is benefit. “1-β” The sample size a function of the study design, effect size, and acceptable type I and type II error. Sensitivity, Specificity, PPV, NPV, and Likelihood Ratios in Diagnostic Testing
“Truth” Disease + Disease -
Test Result
“Positive” True Positive
False Positive
“Negative” False Negative
True Negative
Sensitivity, specificity, and likelihood ratios are inherent to the test; PPV and NPV depend on specificity, sensitivity, and disease prevalence Relative Risk Difference vs. Absolute Risk Difference
N Alive
N Dead Mortality
Absolute Risk
Reduction
Relative Risk Reduction
Number Needed to
Treat Study 1 Treatment 80 20 0.20 0.05 1-(0.20/0.25)=0.20 1/ARR=20
N=200 Control 75 25 0.25
Study 2 Treatment 980 20 0.02 0.01 1-(0.02/0.03)=0.33 1/ARR=100
N=2000 Control 975 30 0.03 Greater relative risk reduction ≠ “better” treatment!
𝑆𝑒𝑛𝑠 =𝑇𝑃
𝑇𝑃 + 𝐹𝑁 𝑆𝑝𝑒𝑐 =𝑇𝑁
𝑇𝑁 + 𝐹𝑃
𝑃𝑃𝑉 =𝑇𝑃
𝑇𝑃 + 𝐹𝑃 𝑁𝑃𝑉 =𝑇𝑁
𝑇𝑁 + 𝐹𝑁
𝐿𝑅+=𝑠𝑒𝑛𝑠
1− 𝑠𝑝𝑒𝑐 𝐿𝑅−=1− 𝑠𝑒𝑛𝑠𝑠𝑝𝑒𝑐
Diagnostic Testing Example: DDimer for VTE—low risk population 1000 pts VTE
Yes No DDimer Positive 56 536 592
Negative 4 404 408 60 940 1000
Example: DDimer for VTE—high risk population 1000 pts VTE
Yes No
DDimer Positive 263 410 673 Negative 17 310 327
280 720 2000
Relative vs. Absolute Risk
N Total N Dead Mortality Absolute
Risk Reduction
Relative Risk Reduction
Number Needed to
Treat Prostate Cancer
Screening Screen 72,952 214 0.0030
Control 89,435 326 0.0036
Low VT for ARDS Low VT 432 134 0.31
Control 429 171 0.40
𝑆𝑒𝑛𝑠 =𝑇𝑃
𝑇𝑃 + 𝐹𝑁 = 𝑆𝑝𝑒𝑐 =𝑇𝑁
𝑇𝑁 + 𝐹𝑃 =
𝑃𝑃𝑉 =𝑇𝑃
𝑇𝑃 + 𝐹𝑃 = 𝑁𝑃𝑉 =𝑇𝑁
𝑇𝑁 + 𝐹𝑁 =
𝐿𝑅+=𝑠𝑒𝑛𝑠
1− 𝑠𝑝𝑒𝑐 = 𝐿𝑅−=1− 𝑠𝑒𝑛𝑠𝑠𝑝𝑒𝑐 =
𝑆𝑒𝑛𝑠 =𝑇𝑃
𝑇𝑃 + 𝐹𝑁 = 𝑆𝑝𝑒𝑐 =𝑇𝑁
𝑇𝑁 + 𝐹𝑃 =
𝑃𝑃𝑉 =𝑇𝑃
𝑇𝑃 + 𝐹𝑃 = 𝑁𝑃𝑉 =𝑇𝑁
𝑇𝑁 + 𝐹𝑁 =
𝐿𝑅+=𝑠𝑒𝑛𝑠
1− 𝑠𝑝𝑒𝑐 = 𝐿𝑅−=1− 𝑠𝑒𝑛𝑠𝑠𝑝𝑒𝑐 =
Diagnostic Testing Example: DDimer for VTE—low risk population 1000 pts VTE
Yes No DDimer Positive 56 536 592
Negative 4 404 408 60 940 1000
Example: DDimer for VTE—high risk population 1000 pts VTE
Yes No
DDimer Positive 263 410 673 Negative 17 310 327
280 720 2000
Relative vs. Absolute Risk
N Total N Dead Mortality Absolute
Risk Reduction
Relative Risk Reduction
Number Needed to
Treat Prostate Cancer
Screening Screen 72,952 214 0.0030 0.0006 1-(0.0030/0.0036)
=0.17 1/ARR=
1667
Control 89,435 326 0.0036
Low VT for ARDS Low VT 432 134 0.31 0.09 1-(0.31/0.40)=0.23 1/ARR=11
Control 429 171 0.40
𝑆𝑒𝑛𝑠 =𝑇𝑃
𝑇𝑃 + 𝐹𝑁 =5660 = 0.93 𝑆𝑝𝑒𝑐 =
𝑇𝑁𝑇𝑁 + 𝐹𝑃 =
404940 = 0.43
𝑃𝑃𝑉 =𝑇𝑃
𝑇𝑃 + 𝐹𝑃 =56592 = 0.09 𝑁𝑃𝑉 =
𝑇𝑁𝑇𝑁 + 𝐹𝑁 =
404408 = 0.99
𝐿𝑅+=𝑠𝑒𝑛𝑠
1− 𝑠𝑝𝑒𝑐 =0.93
1− 0.43 = 1.63 𝐿𝑅−=1− 𝑠𝑒𝑛𝑠𝑠𝑝𝑒𝑐 =
1− 0.930.43 = 0.16
𝑆𝑒𝑛𝑠 =𝑇𝑃
𝑇𝑃 + 𝐹𝑁 =263280 = ~0.93 𝑆𝑝𝑒𝑐 =
𝑇𝑁𝑇𝑁 + 𝐹𝑃 =
310720 = 0.43
𝑃𝑃𝑉 =𝑇𝑃
𝑇𝑃 + 𝐹𝑃 =263673 = 0.39 𝑁𝑃𝑉 =
𝑇𝑁𝑇𝑁 + 𝐹𝑁 =
310327 = 0.95
𝐿𝑅+=𝑠𝑒𝑛𝑠
1− 𝑠𝑝𝑒𝑐 =0.93
1− 0.43 = 1.63 𝐿𝑅−=1− 𝑠𝑒𝑛𝑠𝑠𝑝𝑒𝑐 =
1− 0.930.43 = 0.16
Top Related