Dichotomous Tests (Tom)

•Their results change the probability of disease

Negative test Positive test

Reassurance TreatmentOrder a Test

•A good test moves us across action thresholds.

0% 100%

T+T-

•The best tests are definitive

What tests do

Post-Test Probability of Disease Depends on 2 Things

1. Where you started from (low, medium, high)

2. Length and direction of the “arrow” Basic paradigm:

What we thought before + test result = what we think now

Prior probability + LR from test = post-test probability

LR = P(Result|Disease)/P(Result|No Disease)

Assessing information from dichotomous tests (review):

  Disease + Disease - Total

Test + a b a+b

True Positives False Positives Total Positives

Test - c d c+d

False Negatives True Negatives Total Negatives

Total a+c b+d Total N

Total With Disease

Total without Disease

Sensitivity=a/(a+c) Specificity =d/(b+d)Positive predictive value (PPV) = a/(a+b); Negative predictive value (NPV) d/(c+d)Prior probability = P(D); Posterior probability = P(D|test result)

False-negative confusion

Sensitivity of rapid strep test is 85%

Therefore, false negative rate is 15%

15% is too high, so always culture to confirm negative rapid strep tests

What’s wrong?Strep No StrepTotal

Rapid Test + TP FP TP+FPRapid Test - FN TN TN+FN

TP+FN FP+TN 2 definitions of “false negative rate”

1-sensitivity = FN/(TP+FN). This one is easier because it’s (assumed to be) constant.

1 - negative predictive value = FN/(FN+TN). This one is harder because it depends on prior probability, but it is the one that should determine clinical decisions.

If prior probability of strep = 20%

False negative rate (def #2) = 15/407 = 3.7%

NNC (number needed to culture) = 1/.037 = 27 to identify 1 false negative rapid test. (Pre-test probability of 20%)

At some prior probability of strep, culture after negative quick test is not indicated.

Strep No Strep TotalRapid test + 85 8 95Rapid test - 15 392 407Total 100 400 500

(Assumes 98% specificity)

Similar examples:

Sensitivity of UA for UTI is only 80%, therefore always culture after a negative UA

Sensitivity of CT scan for subarachnoid hemorrhage is only 90%, therefore always do LP after a negative CT

Importance of Sampling Scheme

If sampling separately from Disease+ and Disease– groups (case-control sampling), cannot calculate prevalence, positive predictive value, or negative predictive value.

Dx Test:Case-Control SamplingDisease +Sampled

Separately

Disease –Sampled

Separately

Test +a

True Positivesb

False Positives

Test -c

False Negatives

dTrue

Negatives

Total

a + cTotal With

Disease

b + dTotal

WithoutDiseaseSensitivity = a/(a + c)

Specificity = d/(b + d)

Dx Test: Cross-sectional Sampling

Prevalence = (a + c)/NPositive Predictive Value = a/(a + b)Negative Predictive Value = d/(c + d)

Disease + Disease - Total

Test + aTrue Positives

bFalse Positives

a + bTotal

Positives

Test - cFalse

Negatives

dTrue

Negatives

c + dTotal

Negatives

Total a + cTotal With

Disease

b + dTotal

WithoutDisease

a + b + c + dTotal N

R. henselae titers and Cat Scratch Disease*

Case Control

R. henselae titer

Positive 38 4 42

Negative 4 108 122

45 112

*Zangwill, N Engl J Med. 1993;329:8-13. EBD Problem 3.2

Authors stated negative predictive value = 38/42 = 90.5%. Is there a problem?

Example from Chapter 3

65-year-old woman with mammogram suspicious for malignancy

Pre-test probability ≈ 0.015LR(“suspicious for malignancy”) ≈

100Post-test probability = ?

Update Pre-Test Probability Using LR(test result)

1) Convert pre-test probability (P) to pre-test odds. Pre-Test Odds = P/(1-P)

2) Calculate LR. P(result|D+)/P(result|D-). 3) Post-Test Odds = Pre-Test Odds × LR4) Convert post-test odds to post-test

probability. Prob = Odds/(1+Odds)

Update Pre-Test Probability Using LR(test result)

1) Pre-test probability P = 0.015Pre-test odds = P/(1-P) ≈ 0.015

2) LR(“Suspicious for Malignancy”) = 100

3) Post-Test Odds = 0.015 × 100 = 1.54) Post-test probability =

Odds/(1+Odds) = 1.5/2.5 = 0.60

Can Use Slide Rule

Threshold Model

Single disease (D+,D-) with single treatment (no further testing available)

Cost of failing to treat D+ = B Cost of treating D- unnecessarily = C Treat if P(D) > C/(C+B) C/(C+B) = Treatment Threshold

Probability = PTTPauker SG, Kassirer JP.. N Engl J Med. 1975 Jul

31;293(5):229-34.

Pauker SG, Kassirer JP.. N Engl J Med. 1975 Jul 31;293(5):229-34.

Define Costs B and C

“X-Graph”

Introduce a Dichotomous (+/-) Test

P(+|D+) = Probability of positive test “given” D+ = Sensitivity

P(-|D-) = Probability of negative test “given” D- = Specificity

P(+|D-) = 1 – Specificity or “False Positive Rate”

P(-|D+) = 1 – Sensitivity of “False Negative Rate”

T = Cost of Test

Pauker SG, Kassirer JP. N Engl J Med. 1980 May 15;302(20):1109-17.

“X-Graph”

New “X-Graph”

Threshold Formulas

Assumptions in the Threshold ModelThreshold Model: One disease One dichotomous test Only two post-test options: treat and no treat

Real world: Multiple possible diseases Multiple possible test results (not just

+/-) Multiple possible tests Multiple post-test options including

observation and additional testing

2) Multilevel Tests (Michael)

Likelihood ratios for results other than “+” or “-”