Jeffrey P Schaefer MSc MD FRCPC April 24, 2009 2 nd Annual Resident Evidence-Based Medicine Workshop...

Jeffrey P Schaefer MSc MD FRCPC

April 24, 2009

2nd Annual Resident Evidence-Based Medicine Workshop

Diagnosis

Objectives

• Diagnostics– General Issues– Gold Standard– Test Characteristics– Critical Appraisal

Medical Diagnostics

historydiagnostic

testsphysical

examination

Medical Diagnosticsgeneral process

historydiagnostic

testsphysical

examination

Diagnostic Testing

• Advantages– can assess parameters beyond the 5 senses– can be more ‘objective’ than clinical data

• Disadvantages– test results can be incorrect– test results may lead you in the wrong direction– tests cost money– tests may confer risk– some diseases have no diagnostic test

Issues in Diagnostic Testing

• Risk– urine sample versus brain biopsy versus autopsy

• Cost– glucoscan strip ~ $1.00 versus MRI $1,000.00

• Availability– hemogram versus Positive Emission Tomogram

• Patient Acceptability– urine sample versus 3 day fecal fat collection

Clinical Scenario

• A 70 year old man presents to the ED complaining chest pain and shortness of breath for one hour.

• He has prostate cancer and just completed a car trip from Vancouver.

• He’s on no medications nor has allergies.

• He is distressed. Pulse is 130 / min, respiratory rate is 32 / min. There are no other physical findings.

What’s the clinical question?

What’s the DIAGNOSIS?

What are some causes for this man’s presentation?

• We need an approach...

Sidebar: Approach to Diagnosis

• JP Schaefer’s classification scheme….– 7 approaches

1. Epidemiological Approach

• What’s common in this clinical setting?

– Right Lower Quadrant Pain: 18 year old male

– Right Lower Quadrant Pain: 80 year old

appendicitis

cecal carcinoma

2. Physiological Approach

• What pathophysiology is causing the condition?

• Hypoxemia– shunt– v/q mismatch– alveolar hypoventilation– decreased pAO2– increased diffusion gradient– (low mixed venous oxygen pressure)

3. Anatomical Approach

• Where is the problem?

• Chest Pain– skin: shingles– muscle: strain / injury / nail gun– rib / spine: fracture / tumor– lung / pleura: pneumonia, embolism– heart / pericardium: angina / pericarditis– esophagus: spasm / tumor

4. Pathological Approach

• What pathology is involved?

• Left sided weakness– brain tumor (slow)– stroke (fast)– multiple sclerosis (recurrent)

5. Pattern Recognition

• What’s this?– have you seen this before? (Herpes simplex

labialis secondary to HSV Type 1 in an immunocompromised patient)

6. Interventional Efficacy

• Headache– muscle tension– migraine– meningioma– warning bleed of SAH

7. No approach

• Common problem– used to be my method!

Back to the case...• DDx chest pain and shortness of breath…

– epidemiology

–anatomy– physiology– pathology– pattern recognition– interventional efficacy

Test for Pulmonary Embolism

• Gold Standard: pulmonary angiogram– invasive– costly– not readily available– risky

• Other tests:– D-dimer, V/Q scans, Spiral CT scan– ? may be helpful in right setting with right results

- complex

PE - diagnosis

Pulmonary angiogram

- gold standard

PE - diagnosis (spiral CT scan)

PE - diagnosis (V/Q scan)

• high probability V/Q scan (2 defects)

Pulmonary Thromboembolism

How well does the test perform?

• Welcome to the world of

TEST CHARACTERISTICS

Take a deep breath...

Test Characteristics

• Sensitivity

• Specificity

• Positive predictive value

• Negative predictive value

• Accuracy

• Positive Likelihood ratio

• Negative Likelihood ratio

DISEASE

Present Absent

Positive TRUE POSITIVE

FALSE POSITIVE

TEST

Negative FALSE NEGATIVE

TRUE NEGATIVE

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Hypothetical Test Results

Sensitivity

• Probability that test is positive given that disease is present.

P (T+ | D+)

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Sensitivity

80 / (80 + 10) = 88.9%

Specificity

• Probability that test is negative given that disease is absent.

P (T- | D-)

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Specificity

90 / (90 + 20) = 81.8%

Sensitivity - Specificity Trade-Off

• Most test results are not positive or negative.

• There is often a selected value– over which a test is said to be positive – under which a test is said to be negative.

• As a result….– increasing sensitivity results in loss of specificity– increasing specificity results in loss of sensitivity

Sensitivity / Specificity Trade-off

Sensitivity DecreasesSpecificity Increases

Sensitivity / Specificity Trade-off• Receiver Operating Characteristic (ROC) curve

Test Characteristic Issues

• Highly Sensitive Tests:– tend to be less invasive, less risky, less costly– best for screening programs– best for ruling out disease: “SNOUT”


• Highly Specific Tests:– tend to be more invasive, more risky, more costly– best for confirming (ruling in) disease: “SPIN”

Positive Predictive Value

• Probability that disease is present given that the test was positive.

P (D+ | T+)

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Positive Predictive Value

80 / (80 + 20) = 80.0%

Negative Predictive Value

• Probability that disease is absent given that the test was negative.

P (D- | T-)

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Negative Predictive Value

90 / (90 + 10) = 90.0%


• Positive and Negative Predictive Values suffer from depending on disease prevalence

• This is a major drawback.*

(* excellent exam question)

Change Disease Prevalence from 90 to 110 per 200

DISEASE (PE)

Present Absent

Positive

TRUE POSITIVE

a = 80 97.7

FALSE POSITIVE

b = 20 16.4

a + b = 114.1 TEST

(V/Q scan)

Negative

FALSE NEGATIVE

c = 10 12.2

TRUE NEGATIVE

d = 90 73.6

c + d = 85.8

a + c = 90

110 b + d = 110 90

a+b+c+d = 200

prevalence = 110 / 200 = 0.55 = 55% (was 45%)

sensitivity = 97.7 / 110 = 88.8% (unchanged)specificity = 73.6 / 90 = 81.7% (unchanged)

positive predictive value = 86.5% (was 80%)negative predictive value = 85.8% (was 90%)

Accuracy

• Probability that the test is true.

• (not a useful concept as you’ll see later)

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Accuracy

(80+90) / (80+ 20 + 10 + 90) = 85.0%


• Accuracy:– not useful characteristic– high sensitivity / low specificity test may have

same accuracy as low sensitivity / high specifity test

(positive) Likelihood Ratio

• Ratio of:

probability of positive test when disease is present

--------------------------------------------------------------------

probability of positive test when disease is absent

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Positive Likelihood Ratio

(80 / 90) / (20 / 110) = 4.89

Utility of (Positive) Likelihood Ratios

• expresses how many times more likely a test result is to be found in diseased, compared to nondiseased, people.

• can estimate the post-test probability of disease if prevalence is known.

Pre-test Probability of Disease

• Consider: a female presents for a screening breast mammogram for breast cancer.

• What’s her pre-test probability of disease?

Prevalence of Disease

Positive Test Result

• Say that her mammogram show her to have a 1 cm spiculated calcification

• Say that this finding is associated with a likelihood ratio of 20 (a very suspicious lesion).

Highly suspicious lesion

What is the post-test probability of disease?

Answer:

Pretest odds x Likelihood Ratio = Posttest odds

(the use of odds ratios makes the math convoluted)



Assume: prevalence = 10 / 1000 = 1% = P(0.01)

Odds = probability of event / (1 - probability of event)

Pre-test Odds = (10/1000) / (1 - (10/1000)) = 0.0101



0.0101 x 20 = 0.2020

Probability = Odds / (1 + Odds)

Posttest Probability = 0.2020 / (1 + 0.2020)

Posttest Probability = 0.167 = 16.7%

Utility of (Positive) Likelihood Ratio

Pre-test Probability = 1%

Post-test Probability = 16.7%

Prudent Course: move from screening test to confirmatory test!

Critical AppraisalArticles about Diagnosis

• Are the results in the study valid?

• What are the results?

• Will the results help care for my patients?

Validity

1. Was there an independent, blind comparison with a reference standard?

2. Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice?

3. Did the results of the test being evaluated influence the decision to perform the reference standard?

4. Were the methods for performing the test described in sufficient detail to permit replication?

Results

• Are likelihood ratios for the test results presented or data necessary for their calculation provided?

How much do Likelihood Ratios (LRs) change disease likelihood?

LRs >10 or <0.1 cause large changes in likelihood.

LRs 5-10 or 0.1-0.2 cause moderate changes.

LRs 2-5 or 0.2-0.5 cause small changes.

LRs between <2 and 0.5 cause little or no change.

Applicability

• Will the reproducibility of the test result and its interpretation be satisfactory in my setting?

• Are the results applicable to my patient?

• Will the results change my management?

• Will patients be better off as a result of the test?

Volume 292 January 2, 1975 Number 1

• Immunoblastic lymphadenopathy. A hyperimmune entity resembling Hodgkin's disease

• Immunoblastic lymphadenopathy with mixed cryoglobulinemia. A detailed case study

• Vinyl-chloride-induced liver disease. From idiopathic portal hypertension (Banti's syndrome) to Angiosarcomas

• Hodgkin's Disease, tonsillectomy and family size

• Reduction of ischemic injury by nitroglycerin during acute myocardial infarction (no abstract available)

• Frederick Stohlman, Jr., M.D

TREATMENT EFFECT

Jeffrey P Schaefer MSc MD FRCPC April 24, 2009 2 nd Annual Resident Evidence-Based Medicine Workshop...

Documents

Transcript of Jeffrey P Schaefer MSc MD FRCPC April 24, 2009 2 nd Annual Resident Evidence-Based Medicine Workshop...