Family Med EBM Conference

Some History

• Often MI’s > abnormal beats (Ventricular Premature Depolarizations).

• Increase beats = increase risk of Sudden Death

• Therefore, Give meds & decrease extra beats = decrease risk & increase survival

• And in the 80’s, we did,…

Question?

• “Are we really saving lives?”

• No research comparing encainide/flecainide vs placebo.

• So researcher attempted to prove it,..

• After 10 months they looked at the blinded data and saw,…

I told you so,…

• A clear win for medicine,

• Stopped trial & opened envelopes

Group A

(730)

Group B

(725)

Mortality 56 (7.7%)

22 (3.0%)

arrhythmia death or cardiac arrests

33

(4.5%)

9

(1.2%)

NEJM 1989; 321(6): 406-12

That can’t be,…

• Actually, Treatment (encainide/flecainide)was harming people

• Relative risk of dying was 157% higher

• The Number Needed to Harm (kill) 1 extra patient was only 21.

Treatment

(730)

Placebo

(725)

Mortality 56 (7.7%)

22 (3.0%)

arrhythmia death or cardiac arrests

33

(4.5%)

9

(1.2%)

i. NEJM 1989; 321(6): 406-12

It happens all the time,…

• Past teaching proven incorrect with evidence

– Vitamins prevent oxidation

– OTC cough suppressants for kids

– Lubricant on speculum wrecks PAP tests

– Morphine reduces Dx accuracy in Abdo pain

– Low Carb Diets are the solution.

– Topical NSAID is an inefficacious CAM Tx

– Oral HRT for urinary incontinenceLancet 2002; 360: 23-33 & Lancet 2003;361:2017-23. Arch Dis Child 2002; 86: 170-5. Obstet Gynecol 2002; 100: 889-92. J Am Coll Surg 2003; 196: 18-31. JAMA 2003; 289 (14): 1837-50 & NEJM 2003; 348 (21): 2082-90. www.Bandolier.com March 05. JAMA 2005; 293(8): 935-48

If it’s not RCT,…

• Case study & series lowest on the list but– Thalidomide danger 1st via case report– Torsades with Terfenadine & Itraconazole

• Cohort: Smoking is Bad! (no RCT, ever*)

McBride Lancet 1961;ii, 1358. Pohjola-Sintonen et al. Eur J CLin Pharm 1993;45:191-3.

Doll & Peto. BMJ 1964;i;:1399-414,1460-7.

In Summary?

• EBM is the best we’ve got?

• Understand the basics

• Always think and ask questions ?

So how did we get to this curriculum?

Why This Curriculum: Past Research

• Teaching critical appraisal in isolation doesn’t help doctors become EBM users.

• Doctors don’t do critical appraisal• Doctors spend 2 minutes to look up questions • Workshops increase Doctors Knowledge Base• Integration into daily practice (with clinical questions &

web-resources) changes practice.• Biggest Barriers have been Time and Knowledge

Acad Med. 1999;74:686-94. CMAJ 1998;158:177-81. BMJ. 2002 7;325:1338-41. Acad Med. 2000;75:1212-4. BMJ 1999; 319: 358-61 Acad Med. 2002;77: 741-2. BMJ. 2004;329: 1017. Acad Med. 2003 Mar;78:270-4.

Why This Curriculum: Our Research

• Residents come from varied backgrounds and many report EBM training was limited in Quality and Quantity

• Residents think EBM results better patient care and can be practical.

• Biggest Barrier is time.

Our EBM Curriculum.• Give everyone some basic EBM knowledge

– This Workshop.

• Give everyone easy access to EBM resources– The Desktop.

• An assignment integrated into clinical practice that reflects what is done in future practice. – The BEAR’s

• Give a journal club enhance rapid appraisal – Resident Journal Club

• Keep you up to date– The Quarterly Lit Review

BEAR Work-Sheet (Title): ____________________________________________

Name of Resident: Date:

Question: _________________________________________________________________________________________________________________________________

Search: (Check all that apply)Pubmed/Ovid/Medline: □ Filtered Resources: □ Summary/Review Sites: □

College/Society/Guidelines: □ Other: □ ( Describe: ___________________ )Number of Resources Reviewed: ____

Resources (Top 3)#1 Resource: Abstract □ Paper □ Filtered Article □ Summary □Review/Meta-Analysis □ College/Society/Guideline Paper □ Other Research □

– Abbreviated Citation: __________________________________________– Strengths:___________________________________________________– Weaknesses:_________________________________________________

Take-Home Message:__________________________________________________

#2,…

Bottom-line: _________________________________________________________________________________________________________________________________________________________________________________________________________

Practice (These findings had a): Large Change □ Small Change □ Reassured □ No Help □

Therapy Articles

Much Thanks to: Rob Hayward & Tanya Voth, CCHE

• Generally:– Recognize the primary features which answer

• Are the Results Valid• Number needed to treat.

• Specifically,• Learn the Language • Learn where to look to rapidly get answers.

Objectives: Practical EBM

Users Guides

Basics of the Randomized Control Trial (RCT)

• RCT is an experiment

• We divide people in to 2 or more groups (“Randomly”)

• We subject the groups to different interventions (a Treatment and a “Control”)

• Then compare them

Are the Results Valid

1) Did groups begin the study with a similar prognosis?a) Were patients randomized?

• Without:– Prognostic factors = determinants of outcome– Sometimes we know Prognostic factors,

sometimes we don’t– Outcomes affected by many factors other than the

experimental intervention

Are the Results Valid?

1) Did groups begin the study with a similar prognosis?a) Were patients randomized?

• Remedy:– Effect of randomization is to evenly distribute prognostic

factors between experimental and control groups– Balancing both known

and unknown prognostic factors– eliminating selection & confounding biases

Are the Results Valid?

1) Did groups begin the study with a similar prognosis?b) Was randomization concealed?

(“Allocation Concealment”)• Threat:

– Inconvenience (difficult) and influence (clinicians want power to allocate pts) inconsistent use of randomization

– Randomization can fail

Are the Results Valid?

1) Did groups begin the study with a similar prognosis?b) Was randomization concealed?

• Remedy:– Remote randomization: allocation made by persons

not working with patients– Concealed randomization: e.g., sealed, opaque

envelopes– Only done in 7-55% of trials1 and it can bias trials

33-41%2

1. BMJ 2004 3;328:22-4. ACP J Club. 2000 Mar-Apr;132:A11 2. JAMA 1995; 273: 408-412. Lancet. 1998;352:609-13.

Are the Results Valid?

1) Did groups begin the study with a similar prognosis?c) Were patients analyzed in the groups

to which they were randomized? (“Intention to Treat Analysis”)?

• Threat:– Compliance correlates with outcome– Rates of cross-over or loss to follow-up correlate with

outcome

Are the Results Valid?1) Did groups begin the study with a similar

prognosis?c) Were patients analyzed in the groups to

which they were randomized? (“Intention to Treat Analysis”)?• Remedy:

– Intention to treat analysis balance effect of compliance losses in both groups

– Only 7-48% of articles do ITT (& less properly)– It can bias trials up to 51%

BMJ. 1999; 319: 670-4. J Fam Pract. 2002; 51: 969-71. BMJ 2003;326: 117-5. J Clin Epidemiol. 2003; 56: 833-42

Are the Results Valid?

1) Did groups begin the study with a similar prognosis?d) Were the patients in treatment and

control groups similar with respect to known prognostic factors? (Baseline Characteristics)

• Threat:– Randomization does not always work, or may not be

possible imbalance of prognostic factors

Are the Results Valid?

1) Did groups begin the study with a similar prognosis?d) Were the patients in treatment and

control groups similar with respect to known prognostic factors? (Baseline Characteristics)

• Remedy:– Magnitude of imbalance? – Strength of poorly distributed factor(s)?– Are adjusted and unadjusted analyses the same?

Are the Results Valid?

2) Did groups retain a similar prognosis after the study started?a) Were patients clinicians and analyzers

aware of the group allocation? (Blinded)• Threat:

– Placebo effects, expectation bias, Co-interventions, • Remedy:

– Mask (blind) patients to exposure type– Inadequate blinding favours treatment by about

17%

JAMA. 1995;273:408-12.

Are the Results Valid?

2) Did groups retain a similar prognosis after the study started?d) Was follow-up complete?

• Threat:– Losses are all persons whose status is not

known at the end of the study.• Remedy:

– Equivalent follow-up for the two groups?– Sensitivity analysis to losses

Users Guides

• Are the results valid?– Prognosis similar at the beginning?– Prognosis similar at the end?

• What are the results?– Size of effect?– Precision of effect?

• How can I apply the results?– Patients like yours?– All important outcomes?

What are the results?

• How large was the treatment effect?Control and Experimental Event Rates? (often Placebo and Drug Event rates = CER & EER)

• Outcome• + - • Risk of Outcome• Experimental a b EER = a / total in Exp• Control c d CER = c / total in

Control

Hypothetical Post-MI Trial of 200 Patients

CER = # with outcome in Control / # who got ControlCER = 20 / 100 = 20%

EER = # with outcome in Experiment / # who got ExpEER = 10 / 100 = 10%

Mortality Total Enrolled/ Group

Placebo 20 100

Treatment 10 100

What are the results?

• How large was the treatment effect?What is the absolute risk reduction?

• The absolute risk reduction is the

• difference in risk between the control

• group and the treated group: •

• ARR = CER-EER

How can I apply the results?

• Are the likely benefits worth the harms and costs?What is the number needed to treat?

• The number needed to treat (NNT) is the• number of patients requiring

• treatment for one outcome to be avoided:

• NNT = 1/ARR = 1/(CER-EER)

RRR, ARR and NNTe.g. Treating Hypertension for MI Risk

(Framingham data)

• ARR = CER – EER• RRR= ARR / CER• NNT = 1 / AR

ARR= 20% - 10% = 10%

RRR= 10% / 20% = 50%

NNT= 1 / 0.1 = 10

CER: 60 y.o. M, HDL 1.0, BP>160 = 20%

EER: 60 y.o. M, HDL 1.0, BP<130 = 10%

RRR when event unlikely &RRR vs ARR (NNT)

Common Diseases• RRR = 50%

• CER = 20%• EER = 10%• ARR = 10%• NNT = 10

Uncommon Diseases• RRR = 50%

• CER = 1%• EER = 0.5%• ARR = 0.5%• NNT = 200

The Media: Women’s Health Initiative

• TV Evening News (September 24, 2003)

• 41% increase in stroke• 29% increase in MI• Double rate of VTE• 26% in Breast Ca• No benefits mentioned

• Actual Risk per 10,000 (JAMA 288(3):321-33)

• 8 strokes in 10,000 p. y.• 7 MI’s in 10,000 p. y.• 18 VTE in 10,000 p. y.• 8 Breast Ca in 10,000 • 6 Colorectal Ca & 5 Hip

Fractures Less/10,000

Bottom Line

• Absolute Excess Adverse Events in the Global Index are 19 in 10,000 patient years

OR

• 100 women on HRT for 5 years = 1 additional adverse event

What are the results?

• How precise was the treatment effect?What is confidence interval on the results?

• 95% CI:the range of treatment effect estimates that includes the true treatment effect 95% of the time.

• 95% CI reflects:precision (reliability) of the result; statistical significance.

Users Guides

• Are the results valid?– Prognosis similar at the beginning?– Prognosis similar at the end?

• What are the results?– Size of effect?– Precision of effect?

• How can I apply the results?– Patients like yours?– All important outcomes Considered?

How can I apply the results?

• Were the study patients similar to your patients?– Do study inclusion criteria fit your patients’

prognostic factors?– Compelling reasons for non-generalizability?– Dependent on subgroup analysis?

How can I apply the results?

• Were all clinically important outcomes considered?– What were the primary and secondary

endpoints?– Were surrogate outcomes used?– Were adverse outcomes considered?

How can I apply the results?

• Are the likely benefits worth the harms and costs?Is the reduction in illness worth the cost and

risk of harm?• NNT vs NNH?• Patients’ preferences and values?• Costs?

• The goal of a RCT is to make exposed and unexposed similar in all respects other than the intervention... and to maintain that balance throughout.

• The size and precision of treatment effects determines the importance of the results.

• Who was enrolled and what was measured (outcomes) are the most important determinants of applicability

Summary

The End

Much Thanks to: Rob Hayward & Tanya Voth, CCHE