An introduction to Evidence-Based

Medicinepart-1

Akbar Soltani, MDTehran University of Medical Sciences

Endocrine and Metabolism Research Center Evidence-Based Medicine Working Team

Shariati Hospitalwww.soltaniebm.com

www.ebm.irwww.avicennact.ir

Some assumptions

• You, the audience, between you know much more than I do about this

• Lao Tzu said: “Those who know do not speak/Those who speak do not know.”

• Kafka: What a silence had been established in the world if every person talk correlated with his/her knowledge

What is Critical Thinking?

• What is the best way of walking?

• What is the best way of thinking?

fshahrtash@gmail.com> 

Agenda

• Definitions: Definitions: Science and EBMScience and EBM

• Dimensions of the problemDimensions of the problem1. Information management (mastery)

2.2. Limitations of current clinical practiceLimitations of current clinical practice

3.3. Heuristic and errorsHeuristic and errors

4. Problems of communication

• EBM

Definition:• Science is devoted to formulating and

testing naturalistic explanations for natural phenomena. It is a process for systematically collecting and recording data about the physical world, then categorizing and studying the collected data in an effort to infer the principles of nature that best explain the observed phenomena.72 Nobel laureates. (From the Amicus Curiae presented in the US Supreme

Court Case of Edwards vs Agullard, 1986)

Observation

Systematic Nonsystematic

Accurate, precise

Not accurate/precise

More scientificMore

pseudoscientific

Reference based medicine

Is conventional/reference based medicine

systematicin search, appraisalextraction of data..?

Trace back to the development of EBM.

• 1972:Archie Cochrane published an book, and told about the important role of RCTs.

• 1980's: Dave Sackett at McMaster University in Canada.

• 1990s :The term was generated and coined by Gordon Guyatt and his colleagues from McMaster University

Trace back to the development of EBM

A model for evidence-based clinical A model for evidence-based clinical decisionsdecisionsSackett et al, 2000

background knowledge + evidence= decision making

Current bestCurrent best evidenceevidence

I.Individual Clinical Expertise:

• Experience: Relates to what we’ve done and to knowledge.

1. Clinical skills

2. Clinical judgment

3. Vital for determining whether the evidence applies to the individual patient at all and, if so, how

Expertise for

Diagnosis,Procedures

Interventions (diagnostic or

therapeutic)

Helps clinicians

Need accurate/precise

information

Do not need accurate/precise

information

Not adequate Adequate

II. Best External Evidence:

• From real clinical research amongintact patients.

• Has a short doubling-time (10 years).

III. Patients’ Values & Expectations

• Have always played a central role in determining whether and which interventions take place

A model for evidence-based clinical A model for evidence-based clinical decisionsdecisionsSackett et al, 2000

Bayesian approach: background knowledge + evidence= decision making

Current bestCurrent best evidenceevidence

Conventional medicine

Expertise(intuition…)

Pathophysiology, references, tradition…

Patient value

Agenda

• Definitions: Definitions: Science and EBMScience and EBM

• Dimensions of the problemDimensions of the problem1. Information management (mastery)

2.2. Limitations of current clinical practiceLimitations of current clinical practice

3.3. Heuristic and errorsHeuristic and errors

4. Problems of communication

• EBM

• MEDLINE has approximately:– 6 million references from – 4.000 journals with about – 400.000 new entries added each year.

Why Is It So Hard to Be Up-to-Why Is It So Hard to Be Up-to-date?date?

• Doubling time of biomedical Doubling time of biomedical science is about 20 months science is about 20 months in in 20012001

Increasing Knowledge

0

2000

4000

6000

8000

1966 1976 1986 1996

Number of articles on Hypertension cited in Medline by Year

Articles

RCT23 Patients

1960

1 20.5

Odds Ratio

Treatment Control

Thrombolytic Therapy & MI mortality

Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of results of meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992;268:240-8

CumulativeYear RCTs Pts

1960

1965

1 20.5

1 232 653 1494 3167 1793

Odds Ratio

Treatment Control

Antman JAMA 92

Thrombolytic Therapy & MI mortality

CumulativeYear RCTs Pts

1960

1965

1970

1975

1980

1985

1990

1 20.5

1 232 653 1494 3167 179310 254411 265115 331117 392922 545223 576727 612533 657165 4718570 48154

Odds Ratio

Treatment Control

p < 0.01

p < 0.001

p < 0.00001

Antman JAMA 92

Thrombolytic Therapy & MI mortality

Thrombolytic Therapy & MI mortalityCumulative

Year RCTs

Pts1960

1965

1970

1975

1980

1985

1990

1 20.5

1 232 653 1494 3167 179310 254411 265115 331117 392922 545223 576727 612533 657165 4718570 48154

Odds Ratio

Treatment Control

p < 0.01

p < 0.001

p < 0.00001

Textbook Recommendations

Rout Specif Exp NOT

21 510 2 8 7 812 4 3 1 1

112

872

111281

515 6

Antman JAMA 92

Antman JAMA 92

Some parts of textbooks are out-of-date

• Fail to recommend Rx up to ten years after it’s been shown to be efficacious.

• Continue to recommend therapy up to ten years after it’s been shown to be useless.

• Different textbooks, different recommendations.

• Textbooks are appraisable?

Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of results of meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992;268:240-8

Expert component • Professor Paul Knipschild has described how

Nobel prize winning biochemist Linus Pauling used selective quotes from the medical literature to "prove" his theory that vitamin C helps you live longer and feel better.

• When Knipschild and his colleagues searched the literature systematically for evidence

They found that

Expert component

• One or two trials did strongly suggest that vitamin C could prevent the onset of the common cold

• There were far more studies which did not show any beneficial effect.

Interesting ExampleInteresting Example

Dr Naserimoghaddam

182 Health authorities selected

2 Articles: 1 on cardiac rehabilitation 1 on breast Ca screening Results of each presented in 4 ways:

RRR (Relative Risk Reduction) ARR (Absolute Risk Reduction) PEFP (Proportion of Event Free Patients) NNT ( Number Needed to Treat)

Dr NaserimoghaddamEvidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)

They were told that these were the results of 4 articles on each topic

Question: According to which set of data you may choose to adopt the method as part of your regional practice policy?

Dr NaserimoghaddamEvidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)

Interesting Results !Interesting Results !

N=140N=140 MammographyMammographyCardiac Cardiac RehabilitationRehabilitation

RRRRRR 79%79% 76%76%

ARRARR 38%38% 56%56%

PEFPPEFP 38%38% 53%53%

NNTNNT 51%51% 62%62%

Dr NaserimoghaddamEvidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)

Only 3 noted that all 4 sets of data are the same!

None were clinicians!

Dr NaserimoghaddamEvidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)

Hypothesis?

Sample size estimation

None!

Failure to detect a difference

=Equivalence?

Assume non-inferiority if the lower limit of 95% CI is

less than –5%,N=904 per group!

Percent correct answers for knowledge questions

0

10

20

30

40

50

60

70

80

90

100

Sen90

Rule

In/O

ut

LR

NN

T

Hip

RCT

Agenda

• Definitions Definitions

• Dimensions of the problemDimensions of the problem1. Information management (mastery)

2.2. Limitations of current clinical practiceLimitations of current clinical practice

3.3. Heuristic and errorsHeuristic and errors

4. Problems of communication

• EBM

• A pervasive problem for primary care physicians attempting to appraise clinical information is the conflicting recommendations by experts.

Global judgment by expertsGlobal judgment by experts

Vote counting!

Variation in prostatectomyVariation in prostatectomy

Do you recommend HRT fore high LDL in postmenopausal patients?

ResidentResident

AttendingAttending : YES because estrogen increase HDL and decrease LDL, Lpa,and ………..

:

Pathophysiologic approachPathophysiologic approach

Evidence Based Fallacy

WHI: Coronary Heart Disease

years 1 2 3 4 5 6

Answering question

logic, mathematics, philosophy,

social science?

Empirical scienceMedicine…

Analyticthinking

Synthetic:Re/search,reading…

The Slippery Slope

years since graduation

r = -0.54p<0.001

...

...

. ... . .... .

....

....

.....

...knowledgeof current best care

100%

0%

50%

Choudhry, Fletcher and Soumerai, Ann Intern Med 2005;142:260-73-94% of 62 studies found decreasing competence for at least some tasks, with increasing physician age.

Agenda

• Definitions: Definitions: Science and EBMScience and EBM

• Dimensions of the problemDimensions of the problem1. Information management (mastery)

2.2. Limitations of current clinical practiceLimitations of current clinical practice

3.3. Heuristic and errorsHeuristic and errors

4. Problems of communication

• EBM

Heuristical errors

• Heuristic = rule of thumb; mental process used to learn, recall, or understand knowledge

• Some examples:– Recency– Rarity– “burned” by missing a case– Regression towards the mean– …

(Tversky& Kahneman, 1974)

Agenda

• Definitions Definitions

• Dimension of problemsDimension of problems1. Information management (mastery)

2.2. Limitations of current clinical practiceLimitations of current clinical practice

3.3. Heuristic and errorsHeuristic and errors

4. Problems of communication

• EBM

Probability estimates of various qualitative verbal expressions

Probable

Likely

Possible

Certain

Low probability

Suggests

High probability

Unlikely

Moderate probability

Pathognomonic

classic

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0Evidence-Based Medicine: A Framework for Clinical Practice by Friedland et. al

West Vs East – Language

American ways-A guide for foreigner Eloquent=expressive

Why Evidence-based Medicine?

• Science is a process for systematically collecting and recording data

• Time not available to find and assimilate evidence into practice

• Doubling time of biomedical science is about oubling time of biomedical science is about 20 months in 200120 months in 2001– Medical Journals: too voluminous– Scientific chaos

Dr Naserimoghadam

Why Evidence-based Medicine?• Traditional sources of info:

– Textbooks :partially (10-30%) outdated before publication

– Experts:• Pathophysiologic approach

• Conflicting recommendations

• Biased towards their own works & knowledge

• Heuristic and errors• Problems of communication

– CME: ineffective

• Clinical judgment / diagnostic skills increase with time, but up-to-date clinical knowledge declines

Dr Naserimoghadam

Recommendations • Think and search systematically!

• Learn methodology

• Learn errors of expertise

• Find EBM recourses

• …

An introduction to Evidence-Based

Medicinepart-2

Akbar Soltani, MD,MS.Tehran University of Medical Sciences

Endocrine and Metabolism Research Center Evidence-Based Medicine Working Team

Shariati Hospital

Esfand 87 (2009)www.soltaniebm.com

www.ebm.irwww.avicennact.ir

Agenda

• Definitions: Definitions: Science and EBMScience and EBM

• Dimensions of the problemDimensions of the problem1. Information management (mastery)

2.2. Limitations of current clinical practiceLimitations of current clinical practice

3.3. Heuristic and errorsHeuristic and errors

4. Problems of communication

• EBM

Evidence Based

Medicine

EBM process

1. Ask

2. Search

3. Appraise

4. Apply

5. EvaluateEvidence-Based Medicine: How to Practice and Teach EBM by David Sackett

Ask

• We need it twice for every 3 outpatients and 2 times for every inpatient

• Questions are most likely to be about treatment

• Most of the questions generated in consultations go unanswered.

P: Among patients with NIDDM who are having MI

I: does tight control of their blood sugar

C: in comparison to conventional methods

O: reduce their risk of dying?"

EBM process

EBM process

1. Ask

2. Search

3. Appraise

4. Apply

5. Evaluate

EBM process

2Efficient track-down of the best evidence –Secondary (pre-appraised) sources e.g.,– Cochrane (systematic reviews)

– E-B Journals

–primary literature

Example of a search strategy

#1 RANDOMIZED-CONTROLLED-TRIAL in PT#2 CONTROLLED-CLINICAL-TRIAL in PT#3 RANDOMIZED-CONTROLLED-TRIALS#4 RANDOM-ALLOCATION#5 DOUBLE-BLIND-METHOD#6 SINGLE-BLIND-METHOD#7 #1 or #2 or #3 or #4 or #5 or #6#8 TG=ANIMAL not (TG=HUMAN andTG=ANIMAL)#9 #7 not #8#10 CLINICAL-TRIAL in PT#11 explode CLINICAL-TRIALS#12 (clin* near trial*) in TI#13 (clin* near trial*) in AB#14 (singl* or doubl* or trebl* or tripl*) near (blind*or mask*)#15 (#14 in TI) or (#14 in AB)#16 PLACEBOS#17 placebo* in TI#18 placebo* in AB#19 random* in TI#20 random* in AB

#21 RESEARCH-DESIGN#22 #10 or #11 or #12 or #13 or #15 or #16 or #17 or#18 or #19 or #20 or #21#23 TG=ANIMAL not (TG=HUMAN andTG=ANIMAL)#24 #22 not #23#25 #24 not #9#26 TG=COMPARATIVE-STUDY#27 explode EVALUATION-STUDIES#28 FOLLOW-UP-STUDIES#29 PROSPECTIVE-STUDIES#30 control* or prospectiv* or volunteer*#31 (#30 in TI) or (#30 in AB)#32 #26 or #27 or #28 or #29 or #31

#33 TG=ANIMAL not (TG=HUMAN andTG=ANIMAL)#34 #32 not #33#35 #34 not (#9 or #25)#36 #9 or #25 or #35

EBM process

1. Ask

2. Search

3. Appraise

4. Apply

5. Evaluate

EBM process

3-Critical appraisal of the evidence for its validity and clinical applicability

Level of evidence for treatmentLevel of evidence for treatment

A

B

C

D

E

SR

Why do I have to bother?Can’t I trust the editors?

Percent of articles meeting quality criteria

NEJM 12.6

Ann Int Med 7.6

JAMA 7.2

Lancet 6.2

BMJ 4.4

Arch Int Med 2.4

EBM process

1. Ask

2. Search

3. Appraise

4. Apply

5. Evaluate

EBM process4 Integration of that critical appraisal

with clinical expertise and the patient’s unique biology and beliefs action.

5 Evaluation: evaluating our effectiveness and efficiency in executing steps 1–4 and seeking ways to improve them both for next time.

Even fully EB-trained clinicians work in all 3 modes

“Searching & appraising” mode for the problems I encounter daily.

“Searching only” mode among E-B resources for problems I encounter once a month.

“Replicating” the practice of experts mode for problems I encounter once a decade .

Sackett et al, 2000

What Proportion of Healthcare is Evidence-Based ?

• BMJ Editorial: about 15%

• Archie Cochrane: less than 10%

• NIH : Diagnostic technology 20 %

Smith R: Where is the wisdom...? The poverty of medical evidence. BMJ 1991;303:798-9.

Barriers to EBP

11.1%Unaware of Scope of

Autonomous Nursing Practice

7%Research Not Valued

23.6%Lack of resources

16.4%Lack of skill accessing/assessing research

N=1033

41.8%Lack of Time

What are the Challenges

• time

• access

• skill in critical appraisal

• language of research

• sense of control over practice

• environment/culture

• applicability

Reference based medicine?

• First, idea second, references

• Vague questions

• systematic search is not usual

• systematic critical appraisal is not usual

• Inadequate evaluation

It’s like pseudoscience, isn’t it?

Reference based medicine?

Conventional medicine

Some times systematicin search, appraisalextraction of data..

Use pathophysiologic

approach

Agenda

• Definitions: Definitions: Science and EBMScience and EBM

• Dimensions of the problemDimensions of the problem1. Information management (mastery)

2.2. Limitations of current clinical practiceLimitations of current clinical practice

3.3. Heuristic and errorsHeuristic and errors

4. Problems of communication

• EBM

• Summary

Why Evidence-based Medicine?

• Physicians have daily need for valid info– 5 times for every inpatient– 2 times for every 3 outpatients

• Traditional sources of info:– Textbooks :partially (10-30%) outdated before publication– Experts: biased towards their own works & knowledge– CME: ineffective– Medical Journals: too voluminous– Scientific chaos

• Clinical judgment / diagnostic skills increase with time, but up-to-date clinical knowledge declines

• Time not available to find and assimilate evidence into practice• Resources are limited

Dr Naserimoghadam

Practicing EBM: New Developments

• New strategies for finding and evaluating evidence

• New tools:• Meta-analyses• Systematic reviews / the Cochrane Collaboration

• Evidence-based journals of secondary publication

• Information systems bring info in seconds

Dr Naserimoghadam

Evidence-Based JokeEvidence-Based JokeA

B

C

D

E

SR

Class0:Things I believe

Class0a:Things I believe despite the available data

Class1:Randomised controlled clinical trials that agree with what I believe

Class2:Other prospectively collected data that agree with what I believe

Class3:Expert opinion that agree with what I believe

Class4:Randomised controlled clinical trials that do not agree with what I believe

Class5:What you believe that I do not

Evidence-Based JokeEvidence-Based Joke

egocentrism/ narcissistic trait/personality

• Imagine your life and the lives of your friends and family placed in the hands of juries and judges who let their biases and stereotypes govern their decisions, who do not attend to the evidence, who are not interested in reasoned inquiry, who do not know how to draw an inference or evaluate one.

Critical Thinking:What It Is and Why It Counts

Peter A. Facione Dean of the College of Arts and Sciences Santa Clara University 1998

think about

Acknowledgment

• Dr Mortaz Hejri for her research and contribution to understand our situation in EBM in TUMS

• Dr Naserimoghadam , Dr Haynes RB, Dr Sackett DL, Guyatt GH for using their slides

• EMRC for financial support

References• Cook DJ, Meade MO, Fink MP: How to keep up with the

critical care literature and avoid being buried alive. Crit Care Med 24:1757-1768, 1996

• Evidence-Based Medicine: A Framework for Clinical Practice by Friedland et. al

• Evidence-Based Medicine: How to Practice and Teach EBM by David Sackett

• How to Read a Paper: The Basics of evidence based medicine by Trisha Greenhalgh

• Studying a Study and Testing a Test by Richard Riegelman and Robert Hirsch

• Smith R: Where is the wisdom...? The poverty of medical evidence. BMJ 1991;303:798-9.

References• Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of

results of meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992;268:240-8

• Bero L, Rennie D. The Cochrane Collaboration. JAMA 1995;274:1935–8. • Villanueva EV, Burrows EA, Fennessy PA, Rajendran M, Anderson JN. Improving

question formulation for use in evidence appraisal in a tertiary care setting: a randomised controlled trial. BMC Med Inform Decis Mak. 2001;1(1):4. Epub 2001 Nov 08.

• Booth A, O'Rourke AJ, Ford NJ. Structuring the pre-search reference interview: a useful technique for handling clinical questions. Bull Med Libr Assoc. 2000 Jul;88(3):239-46

• Haynes RB. Clinical review articles. BMJ. 1992;304:330-1.• Oxman AD, Cook DJ, Guyatt GH. Users’ guides to the medical literature. VI. How

to use an overview. Evidence-Based Medicine Working Group. JAMA. 1994;272:1367-71.

• Evidence-based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA 1992;268:2420-5

• Guyatt GH, Rennie D. Users' guides to the medical literature. JAMA 1993;270:2096-7

• Oxman AD, Sackett DL, Guyatt GH. Users' guides to the medical literature. I. How to get started. JAMA 1993;270:2093-5.

• Evidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)

Thank you