7/30/2019 ebm handbook
1/52
EVIDENCE-BASED MEDICINE
USydMP HANDBOOK
Compiled by:
Evidence-Based Medicine Resource GroupUniversity of Sydney Medical Program (5th Ed.)
7/30/2019 ebm handbook
2/52
1
Contents
Evidence-based Medicine Resource Group Objectives......................................................... 2Checklist of Evidence-based Medicine Skills......................................................................... 5Evidence-based Medicine Course Overview ......................................................................... 7
What is EBM and why do we need it? ..................................................................................................................... 7How EBM is taught in the USydMP.......................................................................................................................... 8
Assessment of EBM................................................................................................................................................11Evidence-based Medicine Glossary .................................................................................... 13
A humorous look at EBM .................................................................................................... 18Asking the right questions and literature searching ............................................................. 19Study types......................................................................................................................... 30Causality............................................................................................................................. 33Frequency and Prognosis ................................................................................................... 35Intervention studies............................................................................................................. 39Diagnostic tests .................................................................................................................. 43Meta-analysis and systematic reviews ................................................................................ 49
Additional resources ........................................................................................................... 51
7/30/2019 ebm handbook
3/52
2
Evidence-based Medicine Resource Group Objectives
The teaching objectives of the EBM Resource Group focus on the identification,
critical appraisal and application of evidence from clinical epidemiological research.
Students should have covered each objective at the end of the year indicated in
brackets. Students should find opportunities to revisit each objective in subsequent
EBM theme sessions and PBL discussions in years one and two, and to reinforce
and apply these objectives in the clinical years three and four.
Overall Goals of Evidence-Based Medicine
Our aim is that graduates make decisions about health problems on the basis of the
best available evidence.
As part of their role in health care, graduates should have the willingness and ability
to:
1. Define a problem in a way that can be addressed by research evidence in the
form of epidemiological data. (Year One)
2. Identify and critically appraise research evidence. (Years One and Two)
3. Integrate this evidence with the details and preferences of individual patients in
the form of a management plan and then determine how the success of the plan
will be assessed. (Years Three and Four)
4. Interpret information on the benefits and costs of health interventions. (Year
Four)
A. Critical Appraisal of the Evidence
To achieve the above aims, students will need to be able to critically appraise the
literature. This will include primary research studies, review articles, including
systematic reviews and meta-analyses (Years One and Two) and clinical practice or
management guidelines.
7/30/2019 ebm handbook
4/52
3
By the time of graduation, students should be able to critically appraise the literature
on the following:
interventions for treatment and prevention of health problems
diagnostic tests
screening programs
causes of health problems (aetiology)
natural history of health problems (prognosis)
prevalence and incidence of health problems
And be familiar with some introductory principles of decision analysis and economic
evaluations.
B. Skills Required For Critical Appraisal
The following skills are needed for critical appraisal:
a) Conduct computerised literature searches to identify relevant high quality
research evidence and information from Medline, other data bases (eg. Cochrane
Library) and Internet sites.
b) Identify the following study types and be able to explain their strengths and
weaknesses: randomised controlled trials, cohort studies, case-control studies
(population and hospital-based), cross-sectional analytic and other descriptive
studies.
c) Identify major sources of bias in medical research: selection bias, confounding,
measurement bias, lead time bias and length bias (studies of screening).
d) Calculate and interpret the following measures of disease frequency and effect:
prevalence, incidence, relative risk, absolute risk reduction (risk difference),
number needed to treat and odds ratio (interpret only).
e) Interpret confidence intervals and P values.
f) Explain the difference between statistical significance and clinical significance.
7/30/2019 ebm handbook
5/52
4
g) Understand and apply criteria for establishing causality in the study of health
problems.
h) Understand the importance of pre-test probability for the interpretation of test
results.
i) Calculate and interpret the following features of diagnostic tests: sensitivity,
specificity, likelihood ratios, post-test probability.
j) For economic evaluations, identify the study question and the viewpoint from
which costs and outcomes have been measured. Explain what is meant by
opportunity cost, the margin and discounting.
Note: By interpret we mean understand and be able to explain in your own words.
The critical appraisal enabling skills are concerned with understanding research and
using results. Only those students interested in actually doing research will need to
be able to perform tests.
C. Application of Evidence to Decision Making
By graduation students should be able to interpret the results from clinical and
population research and decide how to apply these results to individuals or groups of
people. This is introduced in years one and two as results from critically appraised
papers are considered in relation to the PBL problem of the week, and will be
reinforced in the clinical years.
7/30/2019 ebm handbook
6/52
5
Checklist of Evidence-based Medicine Skills
Critical appraisal of literature Comments
Interventions for treatment and prevention of health problems
Diagnostic tests
Screening programs
Causes of health problems (aetiology)
Natural history of health problems (prognosis)
Prevalence and incidence of health problems
Skill
1. Framing appropriate clinical questions
2. Literature searching
Medline
Cochrane Library Internet sites
3. Identify the following study types and explain their strengths andweaknesses
Randomised controlled trials
Cohort studies Case-control studies (population and hospital-based)
Cross-sectional (analytic and descriptive)
7/30/2019 ebm handbook
7/52
6
4. Calculate and interpret measures of disease frequency Incidence Prevalence
5. Calculate and interpret measures of treatment effect Relative risk
Absolute risk reduction (risk difference)
Odds ratio (interpret only)
Number needed to treat
6. Interpret confidence intervals and P-values
7. Explain the difference between statistical significance and clinicalsignificance
8. Understand and apply criteria for establishing causality in the study ofhealth problems
9. Understand the importance of pre-test probability for the interpretation oftest results
10. Calculate and interpret the following features of diagnostic tests:
Sensitivity Specificity
Likelihood ratios
Post-test probability
11. For economic evaluations, identify the study question and the viewpoint
from which costs and outcomes have been measured. Explain what ismeant by:
Opportunity cost
Margin
Discounting
7/30/2019 ebm handbook
8/52
7
Evidence-based Medicine Course Overview
What is EBM and why do we need it?
Evidence-based Medicine is the conscientious, explicit and judicious use of current
best evidence in making decisions about the care of individual patients.1
By best evidence we mean patient-centred clinical research into (for example) the
accuracy and precision of diagnostic tests (including the clinical examination), the power
of prognostic markers and the efficacy and safety of therapeutic and preventive
interventions.
The practice of EBM entails a process of life-long self-directed learning in which caring
for patients creates the need for clinically important information and we then
Convert these information needs into answerable questions
Track down efficiently the best evidence with which to answer them
Critically appraise that evidence for its validity (closeness to truth) and usefulness
Apply the results to patient care.
EBM is new and rapidly growing. There are many reasons why EBM has developednow. These include:
The explosion of medical information. There are over 2 million biomedical papers
published every year -- textbooks can become out-of-date before they are
available in bookstores.
Patients have much greater access to information (eg on the Internet) and
patients are becoming increasingly aware of their rights and responsibilities as
consumers of health services.
1Sackett DL et al. Evidence-based medicine: what it is and what it isnt. BMJ 1996;312:71-72.
http://opac.library.usyd.edu.au/record=b3195783~S47/30/2019 ebm handbook
9/52
8
How EBM is taught in the USydMP
Evidence-based Medicine is part of the Personal and Professional Development
Theme. It is taught in all four years of the USydMP. In Years 1 and 2, students
acquire basic skills in EBM (see EBM Objectives). In Years 3 and 4, these skillsare refined and applied in clinical practice.
In Years 1 and 2, EBM is part of the integrated learning in PBL tutorials.
Students are progressively encouraged to consider the kinds of questions
patients might ask of their health care provider in each case, to search and to
critically appraise the research literature for evidence on which to base answers
to these questions. Students are particularly referred to the Cochrane Database
of Systematic Reviews and Best Evidence. In general, EBM encourages students
to think critically about their learning and to consider the evidence base for
clinical decisions.
EBM teaching in Years 1 and 2 is supplemented by EBM theme sessions.
There are two EBM theme sessions in each block. The first session of each pair
is a large group session in which there will be a presentation of EBM concepts
and skills. The second session of each pair is a small group session in which
students practice the skills demonstrated in the previous session. The small
group sessions are facilitated by tutors with high-level EBM skills and students
are expected to spend some time in preparation before the session. Over time,
there is an increasing expectation on students to think of relevant clinical
questions, to search the literature, and bring worthy papers for appraisal and
discussion in their small group sessions.
In between the formal EBM theme sessions, skills in EBM are reinforced and
consolidated by participation in EBM online activities. There are one to three
EBM activities in each block, each of which is linked to a specific PBL case. The
activities are self-contained, computer based activities and often involve framing
searchable clinical questions followed by a computerized literature search for
7/30/2019 ebm handbook
10/52
9
relevant articles. Participation in the activities is prompted by PBL tutors.
EBM Theme Sessions Years 1 and 2
Block 1 Asking Questions I Large Group SessionAsking Questions II Online Tutorial
Block 2 Study Designs I Large Group SessionStudy Designs II Small Group Tutorial*
Block 3 Causality I Large Group SessionCausality II Small Group Tutorial*
Block 4 Clinical Schools
Block 5 Frequency and Prognosis I Large Group SessionFrequency and Prognosis II Small Group Tutorial*
Block 6 Intervention Studies I Large Group SessionIntervention Studies II Small Group Tutorial*
Block 7 Diagnostic Tests I Large Group SessionDiagnostic Tests II Small Group Tutorial*
Block 8 Meta-analysis and
Systematic Reviews I
Large Group Session
Meta-analysis andSystematic Reviews II
Small Group Tutorial*
Block 9 Clinical Schools
*Indicates that student preparation is required for the session.
7/30/2019 ebm handbook
11/52
10
In years 3 and 4, the EBM curriculum is integrated with the clinical attachments
that make up the medical program. The aim is for students to apply the results of
clinical research to the problems of individuals. Each clinical attachment has its
own integrated EBM activity tailored to its characteristics and timing. The EBM
activities are supported by clinicians with high-level EBM and content expertise.
The clinical attachments and EBM activities are summarized in the tables below.
Details of each activity are available on the GMP website.
Attachment Activity Aim AssessmentIntegrated ClinicalAttachments(ICA)
Student PEARLS To experience using the tools of EBM toanswer real clinical questions
Summative rating ofpresentation + reporton audience feedback
PsychologicalMedicine
(PM)
Journal Club To enhance wider understanding of theevidence on which the practice of
psychological medicine is based.
Participation.Required formative
assessment
Children &Adolescents Health(CA)
Integration inPBL
To incorporate high quality evidence fromguidelines and systematic reviews inmanagement discussions with patients andtheir families.
Participation.Integration in exams.
Perinatal & WomensHealth(PW)
Integration inPBL
To integrate and apply information fromdifferent sources into clinical decision-making.
ParticipationIntegration in exams.
Community Practice(Comm)
CommunityPEARLS - casecommentaries
To apply research findings from groups to theproblems of an individual.
Summative rating of1000 word report
Pre-Internship
(PRINT)
Pre-Intern
PEARLS
To apply the principles of EBM within the
constraints of routine clinical practice
Patient rating of
discussion.Required formativeassessment.
Student PEARLS is the model used in the integrated clinical attachments of
GMP3. Students prepare and give a short, evidence-based presentation
addressing a focused clinical question raised by contact with a specific patient
seen during their integrated clinical attachments. Preparation of the presentation
is facilitated by a series of3 tutorials that the students attend in the preceding 3
weeks supported by a clinician with expertise in EBM. Each PEARLS
presentation lasts 15 minutes with an additional 15 minutes for questions and
feedback.
7/30/2019 ebm handbook
12/52
11
A similar model is used in year 4, except that students develop their
presentations more independently. PEARLS in Psychological Medicine uses a
journal club format. PEARLS in the Community Practice Rotation and Pre-
Internship focus on explaining evidence from research to patients. EBM in
Children & Adolescents Health and Perinatal & Womens Health focuses on
integrating evidence in decision making.
Specific objectives and their corresponding clinical attachments(see appropriate column)
Rotating PracticePlacementsStudents will demonstrate their ability to ICA
Comm CA PW PM
1. Translate real people's problems into answerableclinical questions
2. Find, appraise, and interpret pertinent clinicalresearch
3. Discuss the generalisability of research to clinical
practice
4. Discuss the applicability of research to an individual
5. Discuss evidence with patients, relatives, doctors,
the public
Dr Pt Rel All Pub Pt
6. Consider the preferences of patients, relatives,doctors & the community
Dr Pt Rel All Pub Pt
7. Integrate clinical data, evidence and preferences in
decision making8. Appraise and use systematic reviews and evidence
based guidelines
9. Cope with clinical time constraints
10. Appraise and use imperfect evidence
11. Dealing with conflicts between evidence and
prejudice
(hash - all attachments; black - that attachment)
Assessment of EBM
EBM is a component of Personal and Professional Development. It is necessary
to achieve satisfactory performance in EBM in order to achieve satisfactory
performance in PPD and thus to be able to progress from Years 2 to 3,
Years 3 to 4 and to complete Year 4.
7/30/2019 ebm handbook
13/52
12
In Years 1 and 2, EBM is assessed summatively in a critical appraisal exercise
which is integrated into the year 2 summative assessment. Critical appraisal
exercises are also provided in the formative assessments. In addition there will
be EBM multiple choice questions (SBAs) in the block formative assessments
and in the summative assessment. EBM questions are also included in the on-
line voluntary formative assessments.
In Year 3, the Student PEARLS presentation is assessed summatively by
EBMedicos and formatively by fellow students. In Year 4, there are discrete
required formative assessments in Psychological Medicine (PEARLS
Presentation) and the Pre-Internship (written report); and a discrete summative
assessment in Community Practice (written report). EBM assessment isintegrated in the summative assessments at the end of the rotations in Children
and Adolescent Health, and in Perinatal and Womens Health.
Please see reference:
Sackett DL, Rosenberg WM, Gray J A M Haynes R B, Richardson W S.Evidence based medicine: what it is and what it isnt. BMJ 1996;312:71-72.
http://opac.library.usyd.edu.au/record=b3195783~S47/30/2019 ebm handbook
14/52
7/30/2019 ebm handbook
15/52
14
Cost-Utility Analysis converts effects into personal preferences (or utilities) and
describes how much it costs for some additional quality gain (e.g. cost per
additional quality-adjusted life-year, or QALY).
Crossover Study Design is the administration of two or more experimental
therapies one after the other in a specified or random order to the same group of
patients.
Cross-Sectional Study the observation of a defined population at a single point
in time or time interval. Exposure and outcome are determined simultaneously.
Decision Analysis is the application of explicit, quantitative methods to analyse
decisions under conditions of uncertainty.
Ecological Survey is based on aggregated data for some population as it exists
at some point or points in time; to investigate the relationship of an exposure to a
known or presumed risk factor for a specified outcome.
EER Experimental Event Rate: see Event Rate.
Event Rate is the proportion of patients in a group in whom an event is
observed. Thus, if out of 100 patients, the event is observed in 27, the event rate
is 0.27. Control Event Rate (CER) and Experimental Event Rate (EER) are
used to refer to this in control and experimental groups of patients
respectively.
Evidence-Based Health Care extends the application of the principles of
Evidence-Based Medicine (see below) to all professions associated with health
care, including purchasing and management.
7/30/2019 ebm handbook
16/52
15
Evidence-Based Medicine is the conscientious, explicit and judicious use of
current best evidence in integrating individual clinical expertise with the best
available external clinical evidence from systematic research. See also the article
on EBM: What it is and what it isnt.
Likelihood Ratio is the likelihood of a given test result in a patient with the target
disorder compared to the likelihood of the same result in a patient without that
disorder.
Meta-analysis is an overview which uses quantitative methods to summarise the
results.
N-of-1 Trials The patient undergoes pairs of treatment periods organised so that
one period involves the use of the experimental treatment and one period
involves the use of an alternate or placebo therapy. The patients and physician
are blinded, if possible, and outcomes are monitored. Treatment periods are
replicated until the clinician and patient are convinced that the treatments are
definitely different or definitely not different.
Negative Predictive Value (-PV) is the proportion of people with a negative test
who are free of disease. See also SpPins and SnNouts.
Number Needed to Treat (NNT) is the number of patients who need to be
treated to prevent one bad outcome. It is the inverse of the ARR: NNT = 1/ARR.
Odds are a ratio of events to non-events, e.g., if the event rate for a disease is
0.2 (20%), its non-event ratio is 0.8 (80%), then its odds are 0.2./0.8 = 0.25 (see
also Odds Ratio).
Odds Ratio is the odds of an experimental patient suffering an event relative to
the odds of a control patient.
7/30/2019 ebm handbook
17/52
16
Overview is a systematic review and summary of the medical literature.
Positive Predictive Value (+PV) is the proportion of people with a positive test
who have disease. Also called thepost-test probability of disease after a positive
test. See also SpPins and SnNouts.
Randomised Controlled Clinical Trial a group of patients is randomised into an
experimental group and a control group. These groups are followed up for the
variables/outcomes of interest.
Relative Risk Reduction (RRR) is the percent reduction in events in the treated
group event rate (EER) compared to the control group event rate (CER):
RRR = (CER EER)/CER*100
Risk Ratio is the ratio of risk in the treated group (EER) to the risk in the control
group (CER): RR = EER/CER. RR is used in randomised trials and cohort
studies.
Sensitivity is the proportion of people with disease who have a positive test. See
also SpPins and SnNouts.
SnNout when a sign/test has a high sensitivity, a negative result rules out the
diagnosis; e.g. the sensitivity of a history of ankle swelling for diagnosing ascites
is 92 per cent, therefore if a person does not have a history of ankle swelling, it is
highly unlikely that the person has ascites.
Specificity is the proportion of people free of a disease who have a negative
test.
7/30/2019 ebm handbook
18/52
17
SpPin when a sign/test has a high specificity, a positive result rules in the
diagnosis; e.g. the specificity of fluid wave for diagnosing ascites is 92 per cent.
Therefore, if a person has a fluid wave, it is highly likely that the person has
ascites.
Glossary taken from: http://www.cebm.net/glossary.asp
7/30/2019 ebm handbook
19/52
18
A humorous look at EBM
Please see references: Down End Research Group. Polythenia gravis: the downside of evidence
based medicine. BMJ 1995:311;1666-8. Isaacs D, Fitzgerald D. Seven alternatives to evidence based medicine.
BMJ 1999;319:1618.
http://opac.library.usyd.edu.au/record=b3009800~S4http://opac.library.usyd.edu.au/record=b3610267~S47/30/2019 ebm handbook
20/52
7/30/2019 ebm handbook
21/52
7/30/2019 ebm handbook
22/52
21
Literature Searching Strategies
Important note:
When searching for articles on treatment (intervention) start at Search A,
however when searching for articles on prognosis, aetiology or diagnosis start at
Search B. This is because at the present time the Cochrane Database of
Systematic Reviews only includes systematic reviews of interventions.
Advanced search:
The new Ovid gives you a choice of doing a basic search or an advanced search.
For the purposes of EBM searching it is more efficient to use the advanced
search feature which searches using subject headings. You will find that when
you use the basic search feature you will be given a large number of
unnecessary articles that can be time consuming to go through!
Demonstration search topic:
Is Glucosamine sulphate an effective agent in the short-term treatment of
osteoarthritis?
Structure of clinical question:
Population/Disease = People with osteoarthritis
Intervention (Study factor) = Glucosamine sulphate
Comparator (Often the control) = Placebo
Outcome (Outcome factor) = e.g pain, function
N.B. The words in bold in the above PICO format are the ones which you will use
to do your search. Note that most commonly only the population and intervention
are used to search and there is no need to put in terms such as placebo or any
outcome factor. This is because if you put in too many terms it will limit your
search too much.
7/30/2019 ebm handbook
23/52
7/30/2019 ebm handbook
24/52
23
3. Enter 1st term Osteoarthritis (population/disease). Medline will map your
term to a MeSH term (Medical Subject Heading).
4. Check the explode box to the right of osteoarthritis to widen your search
then click continue.
5. On the next page, do not choose any subheadings and click continue.
6. Enter 2nd term Glucosamine Sulphate (treatment/intervention). Once
again Medline will map your term to an appropriate MeSH term.
7. Check the explode box to the right ofGlucosamine and click continue.
8. Do not choose any subheadings. Click on continue.
9. Combine searches 1 and 2 by ticking the boxes next to them and clicking
AND.
10. Click on limits and then additional limits. Scroll down to publication
types and select meta analysis.
11. Click on limit a search.
An alternative to the last two steps is to type in meta analysis.pt. This will
select all articles referenced as a meta analysis in Medline. pt stands for
publication type. You can then combine this with your other results by using
AND.
Search C
If search B did not produce useful results, move on to search C in which you
limit your search with a Medline Clinical Queries filter.
Filter terms that limit the search differ for different clinical questions i.e.
diagnosis, aetiology, therapy, or prognosis. Emphasis can also be placed on
specificity, sensitivity, or optimization.
A specific filter will retrieve most relevant articles but will probably omit a few.
A sensitive filter will retrieve mostly relevant articles but will probably include
some less relevant ones as well.
7/30/2019 ebm handbook
25/52
7/30/2019 ebm handbook
26/52
25
Step by step instructions.
1. Steps 1-9 as above.
2. Click on limits, choose the EBM Reviews limit and click search.
PubMed Search
This search is an easy alternative to Search C, and is useful if you are off
campus and dont have access to the library databases. PubMed is the US
National Library of Medicines version of Medline and is available free of
charge worldwide.
Search PubMed Clinical Queries using keywords. The software automatically
combines search terms with AND. You do not need to type Boolean
connectors unless you wish to use the OR connector.
Step by step instructions:
1. Select PubMed from the Databases list or go to
http://www.ncbi.nlm.nih.gov/entrez
2. Click on Clinical Queries (on the blue sidebar).
3. UnderSearch byClinical Study Category select Therapy and narrow,
specific.
4. Enter keywords osteoarthritis glucosamine sulphate.
5. Click on Go.
The two tables over the page show you the search strategies that OVID Medline
uses in its clinical queries limits.
7/30/2019 ebm handbook
27/52
7/30/2019 ebm handbook
28/52
27
Question type Best study type Optimised search Most specific search More sensitive search
Aetiology
Cohort study
Case controlstudy
risk.mp
ORmortality.mp
ORcohort.tw
relative risk$.tw
OR
risks.tw
ORcohort stud$.mp
risk$.mp
ORexp cohort studies
ORbetween group$.tw
PrognosisCohort study
Prognosis/ORdiagnosed.twORCohort$.mpORpredictor$.twORdeath.twORexp models, statistical
prognos$.tw
OR
first episode.tw
OR
cohort.tw
Incidence/ORexp mortalityORfollow-up studies/ORMortality/ORPrognos$.twORpredict$.twORCourse$.tw
Filter terms on this table are taken from OVID Medline Clinical Queries and these are based on the work of R. Brian Haynes MD,PhD et al. of the Health Information Research Unit (HIRU) at McMaster University.http://www.ovid.com/site/products/ovidguide/haynes.htm
7/30/2019 ebm handbook
29/52
28
Bibliography
Haynes RB. Wilczynski NL. Optimal search strategies for retrieving scientifically
strong studies of diagnosis from Medline: analytical survey.BMJ2004:
328(7447):1040.Available from: http://www.bmj.com/cgi/content/full/328/7447/1040
Haynes RB. McKibbon KA. for the Hedges Team. Optimal search strategies for
retrieving scientifically strong studies of treatment from Medline: analytical survey.
BMJ2005: 330(7501):1179.Available from:
http://www.bmj.com/cgi/content/full/330/7501/1179
Wilczynski NL. Haynes RB for the Hedges Team. Developing optimal searchstrategies for detecting clinically sound prognostic studies in MEDLINE: an analytic
survey. BMC Med2004; (1):23. Available from:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=441418.
Wilczynski NL, Haynes RB for the Hedges Team. Developing optimal search
strategies for detecting clinically sound causation studies in MEDLINE. Proc AMIA
Annu Symp 2003:719-23.
Extra InformationMedline classes:
For students who do not feel confident searching Medline the Medical Library
offers hands-on tutorials. For more information and to register go to
http://www.library.usyd.edu.au/subjects/medicine/medIlitregistration.html
Online Tutorials:
Medline (Ovid version) tutorial:
http://www.library.usyd.edu.au/subjects/medicine/tutorials/medlinetut/index.html
EBM Literature Searching Guide:
http://www.library.usyd.edu.au/subjects/medicine/tutorials/ebmtut/index.html
7/30/2019 ebm handbook
30/52
29
Free on the Web:
PubMed
http://www.ncbi.nlm.nih.gov/entrez
Cochrane EBM Reviews databases
http://www.thecochranelibrary.com/
Link to other useful EBM sites:
http://www.library.usyd.edu.au/subjects/medicine/links/ebp.html
Centre for Evidence-Based Medicine
http://www.cebm.net/searching.asp
Netting the evidence (provides links to many useful websites)
http://www.shef.ac.uk/scharr/ir/netting/
PICO Tutorial
http://www.cwml-tutorials.blogspot.com/2008/01/formulating-your-questions-
using-pico.html
Many thanks to Dianne van Sommers, Jeremy Cullis and Kathy Thornton for their contribution to thischapter.
Please see references:
Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinicalquestion: a key to evidence-based decisions. [editorial] ACP Journal Club1995;123:A-12.
Oxman AD, Sackett DL, Guyatt GH. Users guides to the medical literature. I.How to get started. JAMA 1993;270:2093-2095.
http://opac.library.usyd.edu.au/record=b3213384~S4http://opac.library.usyd.edu.au/record=b3610293~S47/30/2019 ebm handbook
31/52
7/30/2019 ebm handbook
32/52
7/30/2019 ebm handbook
33/52
7/30/2019 ebm handbook
34/52
33
Causality
Objectives
i. Understand the difference between a cohort and a case-control study
ii. Interpret odds ratios
iii. Understand and apply criteria for establishing causality in the study of health
problems
Odds ratio
An odds ratio is the odds of a person with the disease being exposed divided by the
odds of a control (person without the disease) being exposed. The odds ratio is
usually a good approximation of the relative risk.
ie OR=RR
Interpreting odds ratios and relative risks
For questions of harm where the outcome is disease or death, an RR>1 means that
the risk factor is harmful and an RR
7/30/2019 ebm handbook
35/52
7/30/2019 ebm handbook
36/52
35
Frequency and Prognosis
Objectives
i. Interpret confidence intervals and P values
ii. Understand the difference between statistical significance and clinical
significance
iii. Critically appraise cross-sectional and cohort studies
Incidence
No. of persons experiencing a new event during a time period/No. of persons at risk
at beginning of time period.
Therefore, incidence can only be calculated by following a group of people over time
(i.e. in an RCT or cohort study).
Prevalence
Total no. of persons with attribute at a given time/Total population at a given time.
This can be assessed by a cross-sectional study but to be representative the study
participants need to be selected at random (not volunteers).
Cohort studies on prognosis do not need a control group as the study follows
diseased people over time to determine their outcome.
P Value
A P value refers to the statistical significance of the results. Another way of thinking
of it is that it refers to the probability that the observed results are due to chance
rather than an actual effect.
The role of chance in the results is assessed by what is known as the null
hypothesis. This is the hypothesis that there is no difference between the two
groups. The statistical test looks at the probability of getting the observed result (or
one even more extreme) by chance, if the truth is that the two groups are actually
7/30/2019 ebm handbook
37/52
7/30/2019 ebm handbook
38/52
37
How are 95% CIs and P values related?
If a confidence interval includes no difference then it means that the null hypothesis
(i.e. that there is no real difference between groups) is possible. As the P value is a
measure of the null hypothesis being disproved, this would mean that the results
would not be statistically significant when the CI includes no difference.
NB: When looking at results such as a relative risk which is a ratio, an RR of 1 is
equivalent to no difference between the two groups and therefore if the CI crosses 1
the results will not be statistically significant. However, if the results do not refer to a
ratio but subtraction (e.g. the difference in mean weight loss between 2 groups), then
a value of 0 would indicate no difference and hence a CI crossing 0 would not be
statistically significant.
What P values and CIs dont tell you
These two statistical measures do not tell you how believable the result is. If the
results come from a study that is heavily biased then even statistically significant
results could not be relied upon. This is why it is important to assess a study
according to the JAMA validity criteria before looking at the results.
Statistical significance also does not tell you anything about whether the effect is
important i.e. whether the results are clinically significant. For example, it is possible
that a study looking at a new antihypertensive medication discovers a statistically
significant effect but that the difference in mean BP between the treatment and
control group is only 0.5mmHg which obviously is not clinically important.
There is a further explanation of confidence intervals in the attached paper Primer
on 95% Confidence Intervals.
7/30/2019 ebm handbook
39/52
7/30/2019 ebm handbook
40/52
39
Intervention studies
Objectives
i. Calculate and interpret the following measures of disease frequency and effect:
prevalence, incidence, relative risk, absolute risk reduction (risk difference),
number needed to treat (NTT).
ii. Discuss potential sources of bias in randomised controlled trials
Allocation Concealment
Allocation concealment is a way of protecting the randomisation process from bias. If
allocation is concealed it means that the people involved in the study (investigators,
clinicians, patients) cannot identify which group the patient will be allocated to (i.e.
treatment or control) during randomisation and can therefore in no way influence the
group the patient will be put into. For example, a study might decide to allocate
patients to groups using non-opaque envelopes. Clinicians may hold the envelope
up to the light, see what the allocation is, decide that the allocation is not the best for
the patient (e.g. they are very sick so should get the treatment not placebo) and then
pick the next envelope. Obviously if this were to happen then the study would not be
truly randomised. Allocation concealment differs from blinding in that blinding refers
to lack of knowledge about which group patients are in after the randomisation
process.
Analysis by intention to treat
Analysing patient outcomes based on the group into which they were randomized
regardless of whether they actually received the planned intervention. This analysis
ensures randomization is preserved.
In the diagram below, the 8 patients lost to follow up (5 in the treatment group, 3 in
the control group) cannot be included in the analysis as they will have no available
data. The 95 left in the treatment group should all be analysed in the treatment group
7/30/2019 ebm handbook
41/52
7/30/2019 ebm handbook
42/52
41
Calculation of estimates of effect
Problem 1. Problem 2.
WHEN RELATIVE RISK1
Of the 750 people randomized to receive 17% of people exposed developed the
treatment, 350 people developed the outcome. outcome whereas only 5% of peopleOf the 800 people randomized to no treatment who had not been exposed developed the500 developed the outcome. outcome.
Experimental event rate =EER (incidence in exposed)=350/750=0.47
ie 47% of treated people experienced the outcomeMay be expressed as 47 events per 100
Experimental event rate =EER (incidence in exposed)=17%=0.17
ie 17%=17/100ie 17% of exposed people experienced the outcome (given)May be expresses as 17 events per 100 etc
Control event rate =CER (incidence in non-exposed)
=baseline risk=500/800=0.625
ie 63% of controls experienced the outcomeMay be expressed as 625 events per 1000 etc
Control event rate =CER (incidence in non-exposed)
=baseline risk=5%=0.05
ie 5% of controls experienced the outcome (given )May be expressed as 5 events per 100 etc
Relative Risk=RRRR =EER/CERRR =(350/750)/(500/800)
=0.467/0.625=0.75
From the study it is estimated that people who undergotreatment have 0.75 times the risk of the outcome compared tothose who did not receive the treatment. (ie treatment ISPROTECTIVE)OrIt is estimated that people who undergo the treatment havethree quarters of the risk of the outcome compared to thosewho did not receive the treatment.
OrIt is estimated that people who undergo the treatment have75% of the risk of the outcome compared to those who did notreceive the treatment
Relative Risk=RRRR =EER/CERRR =0.17/0.05
=3.33
From the study it is estimated that people who were exposedhave 3.33 times the risk of the outcome compared to thosenot exposed.OrIt is estimated that people who have been exposed havethree and a third times the risk of the outcome compared tothose not exposed.OrIt is estimated that people who were exposed have 333% of
the risk of the outcome compared to those not exposed.
Relative Risk Reduction=RRR (expressed as apercentage)RRR =(1-RR)x100
=(1-0.75)x100=0.25x100=25%
It is estimated that people who undergo the treatment have a25% relative reduction in the risk of the outcome compared tothose who did not receive the treatment.
Do not calculate RRR when RR>1 ie it is a relative riskincrease
Absolute Risk Reduction=ARR (or RD)May be expressed as a risk reduction or an event rate
reductionARR =CER-EER
=500/800-350/750=0.625-0.467=0.16
=16 events per 100
If a person undergoes treatment he reduces his absolute riskby 16 events per 100Treatment prevents 16 events per 100
When RR>1 better to change the name from Absoluterisk reduction to Risk Difference (RD)
RD =CER-EER=0.05-0.17= - 0.12
Take the absolute value and interpret it with respect to yourresults.
=12 events per 100
If a person is exposed she INCREASES her absolute risk by12 events per 100Exposure results in a risk difference of 12 events per 100
Number Needed to Treat=NNTNNT =1/ARR
=1/0.16
=6.37 (6.3) people need to be treated to prevent one outcome.
7/30/2019 ebm handbook
43/52
42
Please see references:
Guyatt GH, Sackett DL, Cook DJ. Users guides to the medical literature. II.How to use an article about therapy or prevention. A. Are the results of thestudy valid? JAMA 1993;270:2598-2601.
Guyatt GH, Sackett DL, Cook DJ. Users guides to the medical literature. II.
How to use an article about therapy or prevention. B. What were the resultsand will they help me in caring for my patients. JAMA 1994;271:59-63.
http://opac.library.usyd.edu.au/record=b2568418~S4http://opac.library.usyd.edu.au/record=b2574397~S47/30/2019 ebm handbook
44/52
43
Diagnostic tests
Objectives
i. Understand the importance of pre-test probability for the interpretation of test
results
ii. Calculate and interpret the following features of diagnostic tests:sensitivity,
specificity, likelihood ratios, pre-test probability, post-test probability.
Gold Standard
Disease
Yes No TOTAL
Diagnostic Test Positive True positive (TP) False positive (FP) TP + FP
Negative False negative (FN) True negative (TN) FN + TN
TOTAL TP + FN FP + TN
Sensitivity and Specificity of tests
Sensitivity is the proportion of all truly diseased persons (TP + FN) who test positiveusing the diagnostic test.
i.e. SN = TP/(TP+FN)
Specificity is the proportion of all truly non-diseased persons (TN + FP) who test
negative with diagnostic test.
i.e. SP = TN/(TN+FP)
Numerical example:
Gold Standard
Disease
Yes No TOTAL
Diagnostic Test Positive 90 20 110
Negative 10 80 90
TOTAL 100 100 200
7/30/2019 ebm handbook
45/52
44
SN = TP / (TP + FP)
= 90 / (90 + 10)
= 90%
i.e. of those with the disease 90% will test positive (and 10% will test negative)
SP = TN / (TN + FP)
= 80 / (80 + 20)
= 80%
i.e. of those without the disease 80% will test negative (and 20% will test positive)
Predictive values
Predictive values tell you the probability of the test result being correct.
Positive Predictive value=PPV
If you have a patient with a positive test result, this will tell you the probability of that
patient actually having the disease.
As PPV refers to the proportion of people testing positive that are true positives, it is
calculated by:
PPV = TP/(TP+FP)
Negative Predictive Value=NPV
If you have a patient with a negative test result, this will tell you the probability of that
patient actually not having the disease.
As NPV refers to the proportion of people testing negative that are true negatives, it
is calculated by:
NPV = TN/(TN+FN)
Numerical example:
If we look at the previous numerical example for sensitivity and specificity:
PPV = 90 / (90 + 20)
= 82%
7/30/2019 ebm handbook
46/52
45
i.e. of those with a positive test result 82% will have the disease
NPV = 80 / (80 + 10)
= 89%
i.e. of those with a negative test result 89% will not have the disease
Notice that in the above example the prevalence of disease is 50% (as 100 people
have the disease and 100 people do not have the disease). If we change the
prevalence of disease to 10%, we would get the following table (note that the
sensitivity and specificity of the test do not change).
Gold Standard
Disease
Yes No TOTAL
Diagnostic Test Positive 9 20 29
Negative 1 80 81
TOTAL 10 100 110
This would mean that the predictive values would now be:PPV = 9 / (9 + 20)
= 31%
NPV = 80 / (80 + 1)
= 98%
Notice how affected predictive values are by the prevalence of disease. So for
example, the probability of a test result being correct might be very different when
you conduct the test in a situation with low prevalence (e.g. screening test at a
shopping centre) compared to one with a high prevalence (e.g. specialist clinic).
7/30/2019 ebm handbook
47/52
7/30/2019 ebm handbook
48/52
7/30/2019 ebm handbook
49/52
48
Please see references:
Jaeschke R, Guyatt G, Sackett DL. Users guides to the medical literature. III.How to use an article about a diagnostic test. A. Are the results of the studyvalid? JAMA 1994;271:389-391.
Jaeschke R, Guyatt G, Sackett DL. Users guides to the medical literature. III.How to use an article about a diagnostic test. B. What are the results and willthey help me in caring for my patients? JAMA 1994;271:703-707.
http://opac.library.usyd.edu.au/record=b2601502~S4http://opac.library.usyd.edu.au/record=b2601501~S47/30/2019 ebm handbook
50/52
49
Meta-analysis and systematic reviews
In all cases systematic reviews of the stated best study type would be better than
any individual study.
Meta-analysis
Meta analysis refers to the quantitative methods for combining the results of several
studies into a single pooled or summary statistic.
Systematic reviewSystematic review is a critical evaluation of research that attempts to address a focused
clinical question using methods designed to reduce the likelihood of bias. It may include a
meta analysis (summary statistic).
Narrative review
Narrative Reviewis a good place to start when learning about a topic however it is subject to
substantial biases and limitations so is insufficient for clinical decision making.
Publication bias
Publication biasrefers to the preferential publication ofstudies with a statistically
significant positive result.
Duplication bias
Duplication biasrefers to the inclusion of the same study more than once in the
systematic review.
7/30/2019 ebm handbook
51/52
7/30/2019 ebm handbook
52/52
Additional resources
Some useful Internet sites
Cochrane Library: http://www.thecochranelibrary.com
PubMed: http://www.ncbi.nlm.nih.gov/entrez
Oxford Centre for Evidence Based Medicine: http://www.cebm.net/
Netting the Evidence:
http://www.shef.ac.uk/scharr/ir/netting/
Users guides to evidence based practice:http://www.cche.net/usersguides/main.asp
Sydney University Library links to useful EBM sites:http://www.library.usyd.edu.au/subjects/medicine/links/ebp.html
Duke University Introduction to EBM:
http://www.hsl.unc.edu/services/tutorials/ebm/welcome.htm
University of Colorado: A students guide to the medical literature:http://grinch.uchsc.edu/sg/ Some useful references
Evidence-based medicine: how to practice and teach EBM. 3rd edSharon E Strauss, W. Scott Richardson, Paul Glasziou and R Brian Haynes.Churchill Livingstone, 2005.
Evidence-based healthcare.
J.A. Muir Gray.Churchill Livingstone, 2001.
Clinical Epidemiology; a basic science for clinical medicine. 2nd ed
David Sackett et al.Little, Brown, 1991.
A Dictionary of Epidemiology. 5th ed.John Last.Oxford University Press, 2008
Clinical Epidemiology: the essentials. 4th ed.Robert Fletcher and Suzanne Fletcher.Williams and Wilkins, 2005
http://opac.library.usyd.edu.au/record=b2761934~S4http://opac.library.usyd.edu.au/record=b2860849~S4http://opac.library.usyd.edu.au/record=b3557114~S4http://opac.library.usyd.edu.au/record=b1695307~S4http://opac.library.usyd.edu.au/record=b2744845~S4http://opac.library.usyd.edu.au/record=b2863738~S4http://opac.library.usyd.edu.au/record=b2761934~S4