Introduction to Critical Appraisal
description
Transcript of Introduction to Critical Appraisal
11
Introduction to Introduction to Critical AppraisalCritical Appraisal
Yulia Lin, MD, FRCPCYulia Lin, MD, FRCPC
Transfusion Medicine Specialist, Dept of Clinical PathologyTransfusion Medicine Specialist, Dept of Clinical Pathology
Sunnybrook Health Sciences CentreSunnybrook Health Sciences Centre
University of TorontoUniversity of Toronto
TMR Journal Club, September 4, 2008TMR Journal Club, September 4, 2008
22
Why is Critical Appraisal Why is Critical Appraisal important?important?
33
Vast amount of literatureVast amount of literature
44
New Studies inTransfusion
55
New Studies inTransfusion
Past Studies in Transfusion
66
New Studies inTransfusion
Past Studies in TransfusionStudies in other Sub-specialties
77
Why critical appraisal is important?Why critical appraisal is important?
Impossible to keep up with all of the Impossible to keep up with all of the literatureliterature
Enables us to distinguish stronger Enables us to distinguish stronger evidence from weaker evidenceevidence from weaker evidence
Allows us to appropriately incorporate the Allows us to appropriately incorporate the evidence into our practice so as to evidence into our practice so as to improve patient careimprove patient care
88
Medical ExpertMedical Expert• Apply principles to evaluate quality of research Apply principles to evaluate quality of research
publicationspublications ScholarScholar
• Critically appraise sources of medical informationCritically appraise sources of medical information• Understand the concepts of clinical research design Understand the concepts of clinical research design • Demonstrate knowledge of basic statistics and Demonstrate knowledge of basic statistics and
epidemiologyepidemiology
99
Purpose of TMR Journal ClubPurpose of TMR Journal Club
1.1. To present a recent article in the To present a recent article in the transfusion literaturetransfusion literature
2.2. To learn how to critically appraise the To learn how to critically appraise the medical literature medical literature
1010
Steps to a successful JCSteps to a successful JC
1.1. Choosing the articleChoosing the article
2.2. Presenting the articlePresenting the article
3.3. Critically appraising the articleCritically appraising the article
1111
Choosing an article…Choosing an article…
Where to find an article?Where to find an article?• Recent table of contents Recent table of contents
NEJM, Lancet, JAMANEJM, Lancet, JAMA Transfusion, Vox Sanguinis, Blood, etc.Transfusion, Vox Sanguinis, Blood, etc.
• Transfusion Medicine Reviews – Journal ClubTransfusion Medicine Reviews – Journal Club• Transfusion Medicine CommunityTransfusion Medicine Community• List of interesting articlesList of interesting articles
1212
Choosing the article…Choosing the article…
Which article?Which article?• RelevantRelevant• Interesting to youInteresting to you
What type of article?What type of article?• Any articleAny article• But some are easier than others…But some are easier than others…
1313
Type of articlesType of articles
Randomized controlled trials – TherapyRandomized controlled trials – Therapy Observational studies – HarmObservational studies – Harm Systematic ReviewsSystematic Reviews GuidelinesGuidelines
DiagnosisDiagnosis PrognosisPrognosis
1414
First, a review…First, a review…
1515
Randomized Controlled TrialRandomized Controlled Trial
Populationselection
Randomization
Intervention A
Intervention B
Outcomesmeasured
1616
Randomized Controlled TrialRandomized Controlled Trial
Population Randomization
Intervention A
Intervention B
Outcomesmeasured
Is aprotinin superior to lysine analogs in high-risk cardiac surgery?
Fergusson DA et al. NEJM May 29, 2008.
1717
Observational Case-ControlObservational Case-Control
Population
Cases(+Disease)
Control(No disease)
Exposure to Risk Factor
Exposed
Yes
No
Yes
No
1818
Observational Case-ControlObservational Case-Control
Population
Cases(+Disease)
Control(No disease)
Exposure to Risk Factor
Exposed
Yes
No
Yes
No
Retrospective
1919
Observational Case-ControlObservational Case-Control
Population
Cases(+Disease)
Control(No disease)
Exposure to Risk Factor
Exposed
Yes
No
Yes
No
Blood transfusion, anesthesia, surgery and risk of NHLin a population-based case-control study.Cerhan JR et al. Int J Cancer 2008 Aug 15.
2020
Observational CohortObservational Cohort
Cohort
Cohort without Disease
Exposure to Risk Factor
Population
Exposed
Not exposed
Disease
Yes
No
Yes
No
2121
Cohort StudiesCohort StudiesPast Present Future
Follow-upCohort assembled
RetrospectiveCohort Study
2222
Cohort StudiesCohort StudiesPast Present Future
Follow-upCohort assembled
RetrospectiveCohort Study
The ratio of fibrinogen to red cells transfused affects survival incasualties receiving massive transfusions
at an army combat support hospital.Stinger HK et al., J Trauma February 2008;64:S79-85.
Red blood cell to plasma ratios transfused during massive transfusion are associated with mortality in severe multiplyInjury: a retrospective analysis from Deutsche Gesellschaft.
Maegele M et al. Vox Sang August 2008;95:112-119.
2323
Cohort StudiesCohort StudiesPast Present Future
Follow-upCohort assembled
RetrospectiveCohort Study
Follow-upCohort assembled
ProspectiveCohort Study
2424
Cohort StudiesCohort StudiesPast Present Future
Follow-upCohort assembled
RetrospectiveCohort Study
Follow-upCohort assembled
ProspectiveCohort Study
Prospective evaluation of a transfusion policy of D+ red blood cells into D- patients.
Gonzalez-Porras JR et al. Transfusion July 2008.
2525
Systematic ReviewSystematic Review
OverviewOverview• Summary of the medical literature that Summary of the medical literature that
attempts to address a focused clinical questionattempts to address a focused clinical question Systematic reviewSystematic review
• Using methods designed to reduce the Using methods designed to reduce the likelihood of biaslikelihood of bias
Meta-analysisMeta-analysis• Review that uses quantitative methods to Review that uses quantitative methods to
summarize the resultssummarize the results
2626
Systematic ReviewSystematic Review
Systematic reviewSystematic review• Using methods designed to reduce the Using methods designed to reduce the
likelihood of biaslikelihood of bias Meta-analysisMeta-analysis
• Review that uses quantitative methods to Review that uses quantitative methods to summarize the resultssummarize the results
Comparing the efficacy and safety of apheresis and whole-blood derived platelet transfusions: a systematic review.
Heddle NM et al. Transfusion July 2008.
2727
GuidelinesGuidelines
““Systematically developed statements to Systematically developed statements to assist practitioner and patient decisions assist practitioner and patient decisions about appropriate health care for specific about appropriate health care for specific clinical circumstances”clinical circumstances”• Attempt to address all issues and values relevant Attempt to address all issues and values relevant
to a clinical decisionto a clinical decision• Attempt to distill a large body of medical Attempt to distill a large body of medical
expertise into a convenient, readily usable formatexpertise into a convenient, readily usable format• Make explicit recommendations with definite Make explicit recommendations with definite
intent to influence what clinicians dointent to influence what clinicians do
Field MJ. Clinical practice guidelines. 1990Hayward et al. JAMA 1995;274:570-4
Cook et al. Ann Int Med 1997;127:210-6
2828
GuidelinesGuidelines
““Systematically developed statements to Systematically developed statements to assist practitioner and patient decisions assist practitioner and patient decisions about appropriate health care for specific about appropriate health care for specific clinical circumstances”clinical circumstances”
Guidelines for policies on alternatives to allogeneic blood transfusion:1. Predeposit autologous blood donation and transfusionBoulton FE et al. Transfusion Medicine 2007;17:354-65
2929
How to do a critical appraisal?How to do a critical appraisal?
3030
Critical appraisal methodsCritical appraisal methods
Most are based on Users’ Guides to the Most are based on Users’ Guides to the Medical Literature Medical Literature • Available in a textbook form (2Available in a textbook form (2ndnd ed) but also ed) but also
on-line at on-line at http://ugi.usersguides.org/usersguides/hg/hh_sthttp://ugi.usersguides.org/usersguides/hg/hh_start.aspart.asp
• Centre for Health Evidence Website: Centre for Health Evidence Website: http://www.cche.net/usersguides/main.asphttp://www.cche.net/usersguides/main.asp
For guidelines, different instrumentsFor guidelines, different instruments• AGREE instrument: AGREE instrument:
http://www.agreetrust.org/instrument.htmhttp://www.agreetrust.org/instrument.htm• GRADE instrument available at Blood DepotGRADE instrument available at Blood Depot
3131
3232
3333
3434
Steps to Critical AppraisalSteps to Critical Appraisal
1.1. Are the results valid?Are the results valid?• Have the results been influenced in a Have the results been influenced in a
systematic fashion so as to lead to a false systematic fashion so as to lead to a false conclusion?conclusion?
2.2. What are the results?What are the results?
3.3. How can I apply the results to patient How can I apply the results to patient care? care?
3535
RCTRCT
Population Randomization
Intervention A
Intervention B
Outcomesmeasured
3636
RCT - Are the results valid?RCT - Are the results valid?
Population Randomization
Intervention A
Intervention B
Outcomesmeasured
Did experimental & control groups begin the study with a similar prognosis? Were patients randomized? Was randomization concealed? Were patients analyzed in the groups to which they were randomized? Were groups similar with respect to known prognostic variables?
3737
RCT - Are the results valid?RCT - Are the results valid?
Population Randomization
Intervention A
Intervention B
Outcomesmeasured
Did experimental & control groups retain a similar prognosis after the study started? Were patients, clinicians, outcome assessors blinded? Was follow-up complete?
3838
RCT - What were the results?RCT - What were the results?
Population Randomization
Intervention A
Intervention B
Outcomesmeasured
How large was the treatment effect?How precise was the estimate of the treatment effect?
3939
RCT - How can I apply the RCT - How can I apply the results to patient care?results to patient care?
Population Randomization
Intervention A
Intervention B
Outcomesmeasured
Were the study patients similar to the patient in my practice?
4040
RCT - How can I apply the RCT - How can I apply the results to patient care?results to patient care?
Population Randomization
Intervention A
Intervention B
Outcomesmeasured
Were the study patients similar to the patient in my practice?Were all clinically important outcomes considered?Are the likely treatment benefits worth the potential harm and costs?
4141
Observational CohortObservational Cohort
Cohort
Cohort without Disease
Exposure to Risk Factor
Population
Exposed
Not exposed
Disease
Yes
No
Yes
No
4242
Cohort - Are the results valid?Cohort - Are the results valid?
Cohort
Cohort without Disease
Exposure to Risk Factor
Population
Exposed
Not exposed
Disease
Yes
No
Yes
No
Did expt and control groups begin the study with a similar prognosis? Did the investigators demonstrate similarity in all known determinants of outcome? Did they adjust for differences in the analysis?
4343
Cohort - Are the results valid?Cohort - Are the results valid?
Cohort
Cohort without Disease
Exposure to Risk Factor
Population
Exposed
Not exposed
Disease
Yes
No
Yes
No
Did expt and control groups retain a similar prognosis after the study started? Were the outcomes measured in the same way in the groups being compared? Was follow-up sufficiently complete?
4444
Cohort - What are the results?Cohort - What are the results?
Cohort
Cohort without Disease
Exposure to Risk Factor
Population
Exposed
Not exposed
Disease
Yes
No
Yes
No
How strong is the association between exposure and outcome? Consider magnitude and dose response?How precise is the estimate of the risk?
4545
Cohort - How can I apply the results?Cohort - How can I apply the results?
Cohort
Cohort without Disease
Exposure to Risk Factor
Population
Exposed
Not exposed
Disease
Yes
No
Yes
No
Were the study patients similar to my patients?
4646
Cohort - How can I apply the results?Cohort - How can I apply the results?
Cohort
Cohort without Disease
Exposure to Risk Factor
Population
Exposed
Not exposed
Disease
Yes
No
Yes
No
Were the study patients similar to my patients?Was the duration of follow-up adequate?Was was the magnitude of the risk?Should I attempt to stop the exposure?
4747
Putting it all together…Putting it all together…
4848
Tips for TMR Journal ClubTips for TMR Journal Club
Choose the article carefully Choose the article carefully Present the article briefly (10 slides)Present the article briefly (10 slides) The main objective is the critical appraisalThe main objective is the critical appraisal
• Internal validity: Internal validity: Focus on 1-2 key pointsFocus on 1-2 key points Try explaining a new concept (bias, Try explaining a new concept (bias,
statistical concept)statistical concept)• External validity: Take a standExternal validity: Take a stand
Allow ½ the time for discussionAllow ½ the time for discussion