EBM: Critical Appraisal of an Article on Therapy

Journal Club: Critical Appraisal of an Article on Prognosis

Sudan Evidence-Based Medicine Association TemplateDeveloped by:

Dr. Imad Salah Ahmed Hassan MD (UK) FACP FRCPI MemAcadMEd (UK) MSc (UK) MBBS (U of K)

2

Clinical Scenario

Formulate a Well-Built Clinical QuestionPatientInterventionComparisonOutcome

Outline your Search Strategy:Search Terms/Keywords/Mesh termsSearch site/Search EnginePubmed via PICOPubmed Search Box & LimitsPubmed Clinical QueriesPubmed Advanced SearchCochrane LibraryOther

5

Outline Current KnowledgeTip: From the Introduction Section or from a Guideline

6

Select a Reasonable Piece of Research

Tip:Paste the title, authors, journal details hereType of Study: RCT Retrospective Cohort Case-Control Other:Tip: Study Strength: RCT>cohort>case control>case series Date published:Number of Pages:Quality of journal i.e. impact factor (get from Google!)Linked-In: Author Details/Picture

Reason(s) for Selecting this Piece of ResearchTip: Common ReasonsAnswers my QUESTIONFree articleRecent ArticleReputable Journal Other

Summarize the Results of your ArticleTip: Paste the Abstract/Conclusion from the Abstract here

Paste Abstract here: use several slides if it is long-remember the 6X6 rule in power-point presentations!

9

My Appraisal MapAppropriateness of chosen Title?Trustworthy Authors?Truthful Abstract?Comprehensiveness of Introduction section?Relevance: Importance for PracticeValidity: Appropriate Methodology (Validity: Internal)?Results: Trustworthy Results?Applicability?Impact of Findings on Current Practice and Future Recommendations?

10

Apply Your Appraisal SchemeTip: Use the RVRA Scheme: Based on the National & Gulf Center for Evidence-based Health PracticeRelevanceValidityResultsApplicability

Apply Your Appraisal SchemeTip: Use the RVRA SchemeRelevancePersonal KnowledgeValidityMethods SectionResults (Impact)Results SectionApplicabilityPersonal Interpretation/Experience

Apply Your Appraisal SchemeRelevanceDoes the study address a common problem in your practice? Does the study address an important outcome to you or to your patient? (DOE vs. POEM) Assuming that the study conclusion is true; would it lead to a change in your practice?

13

Apply Your Appraisal SchemeValidityDescribe the Research DesignTip: Name the Study Design

Apply Your Appraisal SchemeValidityRecruitment: Was the defined representative sample of patients assembled at a common (usually early) point in the course of their disease i.e. homogeneous sample?Tip:Selection and Sampling: )-----Referral Bias: Primary vs Tertiary Centres (Severity), Inception Bias/ Lead-time Bias: Timing of Inclusion/Homogeneous SampleFind me in the Methods Section: Selection of PatientsCopy the Inclusion & Exclusion CriteriaCopy the Study Flow Chart to your Presentation

Apply Your Appraisal SchemeValidityCopy the Inclusion & Exclusion Criteria

Inclusion Criteria:Exclusion Criteria:

Apply Your Appraisal SchemeValidityMaintenance: Was the follow-up of these patients sufficiently long and complete?Follow-up Attrition Bias: >20% loss of follow-up: Non-dependable Results will vary depending on the outcome (e.g., for pregnancy outcomes, nine months; for cancer, many years)Find me in the Results Section.

Apply Your Appraisal SchemeValidityOutcome Measures: Primary & SecondaryDetection Bias (Failure of Blinding): Objective (Death, amputation, infection rates etc.) or strictly defined Subjective outcomes (quality of life, pain, dependence) PLUS Blinding (to clinical and prognostic characteristics of the patients) Find me in the Methods or Results Section.

Primary:Secondary:

Apply Your Appraisal SchemeValidityMeasurement (Objective & Blinded): Were objective and unbiased outcome criteria clearly defined? Consider: Did the individual assessing the outcome criteria know whether or not the patient had a potential prognostic factor, i.e. were they blinded?

Apply Your Appraisal SchemeValidityAdjustment: Was there adjustment for important prognostic factors?Consider: Was there standardization for potentially important prognostic factors e.g. age? Were different sub-groups compared?

Apply Your Appraisal SchemeValidityValidation: Was there validation in an independent group ("test-set") of patients?Was there validation in an independent group of patients?Adjustment Bias: If the patient groups are clinically dissimilar. ?Regression AnalysisFind me in the Methods or Results Section.Test vs Development Group in Adjustment BiasThe most effective way to do this is to create a second group, called the test group, to analyze the results, that is different from the first group (the development group) and that is used for determining the factors in question.This eliminates the possibility that chance was responsible for the initial identification of prognostic factors. If this second study effectively validates the findings of the first, then we have a clinical prediction guide (CPG) of considerable practical use.

Apply Your Appraisal SchemeValidityStatistical Tests (Appropriateness)

Validity Overall, is the method of the study valid? [ ] Yes (100%) [ ] No (0%) [ ] In-between ( ___%) [ ] Can't Tell

ResultsResults clinical significance (in MP : Magnitude & Precision) Magnitude of the effect: How likely are the outcomes over time?

24

ResultsResults clinical significance (in MP : Magnitude & Precision) Precision of the effect: How precise was the estimate of the treatment effect? Confidence intervals? Narrow or WideStatistical vs. Clinical Significance

25

Summarize the Study BiasesSelection, Attrition, Performance, Detection Biases, Conflict of Interest etc.

Summarize the Study StrengthTip: From the Discussion or Letter to the Editor/Editorial Comment etc.E.G.: Number of Patients, Randomization Method, Least Biases, Statistical Tests used, Outcome Measures, Ascertainment, Drop-outs, Length of Follow-up etc.

Summarize the Study WeaknessesTip: From the Discussion or Letter to the Editor/Editorial Comment etc.E.G.: Number of Patients, Randomization Method, Least Biases, Statistical Tests used, Outcome Measures, Ascertainment, Drop-outs, Length of Follow-up etc.

28

Summarize the StudyTip: From the Discussion or Letter to the Editor/Editorial Comment etc.

29

Unanswered Questions from the StudyTip: From the Discussion or Letter to the Editor/Editorial Comment etc.

ApplicabilityApplicability I (in IPPP: Intervention Population - Preferences-Potential Benefits - Harm

32

Resolution of your PICO Question

Narrate/Write your PICO Question & Why is it important.Summarize the Study FindingsSummarize the weaknesses, biases, Survival, Prediction Rule etc.Indicate the Level of Evidence & whether you are going to apply the findings to your patient:

33

Resolution of your PICO Question: Recommendations

New Research: Is there a need to do more research of the subject? Y N Suggested research if the answer is yes:Audit: Is there a need to audit your practice? Y N Suggest an audit cycle.Publication: Is it worth publishing your Journal Club presentation in an EBM CAT journal or website? May be start your own JC website?

34

Open the Floor for Discussion

Useful LinksEBM Glossary of Terms: http://www.cebm.net/glossary/EBM Calculators: http://ktclearinghouse.ca/cebm/practise/ca/calculatorsEBM NNT, LR etc. Websites:http://ktclearinghouse.ca/cebm/toolboxhttp://www.thennt.com/EBM Critical Appraisal Worksheets: http://ktclearinghouse.ca/cebm/practise/ca/worksheetsDiagnosis worksheetHarm worksheetPrognosis worksheetSystematic review (of therapy) worksheetTherapy worksheet

36

37

Applicability (External Validity): Can the results be applied to my patients?

Guide

Tip

Answer

Were the study patients similar to this patient?Article should list the patients important clinical characteristics, along with the definitions used for these characteristics

Will the results lead directly to selecting or avoiding a treatment? Prognostic data often provide the basis for sensible decisions about therapy

Are the results useful for reassuring or counseling my patient?Will the evidence make a clinically important impact on your conclusions about what to offer or tell this patient?Uniformly good prognosis ReassuringUniformly poor prognosis - Counseling

Can the results be used in my clinical practice?

Availability of diagnostic tests, treatments, Cost etc.

Additional Points