Cochrane Methods Symposium 2 October 2015
Transcript of Cochrane Methods Symposium 2 October 2015
![Page 1: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/1.jpg)
Cochrane Methods Symposium
2 October 2015
'Summary of findings' Tables: how are they being evaluated?
Elie Akl
American University of Beirut, Lebanon
![Page 2: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/2.jpg)
Disclosure
• No financial conflict of interest
• Contributed to some of the studies discussed
• Member of the GRADE working group
• Cochrane reviewer
![Page 3: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/3.jpg)
GRADE tables
• GRADE tables summarize quality of evidence and main findings by outcomes
• GRADE tables:
– Summary of Findings (SoF)
– Evidence Profiles (EP)
![Page 4: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/4.jpg)
![Page 5: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/5.jpg)
MECIR standards
• It is ‘highly desirable’ to include a Summary of Findings (SoF) in a Cochrane systematic review to present the statistical results and the quality of evidence for the most important outcomes.
![Page 6: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/6.jpg)
![Page 7: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/7.jpg)
![Page 8: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/8.jpg)
MIF SoF trial
• Two arm, parallel, non-inferiority RCT; conducted online
• Health professionals, clinical practice guidelines developers, and researchers (N=290)
• Compared current format to alternative format
• Assessed: understanding, accessibility, satisfation, and preference
![Page 9: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/9.jpg)
Email invitation to participate
Questions about baseline information and background
Clinician
R R
Table A Table B
Understanding
Accessibility to information
Satisfaction
Understanding
Accessibility to information
Satisfaction
PreferencePreference
Guideline developer Researcher
R
Table A Table B Table A Table B
Table B Table A Table B Table A Table B Table A
Understanding
Accessibility to information
Satisfaction
Preference
Disclosure of the other table Disclosure of the other table Disclosure of the other table
![Page 10: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/10.jpg)
![Page 11: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/11.jpg)
Outcome: Understanding
Table5.PercentageofparticipantsthatansweredcorrectlyunderstandingquestionsConcept
Questionasked
Alternativeformats
(N=122)
Currentformats
(N=168)
Difference
Pvalue
Abilitytointerpretfootnotes
Fortheoutcomeadverseevents,whyisthequalityofevidenceratedaslow?
89% 82% 7% 0.18
Abilitytointerpretrisk Willfewerchildren<5yearsoldhavediarrheaiftheytaketheprobiotics?
96% 96% 0% 0.99
Abilitytodetermineriskdifference
Howmanyfewerchildren<5yearswillhavediarrheaiftheyhaveprobioticsthaniftheydonot?
98% 35% 63% <0.001
Understandingofqualityofevidenceandtreatmenteffect
Whichofthefollowingstatementsbestrepresentstheresultsinformingtheoutcomeadverseevents?
88% 26% 62% <0.001
Understandingofqualityofevidence
Inchildren<5yearsold,whatresultismostcertain?
97% 90% 7% 0.06
AbilitytorelateNºofparticipant/studiesandoutcomes
Howmanyparticipantsandstudiesareinformingtheoutcomeadverseevents?
99% 99% 0% 1.00
Abilitytoquantifyrisk Inchildren>5yearsold,howmanyfewerormorechildrenwillhavediarrheaiftheytookprobioticsasanadjuncttoantibiotics
comparedtothosewhodidnottakeprobiotics?
94% 88% 6% 0.06
![Page 12: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/12.jpg)
0 10 20 30 40-10
1. Ability to interpret footnotes and reasons for downgrading
quality of evidence
2. Ability to interpret risks – overall treatment effect
interpretation
3. Ability to determine a risk difference
4. Understanding of quality of the evidence and treatment
effect
5. Understanding of quality of evidence implications
6. Ability to relate Nº of participant/ studies and outcomes
7. Ability to quantify risks
(-2, 15)
Current format better Alternative format better
Difference in proportion of correct answers
Fig 1. Confidence intervals and non-inferiority margin for the outcome understanding disaggregated at a question level
50 60 60
(-5, 5)
(55, 71)
(52, 71)
(0, 12)
(-8, 2)
(0, 13)
![Page 13: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/13.jpg)
Outcome: Satisfaction
C
A
A
A
A
A
C: current formats; A: Alternative formats
![Page 14: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/14.jpg)
Additional findings
• Alternative formats were overall more accessible to users than the current formats
• Systematic review users preferred alternative formats of SoF tables to current ones
![Page 15: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/15.jpg)
Challenges
• Methodological (multiple testing, online research)
• Evaluating different presentation elements at the same time
• Are the outcomes good enough?
• Presentation formats are evolving (e.g., interactive information, new platforms)
![Page 16: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/16.jpg)
Acknowledgments
• Cochrane Methods Innovation Fund and McMaster University funded this study
• Alonso Carrasco-Labra for sharing slides and leading the MIF SoF trial
![Page 17: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/17.jpg)
Thank you!
Questions or comments?
![Page 18: Cochrane Methods Symposium 2 October 2015](https://reader034.fdocuments.in/reader034/viewer/2022052004/6285c4248848c025a80886c5/html5/thumbnails/18.jpg)
Table 1. Comparison between items included in the current and alternative SoF tables
formats
Current formats (Table B)
Alternative formats (Table A)
1 Inclusion of the Nº of participants and studies
column
Exclusion of the Nº of participants and studies
column. Information presented in the outcomes
column
2 Quality of evidence presented with symbols
and labeled as High, moderate, low, or very
low. Reasons for downgrading presented in
the footnotes
Quality of evidence presented with main
reasons for downgrading in the same column
(e.g. MODERATE due to imprecision)
3 “Footnotes” label “Explanations” label
4 Baseline risk and corresponding risk
expressed as natural frequencies
Baseline risk and corresponding risk expressed
as percentages
5 No column specific column presenting
absolute risk reduction (risk difference) or
mean difference
Inclusion of a column presenting absolute risk
reduction (risk difference) expressed as
percentage for benefit and harm or mean
difference
6 Comments column included Comments column deleted
7 No “what happens” column* “What happens” column included*
8 Description of the GRADE Working Group
grades of evidence definitions below the
table
No description of the GRADE Working Group
grades of evidence definitions