Critical Appraisal of Scientific Literature
Transcript of Critical Appraisal of Scientific Literature
Critical Appraisal of Scientific Literature
André Valdez, PhD
Stanford Health Care
Stanford University School of Medicine
Confidential – For Discussion Purposes Only
Learning Objectives
Understand the major components of a scientific article
Learn about some of the most common study designs and analyses
Become more familiar with literature appraisal tools
2
Confidential – For Discussion Purposes Only
Why Does This Matter?
Greater understanding of the literature = better advocacy for EBP
Increased literacy on a given clinical topic and gaps in knowledge
Increased awareness of a given study’s potential and its limitations
Grounding of clinical decision making in sound scientific reasoning
3
Confidential – For Discussion Purposes Only
Sections of a Scientific Article – Abstract
Introduction
Objectives/hypothesis
Methods
Results
Discussion/Conclusion
Two general types: structured and unstructured
− Structured contains headings for each major section of a study
− Unstructured does not contain headings
4
Confidential – For Discussion Purposes Only
Sections of a Scientific Article – Introduction
Helps the reader become familiar with the topic
Refers to recent literature and clearly maps out study’s rationale
Supports central argument with citations
What question(s) does the study attempt to answer?
Why is the study design appropriate?
5
(duPrel et al. 2009)
Confidential – For Discussion Purposes Only
Sections of a Scientific Article – Methods
Highlights data for appraisal of study’s validity
Describes all phases of planning, study sample, procedures, and statistical tests
Lays out study design, which is paramount
Precise description of inclusion/exclusion criteria
Thoroughly describes how data were collected, measurement scales of
variables, etc.
Sample size calculation
6
(duPrel et al. 2009)
Confidential – For Discussion Purposes Only
Sections of a Scientific Article – Results
Clear and objective presentation of findings
Summary statistics on all variables
Inferential statistics on variables of interest
Tables and figures to improve clarity
7
(duPrel et al. 2009)
Confidential – For Discussion Purposes Only
Sections of a Scientific Article – Discussion
Compares findings with what’s currently known
What has the study added to the existing body of knowledge?
What conclusions may the reader draw?
Do the interpretations follow logically from the results?
Analyzes study limitations
Statistical significance not the same as clinical relevance
8
(duPrel et al. 2009)
Confidential – For Discussion Purposes Only
Common Study Designs
Quality Improvement/Process/Performance Evaluation
− Set up to enhance clinical practice at a local level (not generalized)
− Often but not always quasi/non-experimental (no random assignment)
Uncontrolled pre-post comparison
Controlled pre-post comparison
Qualitative experience
Cost-effectiveness
Human Subject Research
− Set up to generate new knowledge (generalizable)
− Often but not always experimental (random assignment)
Randomized controlled trial
Quasi/non-experimental/observational
− Case-control
− Cohort
− Longitudinal
− Retrospective chart review
(Portela et al. 2015; Topics in Biostatistics 2007)
(Portela et al. 2015)
9
Confidential – For Discussion Purposes Only
Common Statistical Analyses
Variable scale: continuous (parametric: assumptions about data distribution)
− Independent-samples t-test
− Paired t-test
− Analysis of Variance
− Linear regression
Variable scale: categorical
− Chi-square test
− Logistic regression
Variable scale: ordinal (non-parametric: distribution free)
− Signed-rank test
− Rank-sum test
− Kruskal-Wallis test
(http://stattrek.com/)
10
Confidential – For Discussion Purposes Only
Literature Appraisal Tools – Johns Hopkins
11
Evidence Levels
Quality Guides
Level I Experimental study, randomized controlled trial (RCT) Systematic review of RCTs, with or without meta-analysis
A High quality: Consistent, generalizable results; sufficient sample size for the study design; adequate control; definitive conclusions; consistent recommendations based on comprehensive literature review that includes thorough reference to scientific evidence
B Good quality: Reasonably consistent results; sufficient sample size for
the study design; some control, fairly definitive conclusions; reasonably consistent recommendations based on fairly comprehensive literature review that includes some reference to scientific evidence
C Low quality or major flaws: Little evidence with inconsistent results;
insufficient sample size for the study design; conclusions cannot be drawn
Level II Quasi-experimental study Systematic review of a combination of RCTs and quasi-experimental, or quasi-experimental studies only, with or without meta-analysis Level III Non-experimental study Systematic review of a combination of RCTs, quasi-experimental and non-experimental studies, or non-experimental studies only, with or without meta-analysis Qualitative study or systematic review with or without a meta-synthesis
Evidence Levels & Quality Guides
(http://www.hopkinsmedicine.org/evidence-based-practice/jhn_ebp.html)
Confidential – For Discussion Purposes Only
Literature Appraisal Tools – Johns Hopkins
12
Article
# Author &
Date Evidence
Type
Sample, Sample Size &
Setting
Study findings that help answer the EBP question
Limitations Evidence Level & Quality
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Evidence Summary Table
(http://www.hopkinsmedicine.org/evidence-based-practice/jhn_ebp.html)
Confidential – For Discussion Purposes Only
Literature Appraisal Tools – STROBE
13
(http://www.strobe-statement.org/index.php?id=available-checklists)
Confidential – For Discussion Purposes Only
Literature Appraisal Tools – STROBE
14
✔
✔ ✔
✔
✔
✔
(http://www.strobe-statement.org/index.php?id=available-checklists)
Confidential – For Discussion Purposes Only
Literature Appraisal Tools – STROBE
15
✔
✔
✔
✔
(http://www.strobe-statement.org/index.php?id=available-checklists)
Confidential – For Discussion Purposes Only
Summary
Understanding and appraising scientific literature is crucial for effective EBP
Learning about study designs and analyses will help you interpret findings
Using appraisal tools is a good way to organize your literature review
Critical appraisal promotes critical thinking about your topic
16