Critical appraisal
-
Upload
paulafunnell -
Category
Health & Medicine
-
view
580 -
download
4
description
Transcript of Critical appraisal
![Page 1: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/1.jpg)
Critical appraisal
Paula [email protected]
Faculty Liaison Librarian (Medicine and Dentistry)
Introduction to
![Page 2: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/2.jpg)
Outline· Introduction to critical appraisal· Appraising quantitative research
Levels of evidence Statistics Group appraisal
· Appraising qualitative research Qualitative methodologies Group appraisal
· Running a critical appraisal workshop
![Page 3: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/3.jpg)
Introduction
![Page 4: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/4.jpg)
Evidence based healthcare
“When we intervene in the lives of others we should do so on the basis of the best evidence available regarding the likely consequences of that intervention”
G Macdonald, 1998
Macdonald, G. (1998). Promoting Evidence-Based Practice in Child Protection. Clinical Child Psychology and Psychiatry, 3 (1), 71–85.
![Page 5: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/5.jpg)
Steps of evidence based healthcare
1. The patient presents with a clinical problem
2. Formulate a focused research question
3. Search for the best evidence
4. Critically appraise the evidence you find
5. Consider the evidence in the light of your expertise and decide whether to apply it or not
![Page 6: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/6.jpg)
Why critically appraise?
To weigh up how valid and useful
the research will be
![Page 7: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/7.jpg)
Why critically appraise?
• In order to keep up to date, clinicians would have to read 17 articles a day, 365 days a year
• Research is of variable quality• Only an estimated 1% is
judged clinically relevant• Need to find the 1%
![Page 8: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/8.jpg)
How it works
• Involves answering a short questionnaire
• We use the CASP questionnaires at http://www.casp-uk.net/
• The questionnaires were devised by clinicians for clinicians
• Remember to focus on appraisal not criticism
![Page 9: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/9.jpg)
Quantitative research
![Page 10: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/10.jpg)
What is quantitative research?
• Research on something that can be accurately measured
• Tends to use large, statistically representative samples
• Uses statistical methods to analyse data
• Represents findings with numbers• Attempts to eliminate bias
![Page 11: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/11.jpg)
Systematic reviews
• A review of all the literature on a topic• Systematically identified• Appraised• Summarised
![Page 12: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/12.jpg)
Question 1
Did the review ask a clearly focused question?
![Page 13: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/13.jpg)
PICO
A focused clinical question:
P
I
C
O
Population / patient / problem
Intervention
Comparison
Outcome
![Page 14: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/14.jpg)
Question 2
Did the authors look for the right type of papers?
![Page 15: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/15.jpg)
Levels of evidence (therapy)
![Page 16: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/16.jpg)
Systematic reviews
RCTs
Cohort studies
Case control studies
Expert opinion
1a
1b
2
3
4
5
Case series
http://www.cebm.net/index.aspx?o=1025
Levels of evidence (therapy)
![Page 17: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/17.jpg)
Randomised controlled trials
Outcome
Outcome
new treatment
control treatment
![Page 18: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/18.jpg)
Question 3
Do you think the important relevant studies were included?
![Page 19: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/19.jpg)
Publication bias
Papers with “interesting” results are more likely to be:
• Submitted and accepted for publication• Published in a major journal • Published in English• Quoted by authors• Quoted in newspapers
![Page 20: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/20.jpg)
Question 4
Did the review’s authors do enough to assess the quality of
the included studies?
![Page 21: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/21.jpg)
RCT checklist
![Page 22: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/22.jpg)
Quality assessment
• Were the inclusion/exclusion criteria clear?
• Did they use any sort of scoring system?
• Were the studies for inclusion assessed by more than one reviewer?
![Page 23: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/23.jpg)
Question 5
If the results of the review have been combined, was it reasonable
to do so?
![Page 24: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/24.jpg)
Heterogeneity• Are the studies similar in terms of:
– Population– Intervention– Outcome– Methodology
• Are the results similar from study to study?
• Were any tests for heterogeneity carried out?
• Are there any discussions around heterogeneity?
![Page 25: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/25.jpg)
Question 6
What are the overall results of the review?
![Page 26: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/26.jpg)
How are the results presented?
• Number needed to treat (NNT)• Odds Ratio• Relative risk• Mean difference
![Page 27: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/27.jpg)
Odds and risk
Odds of winning
You versus the rest
Risk of winning
You versus all the runners
10 horses running, you bet on 1 horse
1:9 1:10
![Page 28: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/28.jpg)
Odds and risk
Odds of winning
You versus the rest
Risk of winning
You versus all the runners
10 horses running, you bet on 4 horses
4:6 4:10
![Page 29: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/29.jpg)
Forest plots
more than 1less than 1 1
![Page 30: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/30.jpg)
Forest plots
more than 1less than 1 1
Line of no effect
![Page 31: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/31.jpg)
Forest plots
more than 1less than 1 1
Line of no effect
![Page 32: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/32.jpg)
Forest plots
more than 1less than 1 1
Line of no effect
Best estimate
![Page 33: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/33.jpg)
Forest plots
more than 1less than 1 1
Line of no effect
Best estimate
Confidence interval
![Page 34: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/34.jpg)
Forest plots
more than 1less than 1 1
Line of no effect
Best estimate
Confidence interval
![Page 35: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/35.jpg)
Forest plots
more than 1less than 1 1
Line of no effect
Best estimate
Pooled result
Confidence interval
![Page 36: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/36.jpg)
Question 7
How precise are the results?
![Page 37: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/37.jpg)
Confidence intervals
Practical exercise
![Page 38: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/38.jpg)
P-value
Could the result have occurred by chance?
p = 0.001 (1 in 1000)
p = 0.2 (1 in 5)
A p-value of less than 0.05 (1 in 20) is considered to be statistically significant
![Page 39: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/39.jpg)
Question 8
Can the results be applied to the local population?
![Page 40: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/40.jpg)
Application
• Difficult to answer if you don’t have your own population
• Need to rely on experts• Think about whether the study could be
easy replicated with another population
![Page 41: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/41.jpg)
Question 9
Were all the important outcomes considered?
![Page 42: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/42.jpg)
Outcomes
• Do the authors address all the outcomes they set out to study?
• Are the outcomes considered from different viewpoints e.g. patient, practitioners, policy makers?
![Page 43: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/43.jpg)
Question 10
Are the benefits worth the harms and the costs?
![Page 44: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/44.jpg)
Benefit v harms and costs
• How significant are the benefits?
• Is there any discussion of possible side effects etc.?
• Are financial costs mentioned?
![Page 45: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/45.jpg)
Summary
Validity
Is it trustworthy?
Results
What does it say?
Relevance
Will it help?
![Page 46: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/46.jpg)
Qualitative research
![Page 47: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/47.jpg)
What is qualitative research?
• Concerned with the world of human experience from the view of the participants
• Natural settings are used• A study is ‘led’ by the subjects’
experiences• Qualitative research is rigorous and
systematic• Results are not usually generalisable or
totally replicable
![Page 48: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/48.jpg)
Components of qualitative research
Research question
Sampling process
Data collection
Data analysis
![Page 49: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/49.jpg)
Question 1
Was there a clear statement of the aims of the research?
![Page 50: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/50.jpg)
The research question
• Describes why the research is being carried out
• Qualitative research addresses the
questions• Are the aims of the research clear?
![Page 51: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/51.jpg)
Question 2
Is a qualitative methodology appropriate?
![Page 52: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/52.jpg)
Why qualitative research?
• Increases knowledge in an area that is poorly understood
• Challenges assumptions and practices• Acts as a precursor to quantitative
research• Generates new ideas
![Page 53: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/53.jpg)
Question 3
Was the research design appropriate to address the aims of
the research?
![Page 54: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/54.jpg)
Research methodologies
phenomenology
field research
groundedtheory
action research
ethnography
![Page 55: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/55.jpg)
Question 4
Was the recruitment strategy appropriate to the aims of the
research?
![Page 56: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/56.jpg)
Sampling• Sample size can be determined by
– Data analysis– Time constraints
• What population does the sample refer to?
• How was the sample selected?
![Page 57: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/57.jpg)
Sampling
Methods of sampling:• Convenience• Purposive • Quota• Snowballing
![Page 58: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/58.jpg)
Question 5
Was the data collected in a way that addressed the research
issue?
![Page 59: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/59.jpg)
Data collection
• Are the methods the most suitable for the research question?
• Are the methods explicit?• Is the venue clearly described?
![Page 60: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/60.jpg)
Question 6
Has the relationship between researcher and participants been
adequately considered?
![Page 61: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/61.jpg)
Reflexivity and researcher bias
VenueArea being studied
Types of interview questions asked
Meaning given to data
![Page 62: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/62.jpg)
Question 7
Have ethical issues been taken into consideration?
![Page 63: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/63.jpg)
Ethics
Need to consider:• Consent• Confidentiality• Professional responsibility• Reporting• Ethics committee approval
![Page 64: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/64.jpg)
Question 8
Was the data analysis sufficiently rigorous?
![Page 65: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/65.jpg)
Data analysis
• Description of analysis• Clarity of approach• Use of all the data• Potential for bias
![Page 66: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/66.jpg)
Question 9
Is there a clear statement of findings?
![Page 67: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/67.jpg)
The findings
• Are the findings explicit?• Are they credible?
– respondent validation– quality of argument– would another researcher make a similar
interpretation?– are alternative interpretations explored?
![Page 68: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/68.jpg)
Triangulation in
terv
iew
s
observationsquestionnaire
patients
professionals
care
rs
![Page 69: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/69.jpg)
Question 10
How valuable is the research?
![Page 70: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/70.jpg)
Value of the research
Consider your appraisal of the paper in terms of:
• Credibility• Dependability• Transferability• Confirmability
How useful is the paper to you?
![Page 71: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/71.jpg)
Questions to ask
![Page 72: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/72.jpg)
Running a critical appraisal workshop
![Page 73: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/73.jpg)
Before the session
• Who is the group?• Is it qualitative, quantitative, or both?• Know what equipment and facilities are
available• Ask participants to read the article in
advance
![Page 74: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/74.jpg)
Structuring the session
• Dependent on the group and time frame• Presentation• Encourage audience participation• Use interactive activities • Practical critical appraisal in groups• Feedback
![Page 75: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/75.jpg)
Choosing an article• For a mixed group choose a fairly generic
article• For a discipline specific group:
– Do you want to use a discipline specific article?– Ask for topic ideas– Not a good idea to let them choose the articles
• Choose studies with both strengths and weaknesses
• Make sure it’s easily available to all the participants
![Page 76: Critical appraisal](https://reader033.fdocuments.in/reader033/viewer/2022061204/547e825ab4af9fcd4d8b4656/html5/thumbnails/76.jpg)
Any Questions?
The end!
Paula FunnellFaculty Liaison Librarian (Medicine & Dentistry)
Queen Mary University of [email protected]