Critically appraising a paper

Furrat Amen, MRCS(Eng), DOHNS, MSc

November 2005

Journal

• Well known?• International/local peer review• Authors on editorial board?• Impact factor• Important topic?

Impact factor

• For example, the 2003 Impact factor for a journal would be calculated as follows:• A = Number of times articles published

in 2001-2 were cited in tracked journals during 2003

• B = Number of articles published in 2001-2

• 2003 Impact Factor = A/B

Levels of evidence

• 1a - SR of RCTs • 1b - Individual RCT• 1c – all or none case series

• 2a - SR of cohort studies• 2b - Individual cohort study

• 3a - SR of case-control studies• 3b - Individual Case-Control Study

• 4 - Case-series

• 5 - Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"

Authors and Institutions

• Have the authors published in this area before?

• What institution?• Good Name?

Article

• Expectations of the article• Interesting?• Other research in the area?• Well presented, prepared, clear,

informative?

Study

• Prospective / Retrospective• Time between onset of symptoms

and diagnosis• Blind scoring / bias• RCT – impartial interpretation of

results• Multi-centre trial better?

Subsection appraisal

• Title– Does the title do justice to the article and

vice-versa

• Abstract – a good outline?• Aims – clear• Introduction

– Informative and justification for work undertaken

Materials and methods

• Repeatability• Well designed study?• Well controlled study?• Comparable controls?• Sufficient numbers for stats• Appropriate stats?

Results

• Clear and comprehensive?• Authors interpretation correct?• Text, figures, graphs, tables,

illustrations• Your interpretation same as authors?

Discussion

• Data discussed comprehensively?• Absence of discussion in areas?• Bias• Conclusions agree with own

interpretations

References

• Up to date• Citations from reputable journals?• References appropriate?• Self references?

– number

Summary

• Believe article?• Change of views• Improvements• Read article 2-3 times

Long-term effects of Micronized Alloderm Injection for Unilateral Vocal Cord Paralysis

A critical review

Furrat Amen

Summary

• 20 patients• Unilateral vocal cord paralysis• Injected with alloderm (derived from

cadaveric dermis)• Assessment

– Questionnaires– Recordings assessed– videostroboscopy

Journal

• Laryngoscope is the official journal of the American Laryngological, Rhinological and Otological Society

• Well known• National and international peer

reviewed• Authors not on editorial board• Impact factor 1.449• Topic is important

Authors and Institutions

• Claudio Milstein– No previous articles on vocal cord injection

• Lee Akst– No previous articles on vocal cord injection

• Douglas Hicks– Many articles about voice, but non on

injection

• Tom Abelson– No previous articles on VC injection

• Marshall Strome– Papers on laryngeal transplantation

Institution

• Cleveland Clinic Foundation– Cleveland, Ohio, USA

• Not for profit• Research and Clinical medicine• Established 1921

Outpatients!!!

Article

• Expectations– Long periods of follow-up for 20 patients– Interested – never heard of alloderm

before– First described in June 2002

• 5 articles

– Well written, misleading number of patients, but informative

Study

• Retrospective case series (level 4)• Time between vocal cord paralysis and

injection was 4 to 216 months ? Had another procedure in mean time

• “voice experts” assessed blindly– Digital voice samples– Videostroboscopy– Not fully blind as all patients had alloderm!

• Patients were not blind• Bias – on 3 patients – retrospective

questionnaire

Study

• Twenty patients– Only 8 patients had 12+ months FU

• Multicentre trial may have increased numbers

Title

• Long term effects ? justified– Only eight patients 12 months +– No mention of mortality shortening FU

• Abstract skimmed over the shortfalls of study

• Aims clearly defined but not achieved• Introduction – good background and

reasoning

Materials and methods

• It could be repeated• Design would have been better as a

prospective study• No controls• Low numbers involved

Statistics

• Wilcoxon Signed Rank Test– Non parametric alternative to paired t-

test• Does not assume a normally distributed

population• Used for ordinal data (in categories e.g. Likert

scale)

– Use seems appropriate– Impressive

• P<0.0001 for bowing and glottic closure• P<0.01 for self-perception of voice

Results

• Clear and comprehensive• I agreed with interpretation of

authors• Photo Fig 1 – clearly demonstrates

better glottic closure• Table 1. Shows demographics and

length of FU• Table 2. Shows change from best to

worst pre and post injection– Glottic closure / bowing / voice perception

Tables and Figures

• Fig.2 and 3– Repeat data in table 2

• Fig. 4 and 5– Degree of improvement depends on

width of graph

Discussion

• Good discussion– Self-critical– “anecdotal” results– Pilot study/case series – similar to a

study in Laryngoscope in 2002 but with more than 1 month FU

– No discussion about mortality of patients causing short follow up

– Conclusion• Further investigation needed• agree

References

• 17 references• 6 references 2000+• One reference from 1911 in German• References relevant• No self references

Summary

• I do believe article– Viable option for VC injection

• ?better• Had no opinion on topic before• Improvements

– Multicentre trial– Prospective double blind randomised

controlled trial between alloderm and bioplastique

– Longer follow-up