Evaluating the Evaluating the Medical LiteratureMedical Literature

Clista Clanton, MSLS, AHIPClista Clanton, MSLS, AHIP

January 2009January 2009

Haynes, RB. ACPJC Nov/Dec 2006;145(3):A8-9

Start at the top and work your way down.

Computer system

Clinical Evidence or PIER (UpToDate)

ACP Journal Club, InfoPOEMS, Dynamed

Cochrane Library, PubMED Clinical Queries, BMJUpdates,

guidelines

Original Studies

Original Studies

If an original study is your only option…….

IMRAD formatIMRAD format IIntroduction: why the authors decided to ntroduction: why the authors decided to

conduct the research.conduct the research. MMethods: how they conducted the ethods: how they conducted the

research and analyzed their results.research and analyzed their results. RResults: what was found.esults: what was found.

AAndnd DDiscussion: what the authors think the iscussion: what the authors think the

results mean.results mean.

PP-ICONSPP-ICONS

ProblemProblem Patient or populationPatient or population InterventionIntervention ComparisonComparison OutcomeOutcome Number of subjectsNumber of subjects StatisticsStatistics

Flaherty, Robert J. A simple method for evaluating the clinical literature. Fam Prac Mgt, May 2004;47-52. Available online at

http://www.aafp.org/fpm/20040500/47asim.html.

ScenarioScenario

You just saw a nine-year old patient with You just saw a nine-year old patient with common warts on her hands. She is an common warts on her hands. She is an ideal candidate for cryotherapy. Her ideal candidate for cryotherapy. Her mother has heard about treating warts mother has heard about treating warts with duct tape and wants to know if you with duct tape and wants to know if you would recommend this treatment. would recommend this treatment.

Clinical QuestionClinical Question

What is your clinical question?What is your clinical question?

“ “Is duct tape an effective treatment Is duct tape an effective treatment for warts in children?”for warts in children?”

PICO: Patient, Intervention/Comparison, PICO: Patient, Intervention/Comparison, OutcomeOutcome

Write your question down, it can help Write your question down, it can help you to clarify exactly what you are you to clarify exactly what you are looking for.looking for.

SearchSearch

After you have your clinical question, search After you have your clinical question, search the appropriate databases:the appropriate databases: Dynamed, PIER, UpToDate, CochraneDynamed, PIER, UpToDate, Cochrane, Clinical , Clinical

Evidence, Essential Evidence PlusEvidence, Essential Evidence Plus PubMedPubMed

Focht DR 3rd, Spicer C, Fairchok MP.Focht DR 3rd, Spicer C, Fairchok MP.The efficacy of duct tape vs cryotherapy in The efficacy of duct tape vs cryotherapy in the treatment of verruca vulgaris (the the treatment of verruca vulgaris (the common wart). Arch Pediatr Adolesc Med. common wart). Arch Pediatr Adolesc Med. 2002 Oct;156(10):971-4. 2002 Oct;156(10):971-4.

AbstractAbstract OBJECTIVE:OBJECTIVE: To determine if application of duct tape is as effective as cryotherapy in the To determine if application of duct tape is as effective as cryotherapy in the

treatment of common warts. treatment of common warts.

DESIGN:DESIGN: A prospective, randomized controlled trial with 2 treatment arms for warts in A prospective, randomized controlled trial with 2 treatment arms for warts in children. children.

SETTING:SETTING: The general pediatric and adolescent clinics at a military medical center. The general pediatric and adolescent clinics at a military medical center.

PATIENTS:PATIENTS: A total of 61 patients (age range, 3-22 years) were enrolled in the study from A total of 61 patients (age range, 3-22 years) were enrolled in the study from October 31, 2000, to July 25, 2001; 51 patients completed the study and were available for October 31, 2000, to July 25, 2001; 51 patients completed the study and were available for analysis. analysis.

INTERVENTION:INTERVENTION: Patients were randomized using computer-generated codes to receive Patients were randomized using computer-generated codes to receive either cryotherapy (liquid nitrogen applied to each wart for 10 seconds every 2-3 weeks) for either cryotherapy (liquid nitrogen applied to each wart for 10 seconds every 2-3 weeks) for a maximum of 6 treatments or duct tape occlusion (applied directly to the wart) for a a maximum of 6 treatments or duct tape occlusion (applied directly to the wart) for a maximum of 2 months. Patients had their warts measured at baseline and with return visits. maximum of 2 months. Patients had their warts measured at baseline and with return visits.

MAIN OUTCOME MEASURE:MAIN OUTCOME MEASURE: Complete resolution of the wart being studied. Complete resolution of the wart being studied.

RESULTS:RESULTS: Of the 51 patients completing the study, 26 (51%) were treated with duct tape, Of the 51 patients completing the study, 26 (51%) were treated with duct tape, and 25 (49%) were treated with cryotherapy. Twenty-two patients (85%) in the duct tape and 25 (49%) were treated with cryotherapy. Twenty-two patients (85%) in the duct tape arm vs 15 patients (60%) enrolled in the cryotherapy arm had complete resolution of their arm vs 15 patients (60%) enrolled in the cryotherapy arm had complete resolution of their warts (P =.05 by chi(2) analysis). The majority of warts that responded to either therapy did warts (P =.05 by chi(2) analysis). The majority of warts that responded to either therapy did so within the first month of treatment. so within the first month of treatment.

CONCLUSION:CONCLUSION: Duct tape occlusion therapy was significantly more effective than Duct tape occlusion therapy was significantly more effective than cryotherapy for treatment of the common wart.cryotherapy for treatment of the common wart.

Problem (Problem (PPP-ICONS)P-ICONS)

What is the clinical condition that was What is the clinical condition that was studied in the article?studied in the article?

OBJECTIVE: To determine if application of duct tape is as effective as cryotherapy in the treatment of common warts.

The problem studied should be sufficiently similar to your clinical problem, or the results will not be relevant.

Patient or Population Patient or Population (P(PPP-ICONS)-ICONS) Is the study group similar to your patient Is the study group similar to your patient

or practice?or practice?

SETTING: The general pediatric and adolescent clinics at a military medical center. PATIENTS: A total of 61 patients (age range, 3-22 years)

If the patients in the study are not similar If the patients in the study are not similar to your patient (older, sicker, different to your patient (older, sicker, different gender or more clinically complicated), the gender or more clinically complicated), the results may not be relevant.results may not be relevant.

Intervention (PP-Intervention (PP-IICONS)CONS)

Is the intervention the same as what you Is the intervention the same as what you are looking for? are looking for?

Could be a diagnostic test or a treatmentCould be a diagnostic test or a treatment

The patient’s mother has heard about treating warts with duct tape and wants to know if you would recommend this treatment.

Comparison (PP-IComparison (PP-ICCONS)ONS)

The comparison is what the treatment is The comparison is what the treatment is tested against. tested against.

Could be a different diagnostic test, Could be a different diagnostic test, another therapy, placebo, or no treatment another therapy, placebo, or no treatment at all.at all.

INTERVENTION: Patients were randomized using computer-generated codes to receive either cryotherapy (liquid nitrogen applied to each wart for 10 seconds every 2-3 weeks) for a maximum of 6 treatments or duct tape occlusion (applied directly to the wart) for a maximum of 2 months.

Outcome (PP-ICOutcome (PP-ICOONS)NS)

Disease-oriented outcomes (DOEs): usually Disease-oriented outcomes (DOEs): usually reflect changes in physiologic parameters.reflect changes in physiologic parameters. It has long been assumed that improving the It has long been assumed that improving the

physiologic parameters of a disease will result in a physiologic parameters of a disease will result in a better outcome, but this is not always true.better outcome, but this is not always true.

Patient-oriented evidence that matters Patient-oriented evidence that matters (POEMs): look at outcomes such as (POEMs): look at outcomes such as morbidity, mortality and cost. morbidity, mortality and cost.

Therefore, DOEs are interesting but of Therefore, DOEs are interesting but of questionable relevance, whereas POEMs are questionable relevance, whereas POEMs are very interesting and very relevant.very interesting and very relevant.

MAIN OUTCOME MEASURE: Complete resolution of the wart being studied.

Number (PP-ICONumber (PP-ICONNS)S)

Number of subjects in the study is Number of subjects in the study is crucial in whether accurate statistics crucial in whether accurate statistics can be generated from the data.can be generated from the data. Too few patients may not be enough to show that Too few patients may not be enough to show that

a difference really exists between intervention and a difference really exists between intervention and comparison groups (power of a study).comparison groups (power of a study).

Many studies contain <100 subjects, which is Many studies contain <100 subjects, which is usually inadequate to provide reliable statistics.usually inadequate to provide reliable statistics.

Good rule of thumb – 400 subjects needed.Good rule of thumb – 400 subjects needed.88

Krejcie RV, Morgan DW. Determining sample size for research activities. Educational and Psychological Measurements. 1970;30:607-610.

51 patients completed the study

Statistics (PP-ICONStatistics (PP-ICONSS)) Relative risk reduction (RRR): the percent Relative risk reduction (RRR): the percent

reduction in events in the treated group reduction in events in the treated group compared to the control group event rate.compared to the control group event rate. Not a good way to compare outcomesNot a good way to compare outcomes

Amplifies small differences and makes insignificant Amplifies small differences and makes insignificant findings appear significantfindings appear significant

Doesn’t reflect the baseline risk of the outcome eventDoesn’t reflect the baseline risk of the outcome event

Can make weak results look good, thereforeCan make weak results look good, therefore Popular and will be reported in almost every journal Popular and will be reported in almost every journal

articlearticle Ignore – it can mislead youIgnore – it can mislead you

RRR would be (85 percent – 60 percent/60 percent x 100 = 42 percent

I.e. 42 percent more effective than cryotherapy in treating warts

Statistics (PP-ICONStatistics (PP-ICONSS)) Absolute risk reduction (ARR): the difference Absolute risk reduction (ARR): the difference

in the outcome event rate between the in the outcome event rate between the control group and the experimental group.control group and the experimental group.

A better statistic to evaluate outcome, as it A better statistic to evaluate outcome, as it does not amplify small differences, but does not amplify small differences, but shows the true difference between the shows the true difference between the experimental and control interventions.experimental and control interventions.

ARR for the wart study is the outcome event rate (complete resolution of warts) for duct tape (85 percent) minus the outcome event rate for cryotherapy (60 percent) = 25 percent

Statistics (PP-ICONStatistics (PP-ICONSS)) Number needed to treat (NNT): number of patients Number needed to treat (NNT): number of patients

who must be treated to prevent one adverse who must be treated to prevent one adverse outcome OR the number of patients who must be outcome OR the number of patients who must be treated for one patient to benefittreated for one patient to benefit Single most clinically useful statisticSingle most clinically useful statistic Easy to calculate, simply the inverse of the ARR.Easy to calculate, simply the inverse of the ARR.

For the wart study, the NNT is 1/25 percent = 1/0.25 = 4

4 patients need to be treated with duct tape for one to benefit more than if treated by cryotherapy

The lower the NNT, the better. For primary therapies, an The lower the NNT, the better. For primary therapies, an NNT of 10 or less is good, with less than 5 being very NNT of 10 or less is good, with less than 5 being very good.good.

For preventive interventions, the NNT will be higher. An For preventive interventions, the NNT will be higher. An NNT for prevention of less than 20 might be particularly NNT for prevention of less than 20 might be particularly good.good.

Intention to Treat AnalysisIntention to Treat Analysis Attrition: Were patients lost to follow-up, and if so, why?

Intention to treat: subjects are analyzed according to the categories into which they were originally randomized. – Benefits of a treatment are more difficult to demonstrate with

intention-to-treat analysis.

– Helps to mitigate differences by including subjects who are unlikely to have experienced benefit from the intervention.Six patients from cryotherapy group and 4 patients from the duct tape group were lost to follow-up (16% of patients).Worst case scenario: 6 cryotherapy patients had wart resolution and the 4 duct tape patients had residual wart. Wart resolution would then be: duct tape 78% and cryotherapy 68% (95% CI, -17 to 28) – therefore not a statistically significant difference between the two treatments.

Christakis DA, Lehmann HP. Is duct tape occlusion therapy as effective as cryotherapy for the treatment of the common wart? Arch Pediatr Adolesc Med, Oct 2002; vol. 156; 975-977.

Best Type of Study for Your QuestionBest Type of Study for Your Question

Type of QuestionType of Question Suggested Best Type of StudySuggested Best Type of Study

TherapyTherapy RCT > cohort > case control > case seriesRCT > cohort > case control > case series

DiagnosisDiagnosis Prospective, blind comparison to gold standardProspective, blind comparison to gold standard

Etiology / HarmEtiology / Harm RCT > cohort > case control > case seriesRCT > cohort > case control > case series

PrognosisPrognosis Cohort study > case control > case seriesCohort study > case control > case series

PreventionPrevention RCT > cohort study > case control > case seriesRCT > cohort study > case control > case series

Clinical ExamClinical Exam Prospective, blind comparison to gold standardProspective, blind comparison to gold standard

CostCost Economic analysisEconomic analysis

Questions of therapy, etiology and prevention which can best be answered Questions of therapy, etiology and prevention which can best be answered by RCT can also be answered by a meta-analysis or systematic review.by RCT can also be answered by a meta-analysis or systematic review.

ResourcesResources

Presentation/handouts available at:http://southmed.usouthal.edu/library/ebmclass/rotationswinterspring.htm

AssignmentAssignment Identify a clinical problem with a patientIdentify a clinical problem with a patient Formulate a clinical question using PICOFormulate a clinical question using PICO Search the literature for appropriate article(s)Search the literature for appropriate article(s) Evaluate the article(s)Evaluate the article(s) Complete the online assignment within two weeks Complete the online assignment within two weeks

after date of lecture.after date of lecture. http://southmed.usouthal.edu/library/ebmclass/rotationswinterspring.htm