Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am...

Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 1

Qualityofnewsmediareportsabouttheeffectsandcostsofhealthinterventions:SystematicreviewprotocolOxmanMWorkingpaper,April26,2018www.informedhealthchoices.org


Colophon

Title Qualityofnewsmediareportsabouttheeffectsandcostsofhealthinterventions:Systematicreviewprotocol

Authors Oxman, Matt 1,2,3

1. University of Oxford, United Kingdom 2. Faculty of Health Sciences,Oslo Metropolitan University, Norway 3. Centre for Informed Health Choices,Norwegian Institute of Public Health, Norway

Correspondingauthor

Matt [email protected] Telephone: +47 98636788 Oslo Metropolitan University, Faculty of Health Sciences Section for Research & Development Pilestredet Campus P.O. Box 4 St. Olavs plass N-0130 Oslo, Norway

Keywords Newsmedia,Systematicreview,Unreliableinformation,KeyConcepts

Citation Oxman,M.,Qualityofnewsmediareportsabouttheeffectsandcostsofhealthinterventions:Systematicreviewprotocol.IHCWorkingPaper.Oslo:InformedHealthChoices.ISBN: 978-82-8082-971-9

Articlecategory �AboutInformedHealthChoices�Keyconceptsandglossary�Learningresources�Systematicreviews�Developmentandevaluationoflearningresources�Claimevaluationtools�Editorialsandcommentaries�Grantapplicationsþ Other

Date April26,2018


Abstract

Background:Unreliableinformationabouthealthcare,combinedwiththeina-bilitytoassessthereliabilityofsuchinformation,canleadtouninformeddeci-sionsand,ultimately,wasteandunnecessarysuffering.Journalismisaparticu-larlyimportantsourceofhealthinformation.Therehavebeenseveralscientificstudiesofthequality(reliability)ofnewsmediareportsabouttheeffectsofhealthinterventions.Theredoesnotappeartobeasystematicreviewofthesestudies.Theplannedreviewisintendedtoinform:theproductionandcon-sumptionofhealthnews;aswellasfurtherresearch,includingthedevelopmentofinterventionstohelppeopleassesshealthcareinformationinthenews,sotheycanmakewell-informeddecisions.Objectives:1)Toassesscriteriausedtomeasurethequalityofnewsmediare-portsabouttheeffectsandcostsofhealthinterventions;2)toassessthequalityofsuchreports;and3)toexplorefactorsthatmightexplainvariationinquality.Eligibility:Wewillincludescientificstudiesofprint,broadcastandonlinenewsreports(thepopulation).Atleastoneexplicitcriterionmusthavebeenusedbytheresearcherstomeasurequality(thecondition).Itmustbepossibleandsen-sibletoframetheresultsforatleastonecriterionastheproportionofreportsthatsatisfiedit(e.g.theproportionofreportsinwhicheffectestimatesareab-solute,notonlyrelative).Therewillbenolimitsonlanguage,geographyortimeperiodofthereports(thecontext).Tobeincluded,studiesmustspecify:thesamplingframe;theselectioncriteria;andtheselectiontechnique.Methods:Tworeviewerswillextractdata,assessriskofbias,andmakejudge-mentsaboutInformedHealthChoices(IHC)KeyConceptscapturedbycriteriausedinscientificstudies.Ifnecessary,athirdreviewerwillhelpreachconsen-susonanyjudgement.Meta-analyseswillbeconductedtoestimatethepreva-lenceofnewsreportsthatsatisfyrespectivecriteria.Toexplorevariation,sub-groupanalyseswillbeconductedfor:medium(broadcastvs.other);timeperi-od(decade);financialmodel(commercialvs.non-commercial);journalistspe-cialisation(healthorsciencevs.other);andcountryincomelevel(low-incomevs.other).


Background

TheproblemUnreliablehealthinformationThisprotocolisforasystematicreviewthatisultimatelyintendedtohelpad-dressamajor,globalproblem.Theproblemhasthreeparts.Thefirstisunrelia-blehealthinformation.Ineffect,thereisendlessinformationabouttheeffectsofhealthinterventions,ahealthinterventionbeinganyactionintendedtoimproveormaintainthehealthofindividualsorgroups.Thisincludesinformationabout:“modern”,“academ-ic”,“conventional”or“western”medicine;“complementary”,“alternative”,“tra-ditional”or“natural”medicine;screening;surgeryanddevices;diet,exerciseandlifestyle;andsystemsandpolicies.Someofthisinformation,ifnotmostofit,isunreliableinonewayoranother.Theinformationcanbedirectlymislead-ing,suchastheexplicitclaimthataninterventioncausesanoutcomewhenitisonlyassociatedwiththeoutcome.Oritcanbemisleadingbyomission,suchasrelativeeffectestimateswithoutabsoluteestimates,particularlywhenthebase-lineriskissmall(Woloshin,etal.,2008).Researchershavefoundunreliablelayinformationabouttheeffectsofhealthinterventionsin:

• Patientmaterials(Coulter,etal.,1999);productlabels(UnitedStatesGovernmentAccountabilityOffice,2010),

• Varioustypesofwebsites(Culver,etal.,1997;Wolfe,etal.,2002;Glenton,etal.,2005;Spencer,etal.,2016)

• Advertisements(Sansgiry,etal.,1999;Frosch,etal.,2007;Frosch,etal.,2011;Faerber&Kreling,2013;Groven&Braitwaite,2016)

• Thenews(Appendix1)Informationinthenewsmediaisofparticularinterest,forreasonsdescribedlaterinthebackgroundsection.


Finally,researchreferencedinthefollowingsubsectionshowspeopleareuna-bletoassessinformationabouttheeffectsofhealthinterventions.Logically,ifpeopleareunabletoassesssuchinformation,theyaremorelikelytospreadthatwhichisunreliable.Conversely,peoplespreadingunreliableinformation,asdocumentedinthestudiesreferencedinthissubsection,suggeststheyareunabletoassesssuchinformation.InabilitytoassesshealthinformationThesecondpartoftheproblemisthatpeopleareunabletoassessthereliabilityofinformationabouttheeffectsofhealthinterventions.AspartoftheInformedHealthChoices(IHC)project(www.informedhealthchoices.org),theabilitytoassesssuchinformationandmakeinformeddecisionsabouthealthinterven-tionshasbeenbrokendownintotheabilitytoapplyspecificconceptscalledtheIHCKeyConcepts.Forexample,peopleneedtobeabletoapplytheconceptthatassociationisnotthesameascausation.Theoriginallistincluded32concepts(Austvoll-Dahlgren,etal.,2015).Thenewestiterationincludes36(Box1)(Chalmers,etal.,2018).TheabilitytoapplyIHCKeyConceptsispartofhealthliteracyandhasbeenmeasuredbothdirectlyandindirectly.Therehavebeenlargesurveysofhealthliteracy,inEuropebytheEuropeanHealthLiteracyProjectConsortium(HLS-EUConsortium,2013)andintheUnit-edStatesbytheNationalCenterforEducationStatistics(Kutner,etal.,2006).AccordingtotheEuropeansurvey,“Healthliteracyislinkedtoliteracyanden-tailspeople’sknowledge,motivationandcompetencestoaccess,understand,appraise,andapplyhealthinformationinordertomakejudgmentsandtakede-cisionsineverydaylifeconcerninghealthcare,diseasepreventionandhealthpromotiontomaintainorimprovequalityoflifeduringthelifecourse”(HLS-EUConsortium,2013,p.7).FortheAmericansurvey,thetermwasdefinedas:“Thedegreetowhichindividualshavethecapacitytoobtain,process,andunder-standbasichealthinformationandservicesneededtomakeappropriatehealthdecisions”(Kutner,etal.,2006,p.iii).TheabilitytoapplyIHCKeyConcepts(i.e.assessthereliabilityofinformationabouttheeffectsofhealthinterventionsandmakewell-informeddecisions)islogicallyincludedinbothdefinitions.However,neithersurveyappearstohaveobjectivelymeasuredthis.TheEuropeansurveymeasuredthesubjectiveeaseof47tasks.TasksrelatedtoapplyingIHCKeyConcepts,including“judgeiftheinformationaboutillnessinthemediaisreliable”,wererankedamongstthemostdifficult(HLS-EUConsortium,2013).


Box1:ShorttitlesfortheInformedHealthChoices(IHC)KeyConceptsRecognisinganunreliablebasisforaclaim

1.1. Treatmentscanharm.1.2. Anecdotesareunreliableevidence.1.3. Associationisnotthesameascausation.1.4. Commonpracticeisnotalwaysevidence-based.1.5. Newerisnotnecessarilybetter.1.6. Expertopinionisnotalwaysright.1.7. Bewareofconflictinginterests.1.8. Moreisnotnecessarilybetter.1.9. Earlierisnotnecessarilybetter.1.10. Hopemayleadtounrealisticexpectations.1.11. Explanationsabouthowtreatmentsworkcanbewrong.1.12. Dramatictreatmenteffectsarerare.

Understandingwhethercomparisonsarefairandreliable2.1. Comparisonsareneededtoidentifytreatmenteffects.2.2. Comparisongroupsshouldbesimilar.2.3. Peoples’outcomesshouldbeanalysedintheiroriginalgroups.2.4. Comparisongroupsshouldbetreatedequally.2.5. Peopleshouldnotknowwhichtreatmenttheyget.2.6. Peoples’outcomesshouldbeassessedsimilarly.2.7. Allshouldbefollowedup.2.8. Consideralltherelevantfaircomparisons.2.9. Reviewsoffaircomparisonsshouldbesystematic.2.10. Peerreviewandpublicationdoesnotguaranteereliableinformation.2.11. Allfaircomparisonsandoutcomesshouldbereported.2.12. Subgroupanalysesmaybemisleading.2.13. Relativemeasuresofeffectscanbemisleading.2.14. Averagemeasuresofeffectscanbemisleading.2.15. Faircomparisonswithfewpeopleoroutcomeeventscanbemisleading.2.16. Confidenceintervalsshouldbereported.2.17. Donotconfuse‘statisticalsignificance’with‘importance’.2.18. Donotconfuse‘noevidenceofadifference’with‘evidenceofnodifference’.

Makinginformedchoices3.1. Dotheoutcomesmeasuredmattertoyou?3.2. Areyouverydifferentfromthepeoplestudied?3.3. Arethetreatmentspracticalinyoursetting?3.4. Dotreatmentcomparisonsreflectyourcircumstances?3.5. Howcertainistheevidence?3.6. Dotheadvantagesoutweighthedisadvantages?


TheAmericansurveysuggestedaminorityofAmericanadults(12%)had“pro-ficient”healthliteracy(Kutner,etal.,2006,p.v).However,onlyasampleofthehealthliteracyquestionsusedinthatsurveyarepubliclyavailableandnoneoftheavailablequestionsmeasuretheabilitytoassessinformationaboutcausali-ty(www.nces.ed.gov/NAAL/sample.asp).Anothersurrogatemeasureofpeople’sabilitytoapplyIHCKeyConceptsistheirexpectationsabouteffects.Inasystematicreviewofpatients’expectations,themajorityofparticipantsoverestimatedbenefitsandunderestimatedharms(Hoffmann&DelMar,2015).Asystematicreviewofclinicians’expectationsshowedcliniciansdothesame(Hoffmann&DelMar,2017).Othersubstitutesforabilityareattitudesandbeliefs.AccordingtoastudyintheUnitedKingdom,patientsandgeneralpractitionersappeartoplaceadispropor-tionateamountoftrustinthereputationoftheorganisationconductingaclini-caltrial,thequalificationsoftheresearchers,andpeer-review,versusthemeth-ods,whicharewhatinfactdeterminethereliabilityoftheresults(TheAcademyofMedicalSciences,2016).Moreover,aboutathirdoftherespondents(37%)placedahighleveloftrustindatafrommedicaltrials,whileabouttwothirds(65%)placedahighleveloftrustintheexperiencesoffriendsandfamily,i.e.anecdotalevidence.TherehaveindeedbeenstudiesdirectlymeasuringtheabilitytoapplyKeyCon-cepts.InNorway,whereIambased,Oxmanetal.testedarandomsampleof626Norwegianadults(Oxman,etal.,2017).Aboutoneinfive(19%)showedtheywereabletodistinguishbetweenanassociationandcausation.OthersurveyssuggestNorwegianstudentsinpost-secondaryschool(Pettersen,2007)andsecondaryschool(Pettersen,2005)alsostrugglewithassessingclaimsabouttheeffectsoftreatments.AnotherproductoftheInformedHealthChoicesprojectistheCLAIMEvaluationToolsdatabase(www.informedhealthchoices.org/claim-evaluation-tools)(Chalmers,etal.,2018).TheCLAIMEvaluationToolsaremultiple-choiceques-tionsdevelopedspecificallytomeasurepeople’sabilitytoapplytheIHCKeyConcepts(Austvoll-Dahlgren,etal.,2016;Austvoll-Dahlgren,etal.,2017).Be-forethedevelopmentofthedatabase,inasystematicreviewoftoolsformeas-uringsaidability,Austvoll-Dahlgrenetal.identified215discreteinstrumentsorprocedures,capturingupto15oftheconcepts,whereoffourcaptured10ormore(Austvoll-Dahlgren,etal.,2016).Avalidatedsetof26questionsmeasuringtheabilitytoapply13oftheconcepts(i.e.twoquestionsperconcept)wereusedtomeasuretheeffectsofIHClearn-ingresources,intworandomisedtrials(Nsangi,etal.,2017;Semakula,etal.,


2017).Inthefirsttrial,thecontrolgroupwas4430Ugandanprimaryschoolchildren,afterlosstofollow-up(Nsangi,etal.,2017).Inthesecondtrial,afterlosstofollow-up,thecontrolgroupwas273parentsandguardiansofchildrenparticipatingintheinterventionorcontrolgroupofthefirsttrial(Semakula,etal.,2017).Lessthanhalfoftheparticipantsinthecontrolgroupforeithertrialansweredbothquestionscorrectlyforanyofthe13includedconcepts.UninformedhealthdecisionsThecombinationofunreliableinformationabouttheeffectsofhealthinterven-tionsandpeople’sinabilitytoassesssuchinformationcanresultinuninformeddecisions.Thisisthethirdpartoftheproblem.Peopleactingonunreliablein-formation,orfailingtoactonreliableadvice,caninturnleadtowasteandun-necessarysuffering.Thisiswhytheproblemisimportant.Inasystematicreview,lowhealthliteracywasfoundtobeassociatedwithworsehealthoutcomes(Berkman,etal.,2011).Granted,whatwasmeasuredintheincludedstudies,aswiththeaforementionedsurveysofhealthliteracy,maynotincludetheobjectiveabilitytoapplyIHCKeyConcepts.Peoplecertainlyaremakingpoordecisions.Systematicreviewshavefound:worldwideoveruseofmedicalservicesmorelikelytodoharmthangood(Brownlee,etal.,2017);underuseofeffectiveservices(Glasziou,etal.,2017);andovertestingandundertestinginprimarycare(O'Sullivan,etal.,2018).Meanwhile,fortunesarespenton“alternative”medicineanddietarysupple-mentswithoutreliableevidencethattheyarebeneficialorsafe(Optum,2013;Starr,2015).TheinabilitytoapplyIHCKeyConceptsislogicallyafactorinmanyofthesedecisions.ThisreviewWhydothereview?WhileIamawareofscientificstudiesofthequalityofnewsmediareportsabouttheeffectsofhealthinterventions(Appendix1),Iwasunabletofindasystemat-icreviewofsuchprimarystudiesinEpistemonikos(www.epistemonikos.org),TripPro(www.tripdatabase.com),orGoogleScholar(www.scholar.google.com),aslaidoutinAppendix2.NeitheramIawareofasystematicreviewofhowqualityhasbeenmeasuredinsuchstudies.Zeraatkaretal.conductedaliteraturesearchbeforedevelopingtheirownmeasurementtool,namedtheQualityIndexforhealth-relatedMediaReports(QIMR)(2017).However,theyonlyassessedvalidatedtools,andtheydidnotcomparecriteriatotheIHCKeyConcepts.


Byassessingthemethodsusedtoassessqualityineligiblestudies,theplannedreviewcaninformthedevelopmentanduseofmethodsinfurtherresearch.Byprovidinganoverviewofthequalityofnewsmediareportsabouttheeffectsofhealthinterventions,thereviewcaninformthegenerationofnewresearchquestions,forexamplethegeographicfocusofthosequestions.Intwoways,thereviewcanhelpaddresstheproblemofunreliableinformationabouttheeffectsofhealthinterventions,inabilitytoassessthatinformation,anduninformeddecisionsabouttheinterventions.First,theresultscanbeusedtoraiseawarenessabouttheproblem.Second,byshowinghownewsmediare-portsabouttheeffectsofhealthinterventionstendtobeunreliable(i.e.thespe-cificcriteriathatthereportstendnottosatisfy),theresultscaninformthepri-oritisationofIHCKeyConceptswhendevelopinginterventionstohelpjournal-istsproducereliablereportsandhelpconsumersassessinformationinthenews.Whyfocusonthenewsmedia?Thenewsmediaplayaspecialroleinthedisseminationofhealthinformation.IntheUnitedKingdom,despiteanincreaseintheuseofsocialmedia,themajor-ityofadultsstillfindoutaboutsciencemostregularlythroughnewsmedia,ac-cordingtoarepresentativesurveyof1,749adults(Castell,etal.,2014).Granted,theproportionofpeoplefindingoutaboutspecificallyhealthsciencethroughthenewsmaybedifferent.Meanwhile,ChewandEysenbachfoundthatin5395tweetsabouttheH1N1outbreakin2009,mostlinksweretoanewswebsite(23%)(Chew&Eysenbach,2010).Another12%linkedtonewsblogs,feedsornichenews,comparedto1.5%thatlinkedtogovernmentandpublichealthagencies.Whilethereareotherimportantsourcesoflayinformationabouttheeffectsofhealthinterventions,includingtheminthisreviewwouldbe“mixingapplesandoranges”.Inotherwords,thequalityofinformationinthosesourcesislikelytobesignificantlydifferent.Forexample,patientmaterialsarelikelytoincludemoreinformationaswellasmorereliableinformationthannewsreports,giventhatthematerialsarepreparedbyhealthprofessionalswhoaretryingtoinformpatients,notattractanaudience.Themethodsusedtoassessinformationfromdifferentsourcesmayalsobedifferent.Furthermore,includingsourcesbesidesthenewsmediacouldmakethereviewunwieldy.Rather,thisstudycanbeusedtoinformreviewsfocusingonothersources.AsstatedintheCochraneHandbookforSystematicReviewsofInterventions,there


areprosandconstobothbroadandnarrowreviewquestions,butwiththead-ventofoverviewsofsystematicreviews,“Itmayincreasinglybeconsideredde-sirabletoplanaseriesofreviewswitharelativelynarrowscope,alongsideanOverviewtosummarizetheirfindings”(Higgins&Green,2011).


Objectives

Primary

1. Assesscriteriausedtomeasurethequalityofnewsmediareportsabouttheeffectsandcostsofhealthinterventions

2. Assessthequalityofnewsmediareportsabouttheeffectsandcostsofhealthinterventions

Secondary

3. Explorefactorsthatmightexplainvariationinqualityofnewsmediare-portsabouttheeffectsandcostsofhealthinterventions


Eligibility

CoCoPopMunnetal.notethatquestionsaboutprevalencedonotfitthepopulation,in-tervention,comparatorandoutcome(PICO)framework,usedforsystematicre-viewsaboutefficacy(Munn,etal.,2015,pp.148-149).Thereisnointerventionorexposure,noristhereanoutcomeonwhichaninterventionorexposurecanhaveaneffect.Therefore,insteadofthePICOframework,Iwillusethemne-monicCoCoPop(condition,context,andpopulation)tosetinclusioncriteria,asMunnetal.recommend.Ihaveaddedcriteriaforstudydesign,asperthePre-ferredReportingItemsforSystematicreviewandMeta-AnalysisProtocols(PRISMA-P)statement(Moher,etal.,2015).PopulationInNewsAroundtheWorld:Content,PractitionersandthePublic,ShoemakerandCohenwrite:“[Thejournalist]typicallyconstructsamethodforfulfillingthedailyjobrequirements.Heorsherarelyhasanunderlyingtheoreticalunder-standingofwhatdefiningsomethingorsomeoneasnewsworthyentails”(Shoemaker&Cohen,2006,p.7).Inotherwords,evenjournalistswillstruggletotellyouwhatexactly“news”is.Thisincludesme.InAhistoryofnews,Stephensusesthedefinition:“Newinformationaboutasub-jectofsomepublicinterestthatissharedwithsomeportionofthepublic”(Stephens,2007,p.4).Intermsofthereview,applyingsuchabroaddefinitionwouldbeproblematicinseveralways.Forexample,advertisements,pressre-leasesandjournalarticlescouldallbeconsiderednews.Insteadofusingaspecificdefinition,Iwillbepragmatic.Iwillonlyconsiderstudiesofinformationthat:1)isinnewspapersormagazines(print),radio,podcastsortelevision(broadcast)ornewswebsites(online);and2)islabelledbytheauthorsas“news”oranycommonsynonym.Intermsofonline,Iwillfo-cusondedicatednewswebsites,i.e.notsocialmediaplatforms,althoughnewsreportsappearonsuchplatforms.Iwillonlyincludestudiesofreportsaboutthepotentialhealtheffects(negativeorpositivechangesinhealthoutcomes)andthemonetarycostsofhealthinter-ventions(anyactionintendedtoimproveormaintainthehealthofanindividualorgroup).Thereportscanallbeaboutthesameconditionorintervention,ortheycanbeamix.Ifanotherwiseeligiblestudy’ssampleincludesnewsreportsabouttheeffectsofhealthinterventionsaswellasothertypesofnewsreports,Iwillextractdata


onlyabouttheformer,ifpossible.Otherwise,Iwillleaveoutthestudyandlistitinatableofexcludedstudies.Inthesametable,Iwilllistanystudiesofthequalityofotherhealthinformationinnewsreports,besidesinformationabouttheeffectsofhealthinterventions.ConditionBy“quality”,Imeananattributethatiseitherconducivetoinformeddecisionsabouthealthinterventions(e.g.presentspotentialeffectsinabsolutenumbers)ormisleading(e.g.onlypresentspotentialeffectsinrelativenumbers),orthesumofsuchattributes.Theconditionofmypopulation—newsmediareports—iseitherhavingsatisfiedatleastoneexplicitcriterionforqualityornot(e.g.ei-therpresentingpotentialeffectsinabsolutenumbersornot).InpotentiallyeligibleprimarystudiesofwhichIamaware(Appendix1),re-searchershaveuseddifferentsetsofcriteriatomeasurequality.Zeraatkaretal.conductedaliteraturesearchbeforedevelopingtheirowntooltomeasurethequalityofreportingonhealthresearch,namedtheQualityIndexforhealth-relatedMediaReports(QIMR)(2017).Theyidentifiedoneothervalidatedtoolwiththesamepurpose,namedtheIndexofScientificQuality(ISQ)(Oxman,etal.,1993).Theyalsoidentifiedandassessedinstrumentsformeasuringthequalityofpatientinformation:EnsuringQualityInformationforPatients(EQIP)(Moult,etal.,2004)andDISCERN(Charnock,etal.,1999).Zeraatkaretal.con-cludedthatbothEQIPandDISCERNhavepoorcontentrelevancetohealthnewsreports.ItwouldbeproblematictorestrictthereviewtostudiesthatuseeithertheQIMRorISQtomeasurequality.Thiswouldexcludestudiesofreportswherethehealthinformationisnotbasedonresearch—e.g.informationbasedonan-ecdotalevidence.Moreover,theQIMRwaspublishedasrecentlyas2017,andinmostofthepotentiallyeligiblestudiesofwhichIamaware,neithertheQIMRortheISQwereusedtomeasurequality.Inseveralstudies,researchershaveusedcriteriabasedonthoseusedbyMoynihanetal.(Moynihan,etal.,2000)—e.g.Smithetal.(2005).Inotherstudies,researchershavedevelopedandusedtheirowntools,e.g.Marconetal.(2017).Acriterionmaybesensibleevenifthetoolwasnotvalidated,anditmaybesensibletogroupthecriterionwithothersformeta-analysis.ContextThepopulationforthereviewislimitedtonewsintheaforementionedmedi-ums:newspapersandmagazines:radio,podcastsandtelevision;andnewsweb-sites.Iwillnotsetanylimitsintermsofgeography,languageortimeperiod.StudydesignForastudytobeeligible,theresultsmustincludedichotomousorcategoricaldataforatleastoneexplicitcriterionusedtomeasurequality.Itmustbepossi-bletodichotomisethedataaseithersatisfyingthecriterionornot,fortworea-sons:toensuretheresultscanbeeasilyinterpreted,andtosynthesisefindingsacrossstudies.


Somenuance,suchasthenumberofreportsthat“partially”satisfyacriterion,maybelostinthissynthesis.Theimportanceofsuchdetailisquestionable;ifaqualityisimportantenoughtobemeasured,thecriterionusedtomeasurethatqualityshouldbeclearlysatisfied,assumingitisrelevanttothespecificreport.Studiesthatonlyprovideaglobalqualityscore,suchasastarrating,willbeex-cludedandlistedinthetableofexcludedstudies.Globalscoresaredifficulttointerpret,sincetheydonottellyouwhichpartsofanewsreportareproblemat-ic.Furthermore,theyaredifficulttocompareacrossstudiesusingdifferentscoringsystems.Tobeincluded,studiesmustspecify:thesamplingframe(i.e.wherethenewsreportsweresampledfrom);theselectioncriteriaforthereports;andtheselec-tiontechnique.Thisexcludescasestudiesofsinglereports.Iwilllistanycasestudiesthatwouldotherwisebeeligibleinthetableofexcludedstudies.


Methods

SearchstrategyTomakesuremysearchissystematicandtransparent,IwillbasemyapproachontheguidelinedevelopedbyKableetal.(2012).Asnotedinthedescriptionofmysearchforanexistingreview(Appendix2),whilemyquestionisinterdisci-plinary,Iconsideritunlikelythatasystematicreviewdesignedtoansweritwouldhavebeenpublishedinanacademicjournalfocusedonjournalism.How-ever,Iconsideritmorelikelythataneligibleprimarystudyhasbeenpublishedinsuchajournal.Therefore,inadditiontoPubMed(www.ncbi.nlm.nih.gov/pubmed),IwillsearchGoogleScholar(www.scholar.google.com)andScopus(www.scopus.com).IwillsearchforgreyliteratureinOpenGrey(www.opengrey.ey)andGreyLiteratureReport(www.greylit.org),aswellasthesesinProQuestDissertations&Theses(GlobalFulltextplusUKandIrelandabstracts)(www.proquest.com).Iwillnotplacelimitsonlanguageortimeperiodinanyofmysearches.Inthesearchforanexistingreview,Ionlyincludedtermsforthepopulation(newsmediareports)whensearchingEpistemonikosandTripPro,sinceIcouldfilterforsystematicreviews,whichreducedtheresultstoamanageablenum-ber.Intheupcomingsearchforprimarystudies,giventhereisnotareliablefil-terIcanuseforstudydesign,Iwillalsoincludetermsforthecondition(quality)ortermsrelatedtotheanalysis,toimprovethespecificityofthesearchstrategy.IwillidentifyadditionaltermsforthePubMedsearch,includingMedicalSubjectHeadings(MeSH)terms,bylookingupthestudiesinPubMedor,ifthestudyisnotinPubMed,inanotherdatabase,andextractingrelevantterms.TheaimistoensuremysearchcapturesallstudiesofwhichIamaware,aswellassimilarstudies.Iwillalsoaddtermsforspecificcategoriesofmedia(e.g.print).Iwillconducttestsearchestoinformpragmaticdecisionsabouteditingordroppingterms,withtheaimofmakingthesearchandscreeningasefficientaspossible,whileminimisingtheriskofmissingeligiblestudies.IwillusethePubMedsearchasthebasisforsearchesinotherdatabases.Besidessearchingdatabases,Iwillcheckthereferencelistsofeligiblestudies.InScopus,Iwillscreenstudiescitingthosethatareeligible,aswellasstudiescit-ingthedevelopmentoftheISQortheQIMR.Iwilltakeadvantageofmyprofes-sionalnetwork,conductingtargeteddisseminationoftheprotocolviaemailandbroaddisseminationviaTwitter,tocrowdsourceeligiblestudies.Finally,IwillcontacttheauthorsofeligiblestudiesandthedevelopersoftheISQandQIMR,toaskaboutunpublishedstudies.


Studylanguageisnotaneligibilitycriterion,buttobepragmatic,IwillonlysearchEnglishlanguagedatabases.ToscreenanystudieswithoutanEnglishabstractforeligibility,IwilluseGoogleTranslate(www.translate.google.com).Ifnecessary,Iwillseekhelpfromsomeonefluentinthelanguagee.g.anac-quaintanceoroneoftheauthors.Toextractdatafromanon-Englishpaper,Iwillagainseekhelp.Intheexcludedstudiestable,Iwilllistanyapparentlyeli-giblestudiesfromwhichIamunabletoextractdatabecauseofalanguagebar-rierorbecauseofinadequatereportingandinabilitytoobtainmissinginfor-mationfromtheauthors.AsecondreviewerandIwillindependentlyjudgeeligibilitybyreviewingeachabstractand,ifnecessary,(sectionsof)thefulltext.Appendix3istheformwewillusetoconfirmeligibility.Anydisagreementswillbediscussedbeforemov-ingontodataextractionandassessmentofriskofbias.Athirdreviewerwillbebroughtintoarbitrateifnecessary.DataextractionAsecondreviewerandIwillindependentlyextractdataaboutthepopulation,studydesign,criteriausedtoassessthequalityofthereports,andtheresults.Wewillagaindiscussanydisagreements,bringinginathirdreviewertoarbi-trateifnecessary.Appendix4istheformwewillusetoextractdataaboutpopulationanddesign.Appendix5istheseparatespreadsheetinwhichwewillentertheindividualcriteriaandresultsforeachcriterion.Ihavedeterminedwhatdatatoextractbasedonwhatisreportedinthepoten-tiallyeligiblestudiesofwhichIamaware(Appendix1),varioustools(Vandenbroucke,etal.,2007;TheJoannaBriggsInstitute,2014;Moher,etal.,2015;Munn,etal.,2015),myexpertise,andcommonsense.Appendix6con-tainsdummytablesforallthedatatobeextracted,aswellastheriskofbiasas-sessmentsandanalyses.PopulationAsecondreviewerandIwillextractdataaboutthefollowingvariables,intermsofthesamplesofnewsreports:mediumofpublication;geographicareaofpub-lication;timeperiodofpublication;categoryofinterventionreportedon;andwhethertheoutletiscommercialornot.Intermsofmedium,radioandpod-castswillbeonecategory,inadditiontothecategories:newspapers;magazines;television;andnewswebsites.Theninepredeterminedcategoriesofinterven-tionare:“modern”medicine(aka.“academic”,“conventional”or“Western”medicine);“alternative”medicine(aka.“complementary”,“traditional”or“natu-ral”medicine);screening;surgery;devices;diet;exercise;lifestyle;andsystemsandpolicies.StudydesignForeachstudy,wewillrecord:theinstrumentusedtomeasurequality;thestatedobjectives;thesamplingframe;theselectioncriteriaandtechnique;andanyreportedsubgroupanalyses.


Criteria,responseoptionsandresultsFinally,wewillrecordeacheligiblecriterionusedtoassessthequalityofre-ports,aswellastheresponseoptions,inaspreadsheetseparatefromtheformusedtocollectdataaboutthepopulationandstudydesign(Appendix5).Wewillentertheresultsforeachcriterionintothesamespreadsheet.Wheredataareavailableforsubgroupsofnewsreports,wewillrecordtheoverallresultsandtheresultsforeachsubgroup.AssessingriskofbiasInasystematicreviewoftoolsusedtoassessthequalityofprevalencestudiesandotherobservationalresearch,Shamliyanetal.identified46scalesand51checklist,noneofwhichdiscriminatedbetweenreportingversusmethodologi-calquality,orexternalversusinternalvalidity(2010).Throughcitationsearch-es,Iidentifiedseveraltoolsdevelopedafterthereview,ofwhichIhavechosentousethelatest,developedbyMunnetal.(2014),asastartingpoint,basedonfacevalidityandsimplicity.WhileMunnetal.donotrefertothereviewbyShamliyanandcolleagues,theycomparetheirtooltoonedevelopedbyHoyetal.(2012)whodoconsidersaidreview.ThecriticalappraisaltooldevelopedbyMunnetal.includestenitems,withtheresponseoptions“Yes”,“No”,“Unclear”and“Notapplicable”:

1. Wasthesamplerepresentativeofthetargetpopulation?2. Werestudyparticipantsrecruitedinanappropriateway?3. Wasthesamplesizeadequate?4. Werethestudysubjectsandthesettingdescribedindetail?5. Wasthedataanalysisconductedwithsufficientcoverageoftheidenti-

fiedsample?6. Wereobjective,standardcriteriausedforthemeasurementofthecondi-

tion?7. Wastheconditionmeasuredreliably?8. Wasthereappropriatestatisticalanalysis?9. Areallimportantconfoundingfactors/subgroups/differencesidentified

andaccountedfor?10. Weresubpopulationsidentifiedusingobjectivecriteria?

Ofthetenitems,threeassessriskofbiasandarerelevantandapplicabletothisreview:items2,6and7.Item2assessesriskofselectionbias,whileitems6and7measuredetectionbias(aka.Informationbias).FocusingonthesetwotypesofbiasisconsistentwiththeStrengtheningtheReportingofObservationalStudiesinEpidemiology(STROBE)statement(Vandenbroucke,etal.,2007,p.814)andmethodsusedbytheCochraneMethodologyReviewGrouptosystematicallyreviewtheprevalenceofaconditioninapopulationofinformation,asopposedtopeople(Welch,etal.,2010,p.7).AsecondreviewerandIwillindependentlyassessriskofbiasineachstudy,cri-terionbycriterion,giventhatsomecriteriausedtomeasurethequalityofnewsreportsrequiremoresubjectivejudgementthanothers.Theseassessmentswillbeusedtodescribethestrengthsandweaknessesofthestudies.Ifdeemednec-essary,theywillalsobeusedtoconductasensitivityanalysis.Itisunclearhow


agreaterriskofbiaswouldimpacttheresultsofstudiestobeincludedinthisreview,soIhavenohypothesisabouttheresultsofsuchananalysis.Intermsofselectionbias,ifthenewsreportswererandomlyorsequentiallysampledfromthetargetpopulation,wewillconsidertheriskofbiasaslow.Ifthereportswerepurposivelysampled,wewillconsidertheriskofbiasashigh.Ifanyothersamplingtechniquewasused,wewillconsidertheriskofbiasasmoderateorhigh.Intermsofdetectionbias,ifacriterionrequiresminimaljudgement(e.g.acrite-rionthatrisksbepresentedinabsolutenumbers),wewillconsidertheriskofbiasaslow.Ifthecriterionrequiressubstantialjudgement(e.g.whetherthereissufficientinformationaboutpotentialharms),tworesearchersassessedthere-ports,andtheywereblindedtothejournalistandpublication,wewillconsidertheriskofbiasasmoderate.Ifthecriterionrequiressubstantialjudgement,andonlyoneresearcherassessedthereportsorresearchersassessingthereportswerenotblindedtothejournalistandpublication,wewillconsidertheriskofbiasashigh.Iftheriskofeitherselectionordetectionbiasishigh,wewillconsidertheover-allriskofbiasashigh.Iftheriskofeitherismoderate,wewillconsidertheriskasmoderate.Appendix6includesadummytablefortheriskofbiasassess-ments.Appendix7istheformthatwewillusetoassessriskofbias.Itincludesaspaceforrecordinganyconflictsofinterests.Anysuchconflictswillbedis-cussedseparatelywheninterpretingtheresultsofthereview.AnalysesCriteriausedtomeasurequalityHowqualityhasbeenmeasuredwillbeassessedbycomparingcriteriatotheIHCKeyConcepts(Box1).Eacheligiblecriterionfromeachstudywillbeen-teredintoaspreadsheet,diagonallyacrossfromthefullconceptlist(Appendix8).AsecondreviewerandIwilljudgewhichconceptsarecapturedbyeachcri-terion,representingthejudgementswithmarksinthespreadsheet.Wewilldis-cussanyagreementsandbringinathirdresearchertoarbitrate,ifnecessary.TheIHCKeyConceptsdifferfromotherlistsofcriteriaformakingcausalinfer-encesintwoways(Chalmers,etal.,2018,pp.30-31).First,thelistisdifferentinthatitisboth:a)developedusingasystematic,transparentanditerativepro-cess;andb)intendedtohelpchildren,thegeneralpublicandhealthpractition-ersmakeinformeddecisions,notjusthealthresearchers.Second,unlikeothertoolssuchaschecklistsandtipsheets,itisnotaninterventioninandofitself,limitedtoaparticularpopulationorcontext,butacomprehensiveframeworkthatcanbeusedformappingskills,developinginterventionsandevaluatingin-terventions.UsingtheIHCKeyConceptlisttoassessthecriteriausedtomeasurequalityofthenewsreportshasseveraladvantages.First,itprovidesaframeworkforgroupingcriteriafromdifferentstudies,soresultscanmoreeasilybesynthe-sised.Second,itwillshedlightonrelevantqualitiesthathavenottypicallybeenmeasuredormeasuredatall.Conversely,itmayrevealconceptscapturedby


thecriteriamissingfromtheIHCKeyConceptlist.Finally,usingthelistcanin-formtheinterpretationofresultsfromthisreview,includingwhattheresultssuggestaboutwhatconceptsaremostimportantwhendevelopinginterven-tionsforimprovingreportingontheeffectsofinterventions,orhelpingpeopleassessinformationinsuchreports.AsecondreviewerandIwilljudgewhatcriteriausedindifferentstudiescanbegrouped.Oncemore,wewilldiscussanydisagreementsandbringinathirdre-viewertoarbitrateifnecessary.QualityofnewsreportsAsfarasthedatapermits,wewillmeta-analysetheproportionsofsamplesthateithersatisfiedorfailedtosatisfyrespectivecriteria,i.e.theprevalenceofre-portssatisfyingorfailingtosatisfyagivencriterion.SeedummytablesinAp-pendix6.Wewilldichotomisecategoricaldatawherepossibleandsensible.Furthermore,wemayreframedatathatalreadyisdichotomousassatisfyingornotsatisfyingagivencriterion.Notsatisfyingacriterionwillincludeanythingotherthancompletelysatisfyingit.Inotherwords,ifnewsreports“partially”satisfyacriterionoritisunclearwhetherthereportssatisfyit,wewillconsiderthereportsasnothavingsatis-fiedthecriterion.Ifacriterionhasbeendeemedinapplicabletoreports,wewillexcludethosereportsfromthemeta-analysisforthatcriterion.Inadditiontopotentiallygivinguslargersamplesformeta-analysis,thedichotomisationwillmakeinterpretingtheresultsmorestraightforward.Acrossstudies,Ianticipatereportswillgenerallyfailtosatisfymostcriteria,de-spitedifferencesinpopulationsandmethods.Therefore,Iexpectmeanpropor-tionswith95%confidenceintervalswillbemeaningful.Iwillprepareforestplotsandvisuallyanalysetheextenttowhichtherearemeaningfuldifferencesbetweenpointestimates,andIwillconductaChi-squaretesttoseeiftheheter-ogeneityislargerthanonewouldexpectbychance.Foragivencriterion,ifthevisualinterpretationandChi-squaretestsuggestitisappropriate,Iwillconductameta-analysisusingarandomeffectsmodel.Iwillalsoconductsixsubgroupanalyses,whendataareavailable.VariationinqualityAgain,Idonotexpectalotofvariationinthequalityofreportsacrossstudies.Totheextentthatthereisvariation,Idonotexpectalotofdataforexploringit.Thatsaid,thereareseveralvariablesthatcouldlogicallyexplainvariation,evi-dencesupportingsomeofthosehypotheses,andreasonstotestthem.Table1isanoverviewof:thevariablesforwhichwewillconductsubgroupanalyses;thesubgroups;thehypothesiseddifferences;andtherationalesforthehypothesesandforconductingtheanalyses.Thevariablesare:themediuminwhichareportwaspublished;thetimeperiodinwhichareportwaspub-lished;whetherthepublishingoutletwascommercial;whetherthejournalistwasspecialisedinhealthorscience;andtheincomelevelofthecountryinwhichtheoutletissituated.


Ontheonehand,Iwillbecautiousaboutextrapolatingfindingsifthereisnodi-rectevidenceforasubpopulation(e.g.broadcastreports)andthereisreasontobeuncertainabouttheapplicabilityoftheevidencefromothersubpopulations(e.g.printandonlinereports).Ontheotherhand,subgroupanalysesarefre-quentlymisleading,asshownbySunetal.(2014).Toavoidspuriousresults,wewilluseexplicitcriteriaforassessingthecredibil-ityofanydifference,basedonSunandcolleagues’criteria:

• Thedifferenceisofpracticalimportance.• Thedifferenceisbiggerthanwewouldexpectbychance.• Thedifferenceisinthehypothesiseddirection.• Thedifferenceisunlikelytobeexplainedbyotherknownvariables.• Thedifferenceisconsistentacrossstudieswithdifferentmethods.• Thedifferenceisconsistentacrossstudieswithdifferentoutcome

measures.• Thedifferenceisconsistentacrossstudieswithdifferentlevelsofriskof

bias.Foreachexplanatoryvariable,givensufficientdata,wewillconductunivariateregressionanalyseswiththevariable(e.g.medium)astheindependentvaria-ble,andwiththeproportionofsubpopulations(e.g.reportsfromrespectivedecades)thatsatisfythecriterionasthedependentvariable.Wewillalsocon-ductmultipleregressionanalyseswithalloftheexplanatoryvariables.Wewillnoteanywithin-studysubgroupanalyseswheretheresearchershaveusedap-propriatestatisticalmethodsorprovidedthenecessarydataforustodoanap-propriatestatisticalanalysisourselves.Wewillconsiderthoseanalyseswhenmakingjudgementsaboutthecredibilityofsubgroupdifferences.Ihavedecidedagainstconductingasubgroupanalysisforthetypeofreport,i.e.whetherthereportisafeatureorabreakingnewsreport.Featuresaretypicallylongerinwordsandpreparedonlongerdeadlines.Theymightlogicallybehigherqualitythanbreakingnewsreports.However,ananalysisbytypeofre-portwouldlikelybeconfoundedbymedium,financialmodel,andspecialisation.Furthermore,howresearchershavedefinedfeatureversusbreakingnews,andhowtheyhavemeasuredlengthofreportordeadline,maybesodifferentacrossstudiesthatcombiningthedatawouldnotmakesense.


Table1:SubgroupanalysesVariable Grouping Hypothesis RationaleandimportanceMedium Broadcastvs.other(print

andonline)Broadcastreportsarelesslikelytosatisfycriteria

Broadcastallowsforfewerwords,aswellaslesstimeforwriting,giventimespentrecordingandediting.Thiswouldlogicallyhaveanegativeimpactonthequalityofreports.Iamawareofempiricalevidencethatsupportsthishypothesis(Walsh-Childers,etal.,2016).Acredibledifferenceinthehypothesiseddirectionwouldimplyconsumersshouldpaylessattentiontobroadcastreports.

Timeperiod Decades(≥2000;1990-1999;1980-1989;etc.)

Theproportionofreportssatisfyingcriteriaislikelytobesimilaracrossdecades

Ontheonehand,newsreportsmayhaveimprovedsincetheadventof“evidence-basedmedicine”(EBM)(Evidence-BasedMedicineWorkingGroup,1992).Ontheother,therearesustained,systemicbarrierstoreliablehealthjournalismsuchaslackoftime(Larsson,etal.,2003;Pettersen,2005),whichEBMdoesnotaddress.Moreover,revenuelossesmayhavelednewsmediatoinvestlessinstrategiesthatmightimprovethequalityoftheirhealthnews,suchastrainingjournalistsandeditors,andhiringandretainingspecialisedhealthandsciencereporters.However,acredibleimprovementforoneormorecriteriawouldbeanimpetusforresearchtoexplainsaidimprovement,andthatresearchcouldinformthedevelopmentofinterventionstoimprovequalityoverall.

Financialmodel Commercialvs.non-commercial

Reportspublishedbycommercialoutletsarelesslikelytosatisfycrite-ria

Thefinancialinterestsofcommercialoutletsmayleadtosensationalism,toattractaudiences(Larsson,etal.,2003;Pettersen,2005;Wallington,etal.,2010).Moreover,companiesthatsellhealthinterventionsmaybeadvertisers,disincentivisingcriticalreporting.Acredibledifferenceinthehypothesiseddirectionwouldimplyconsumersshouldsupportnon-commercialoutletsandpaylessattentiontoreportspublishedbycommercialoutlets.

Specialisation Healthandsciencejour-nalistsvs.other

Reportspreparedbyhealthorsci-encejournalistsaremorelikelytosatisfycriteria

Journalistwhohavespecialisedinhealthorsciencearemorelikelytohavereceivedtrainingincriticalappraisal.Moreover,comparedtootherjournalists,theymayhavebeeninmorecontactwithscientists,andmaybemorefamiliarwiththelanguage,contentandstructureofresearchpapers.Thereisevidencethatjournalistswithmoreadvanceddegreesaremorelikelytousescientificjournalarticlesassources,andthatjournalistswithfeweryearsofexperiencearemorelikelytosayprovidingenter-tainmentisanimportantpriorityforhealthnews(Wallington,etal.,2010).Acredibledifferenceinthehypothesiseddirectionwouldimplyconsumersshouldpaymoreattentiontoreportspreparedbyspecialisedhealthandsciencejournalists.

Countryincomelevel

Low-incomecountriesvs.other(middleandhigh-incomecountries)

Theproportionofreportssatisfyingcriteriaislikelytobesimilaracrosscountrieswithdifferentincomelevels

Whilenewsmediainmiddleandhigh-incomecountriesmayhavemoreresources,journalistsinthosecountriesstillfacesys-temicbarrierstohigh-qualityreporting(Larsson,etal.,2003;Pettersen,2005;Wallington,etal.,2010).Supportforthishypoth-esiswouldsuggestresearchersshoulddevelopinterventionsthatcanimprovethequalityofreportinginlow-incomecountriesandmiddleorhigh-incomecountriesalike.Itwouldalsoimplythoseinterventionsaddressbarriersbesidelackofresources.

Newspapertype Broadsheetvs.tabloid Reportsinbroadsheetnewspapersaremorelikelytosatisfycriteria

Reportingintabloidnewspapersmaybelessdetailedandmoresensationalthanreportinginbroadsheetnewspapers,andIamawareofevidencethatthequalityofreportsabouttheeffectsofhealthinterventionsislowerintabloids(Entwistle&Hancock-Beaulieu,1992;Robinson,etal.,2013).Acredibledifferenceinthehypothesiseddirectionwouldimplyconsumersshouldpaymoreattentiontobroadsheetnewspapers,andlesstotabloids.


PilotHavingconductedthesearch,wewillpilotforms,spreadsheetsandtables(Ap-pendixes3through6)onthefirstfiveeligiblestudies.Wewilldiscussanyis-suesandmakeanyrevisionsdeemednecessarybeforecompletingdataextrac-tion,riskofbiasassessmentsandanalysesforremainingstudies.RatingthequalityoftheevidenceIhaveadjustedtheGradingofRecommendationsAssessment,DevelopmentandEvaluation(GRADE)approachtoratingthequalityofevidence(Guyatt,etal.,2008).TheGRADEapproachinvolvesconsideringfivefactorsforloweringthecertaintyofevidence:studylimitations,inconsistencyofresults,indirectnessofevidence,imprecision,andpublicationbias.Ihaveadjustedthese,giventhatGRADEwasdevelopedforresearchabouteffects,notprevalence.Appendix9istheformwewillusetoratethequalityoftheevidence.Thefirstadjustmentisthatwewillnotconsiderpublicationbias.Iamnotawareofanyresearchdocumentingpublicationbiasforprevalenceorincidencestudiesgenerally,norspecificallyinthisarea.Besides,Iexpecttoofewstudiesandsamplesthataretoosmallforittobemeaningfultoassesstheriskofpubli-cationbias.Second,wewillnotconsiderdirectnesssinceitisinapplicable.Iwill,however,becautiousaboutextrapolatingevidenceacrosssubpopulationsspecifiedinTable1(e.g.acrossreportsfromdifferenttimeperiods),ifthereisnoevidenceforoneofthesubpopulationsspecific(e.g.reportsfrombeforeaparticulardec-ade).Thisleavesconsideringriskofbias,imprecisionandinconsistency.Intermsofimprecision,wewillusequartilesasaruleofthumb;iftheconfidenceintervalsubstantiallycrossestwoormorequartiles,wewillconsidertheretobeim-portantimprecision,followingguidancedevelopedbytheCochraneEffectivePracticeandOrganisationofCare(EPOC)group(CochraneEffectivePracticeandOrganisationofCare,2017).GRADEfactorsforincreasingthecertaintyoftheevidencearenotrelevanthere.


Ethicalconsiderationsandconflictsofinterest

Thereviewdoesnotinvolveparticularethicalchallenges.IdeclarethatIhavenorelevantconflictsofinterest.


References

Ashorkhani,M.etal.,2012.Qualityofhealthnewsdisseminatedintheprintmediaindevelopingcountries:AcasestudyinIran.BMCPublicHealth,12(627).Austvoll-Dahlgren,A.etal.,2017.Measuringabilitytoassessclaimsabouttreatmenteffects:alatenttraitanalysisofitemsfromthe‘ClaimEvaluationTools’databaseusingRaschmodelling.BMJOpen,Volume7,p.e013184.Austvoll-Dahlgren,A.,Nsangi,A.&Semakula,D.,2016.Interventionsandassessmenttoolsaddressingkeyconceptspeopleneedtoknowtoappraiseclaimsabouttreatmenteffects:asystematicmappingreview.SystematicReviews,5(215).Austvoll-Dahlgren,A.etal.,2015.Keyconceptsthatpeopleneedtounderstandtoassessclaimsabouttreatmenteffects.JournalofEvidence-BasedMedicine,Volume8,pp.112-125.Austvoll-Dahlgren,A.etal.,2016.Measuringabilitytoassessclaimsabouttreatmenteffects:thedevelopmentofthe‘ClaimEvaluationTools’.BMJOpen,Volume7,p.e013184.Berkman,N.D.etal.,2011.Lowhealthliteracyandhealthoutcomes:Anupdatedsystematicreview.AnnalsofInternalMedicine,155(2),pp.97-107.Brownlee,S.etal.,2017.Evidenceforoveruseofmedicalservicesaroundtheworld.Lancet,390(10090),pp.156-168.Castell,S.etal.,2014.PublicAttitudestoScience2014,London:IpsosMORI.Chalmers,I.etal.,2018.KeyConceptsforInformedHealthChoices:aframeworkforhelpingpeoplelearnhowtoassesstreatmentclaimsandmakeinformedchoices.BMJEvidence-BasedMedicine,23(1),pp.29-33.Charnock,D.,Shepperd,S.,Needham,G.&Gann,R.,1999.DISCERN:aninstrumentforjudgingthequalityofwrittenconsumerhealthinformationontreatmentchoices.JournalofEpidemiologyandCommunityHealth,Volume53,pp.105-111.Chew,C.&Eysenbach,G.,2010.PandemicsintheAgeofTwitter:ContentAnalysisofTweetsduringthe2009H1N1Outbreak.PLoSONE,5(11),p.e14118.CochraneEffectivePracticeandOrganisationofCare,2017.ReportingtheeffectsofaninterventioninEPOCreviews.s.l.:CochraneEffectivePracticeandOrganisationofCare.Coulter,A.,Entwistle,V.&Gilbert,D.,1999.Sharingdecisionswithpatients:istheinformationgoodenough?.BMJ,Volume318,pp.318-322.


Culver,J.D.,Gerr,F.&Frumkin,H.,1997.MedicalinformationontheInternet:Astudyofanelectronicbulletinboard.JournalofGeneralInternalMedicine,12(8),pp.466-470.Entwistle,V.&Hancock-Beaulieu,M.,1992.HealthandmedicalcoverageintheUKnationalpress.PublicUnderstandingofScience,Volume1,pp.367-382.Evidence-BasedMedicineWorkingGroup,1992.Evidence-BasedMedicine:ANewApproachtoTeachingthePracticeofMedicine.JAMA,268(17),pp.2420-2425.Faerber,A.E.&Kreling,D.H.,2013.ContentAnalysisofFalseandMisleadingClaimsinTelevisionAdvertisingforPrescriptionandNonprescriptionDrugs.JournalofGeneralInternalMedicine,29(1),pp.110-118.Frosch,D.L.etal.,2007.CreatingDemandforPrescriptionDrugs:AContentAnalysisofTelevisionDirect-to-ConsumerAdvertising.AnnalsofFamilyMedicine,5(1),pp.6-13.Frosch,D.L.,May,S.G.,Tietbohl,C.&Pagán,J.A.,2011.Livinginthe“landofno”?Consumerperceptionsofhealthylifestyleportrayalsindirect-to-consumeradvertisementsofprescriptiondrugs.SocialScience&Medicine,Volume73,pp.995-1002.Glasziou,P.etal.,2017.Evidenceforunderuseofeffectivemedicalservicesaroundtheworld.Lancet,390(10090),pp.169-177.Glenton,C.,Paulsen,E.J.&Oxman,A.D.,2005.PortalstoWonderland:Healthportalsleadtoconfusinginformationabouttheeffectsofhealthcare.BMCMedicalInformaticsandDecisionMaking,5(7).Groven,K.S.&Braitwaite,J.,2016.Happily-ever-after:Personalnarrativesinweight-losssurgeryadvertising.HealthCareforWomenInternational,37(11),pp.1221-1238.Guyatt,G.H.etal.,2008.GRADE:whatis“qualityofevidence”andwhyisitimportanttoclinicians?.BMJ,Volume336,pp.995-998.Higgins,J.&Green,S.eds.,2011.Definingthescopeofareviewquestion(broadversusnarrow).In:CochraneHandbookforSystematicReviewsofInterventions.s.l.:TheCochraneCollaboration.HLS-EUConsortium,2013.ComparativereportofhealthliteracyineightEUmembersstates,s.l.:s.n.Hoffmann,T.C.&DelMar,C.,2015.Patients’ExpectationsoftheBenefitsandHarmsofTreatments,Screening,andTests:ASystematicReview.JAMAInternalMedicine,175(2),pp.274-286.Hoffmann,T.C.&DelMar,C.,2017.Clinicians’ExpectationsoftheBenefitsandHarmsofTreatments,Screening,andTests:ASystematicReview.JAMAInternalMedicine,177(3),pp.407-419.Hoy,D.etal.,2012.Assessingriskofbiasinprevalencestudies:modificationofanexistingtoolandevidenceofinterrateragreement.JournalofClinicalEpidemiology,Volume65,pp.934-939.


Kable,A.K.,Pich,J.&Maslin-Prothero,S.E.,2012.Astructuredapproachtodocumentingasearchstrategyforpublication:A12stepguidelineforauthors.NurseEducationToday,Volume32,pp.878-886.Kutner,M.,Greenberg,E.,Jin,Y.&Paulsen,C.,2006.TheHealthLiteracyofAmerica’sAdults:ResultsFromthe2003NationalAssessmentofAdultLiteracy,Washington,DC:NationalCenterforEducationStatistics.Larsson,A.,Oxman,A.D.,Carling,C.&Herrin,J.,2003.Medicalmessagesinthemedia–barriersandsolutionstoimprovingmedicaljournalism.HealthExpectations,Volume6,pp.323-331.Marcon,A.R.,Murdoch,B.&Caulfield,T.,2017.Fakenewsportrayalsofstemcellsandstemcellresearch.RegenerativeMedicine,12(7),pp.765-775.Moher,D.etal.,2015.Preferredreportingitemsforsystematicreviewandmeta-analysisprotocols(PRISMA-P)2015statement.Systematicreviews,4(1).Moult,B.,Franck,L.S.&Brady,H.,2004.EnsuringQualityInformationforPatients:developmentandpreliminaryvalidationofanewinstrumenttoimprovethequalityofwrittenhealthcareinformation.HealthExpectations,Volume7,pp.165-175.Moynihan,R.etal.,2000.Coveragebythenewsmediaofthebenefitsandrisksofmedications.TheNewEnglandJournalofMedicine,342(22),pp.1645-1650.Munn,Z.etal.,2015.Methodologicalguidanceforsystematicreviewsofobservationalepidemiologicalstudiesreportingprevalenceandcumulativeincidencedata.InternationalJournalofEvidence-BasedHealthcare,Volume13,pp.147-153.Munn,Z.,Moola,S.,Riitano,D.&Lisy,K.,2014.Thedevelopmentofacriticalappraisaltoolforuseinsystematicreviewsaddressingquestionsofprevalence.InternationalJournalofHealthPolicyandManagement,3(3),pp.123-128.Nsangi,A.etal.,2017.EffectsoftheInformedHealthChoicesprimaryschoolinterventionontheabilityofchildreninUgandatoassessthereliabilityofclaimsabouttreatmenteffects:acluster-randomisedcontrolledtrial.Lancet,390(10092),pp.374-388.Optum,2013.Effectivenessofhomeopathyforclinicalconditions:Evaluationoftheevidence,s.l.:Optum.O'Sullivan,J.W.etal.,2018.Overtestingandundertestinginprimarycare:asystematicreviewandmeta-analysis.BMJOpen,Volume8,p.e018557.Oxman,A.D.,Austvoll-Dahlgren,A.,Garrat,A.&Rosenbaum,S.,2017.Understandingofkeyconceptsrelevanttoassessingclaimsabouttreatmenteffects:asurveyofNorwegianadults.IHCWorkingPaper.Oslo:InformedHealthChoices.Oxman,A.D.etal.,1993.AnIndexofScientificQualityforHealthReportsintheLayPress.JournalofClinicalEpidemiology,46(9),pp.987-1001.Pettersen,S.,2005.CriticalThinkinginNorwegianUpperSecondaryBiologyEducation:TheCasesofComplementary-Alternative-MedicineandHealthClaimsintheMedia.Nordina,1(2),pp.61-71.


Pettersen,S.,2005.Norwegianhealthjournalists’abilitytoreportonhealthresearch:Aconcerntoscienceeducation?.Nordina,1(1),pp.5-16.Pettersen,S.,2007.HealthClaimsandScientificKnowledge:Astudyofhowstudentsofhealthsciences,theirteachers,andnewspaperjournalistsrelatetohealthclaimsinsociety.Oslo:UniversityofOslo.Robinson,A.,Coutinho,A.,Bryden,A.&McKee,M.,2013.AnalysisofhealthstoriesindailynewspapersintheUK.PublicHealth,Volume127,pp.39-45.Sansgiry,S.,Sharp,W.T.&Sansgiry,S.S.,1999.AccuracyofInformationonPrintedOver-the-CounterDrugAdvertisements.HealthMarketingQuarterly,17(2),pp.7-18.Semakula,D.etal.,2017.EffectsoftheInformedHealthChoicespodcastontheabilityofparentsofprimaryschoolchildreninUgandatoassessclaimsabouttreatmenteffects:arandomisedcontrolledtrial.Lancet,390(10092),pp.389-398.Shamliyan,T.,Kane,R.L.&Dickinson,S.,2010.Asystematicreviewoftoolsusedtoassessthequalityofobservationalstudiesthatexamineincidenceorprevalenceandriskfactorsfordiseases.JournalofClinicalEpidemiology,Volume63,pp.1061-1070.Shoemaker,P.J.&Cohen,A.A.,2006.Part1:TheoryandMethod.In:P.J.Shoemaker&A.A.Cohen,eds.NewsAroundtheWorld.NewYork:Routledge,pp.1-34.Smith,D.E.,Wilson,A.J.&Henry,D.A.,2005.Monitoringthequalityofmedicalnewsreporting:earlyexperiencewithmediadoctor.MJA,Volume183,pp.190-193.Spencer,E.A.,Mahtani,K.R.,Goldacre,B.&Heneghan,C.,2016.Claimsforfertilityinterventions:asystematicassessmentofstatementsonUKfertilitycentrewebsites.BMJOpen,Volume6,p.e013940.Starr,R.R.,2015.TooLittle,TooLate:IneffectiveRegulationofDietarySupplementsintheUnitedStates.AmericanJournalofPublicHealth,105(3),pp.478-485.Stephens,M.,2007.AHistoryofNews.3rdEditioned.NewYork:OxfordUniversityPress.Sun,X.etal.,2014.HowtoUseaSubgroupAnalysis:Users'GuidestotheMedicalLiterature.JAMA,311(4),pp.405-411.TheAcademyofMedicalSciences,2016.TheAcademyofMedicalSciences:MedicalInformationSurvey,London:TheAcademyofMedicalSciences.TheJoannaBriggsInstitute,2014.Thesystematicreviewofprevalenceandincidencedata.Adelaide:TheJoannaBriggsInstitute.UnitedStatesGovernmentAccountabilityOffice,2010.HerbalDietarySupplements:ExamplesofDeceptiveorQuestionableMarketingPracticesandPotentiallyDangerousAdvice,s.l.:UnitedStatesGovernmentAccountabilityOffice.


Vandenbroucke,J.P.etal.,2007.StrengtheningtheReportingofObservationalStudiesinEpidemiology(STROBE):Explanationandelaboration.Epidemiology,18(6),pp.805-835.Wallington,S.F.,Blake,K.,Taylor-Clark,K.&Viswanath,K.,2010.AntecedentstoAgendaSettingandFraminginHealthNews:AnExaminationofPriority,Angle,Source,andResourceUsagefromaNationalSurveyofU.S.HealthReportersandEditors.JournalofHealthCommunication,15(1),pp.76-94.Walsh-Childers,K.,Braddock,J.,Rabaza,C.&Schwitzer,G.,2016.OneStepForward,OneStepBack:ChangesinNewsCoverageofMedicalInterventions.HealthCommunication,33(2),pp.174-187.Welch,V.etal.,2010.Howeffectsonhealthequityareassessedinsystematicreviewsofinterventions(Review).CochraneDatabaseofSystematicReviews,Issue12.Wolfe,R.M.,Sharp,L.K.&Lipsky,M.S.,2002.ContentandDesignAttributesofAntivaccinationWebSites.JAMA,287(24),pp.3245-3248.Woloshin,S.,Schwartz,L.M.&Welch,H.G.,2008.Knowyourchances:understandinghealthstatistics.Berkeley:UniversityofCaliforniaPress.Zeraatkar,D.,Obeda,M.,Ginsberg,J.S.&Hirsh,J.,2017.Thedevelopmentandvalidationofaninstrumenttomeasurethequalityofhealthresearchreportsinthelaymedia.BMCPublicHealth,17(343).

Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am...

Documents

Transcript of Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am...