The challenge for all of us informaon Checking updates3 Asian and Pacific Rim Research Integrity...
Transcript of The challenge for all of us informaon Checking updates3 Asian and Pacific Rim Research Integrity...
1
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
Thechallengeforallofus
PloSMedicine2005doi:10.1371/journal.pmed.0020124
DavidDobbs,TheAtlan<c,18thSEPT2015
“Fi>y-threepapersweredeemed'landmark'studies.…Nevertheless,scienKficfindingswereconfirmedinonly6(11%)cases.”Nature483,531–533(2012)doi:10.1038/483531a
2
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
ThechallengeforjournalsRetrac<onsareevermorefrequentandscalewithimpactfactor
Source: Ferric C. Fang, F. C. & Casadevall, A. Retracted Science and the Retraction Index.
Infect. Immun. 79, 3855-3859 (2011).
3
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
ThechallengeforjournalsCorrec<onsareontherise,too
0
20
40
60
80
100
120
140
Publishe
dpe
ryear
Correc%ons&retrac%onsinNaturejournals
Corrigendum
Retrac<on
4
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.ThereforeyoumayneedtocheckandmanuallyupdateotherslidesWearenottalkingaboutresultsthatwere:
• Falsified—misconductisaproblembutit’ss<llintheminority• Wrong—legi<mateobserva<onsbutsubsequentworkdisprovesthe
hypothesis—that’showscienceismeanttowork!Wearetalkingaboutresultsthatare• Poorlydescribedpreven%ngverifica%on—independentexperimenter
cannotobservethesameresultsundersimilarcondi<ons.• Overstated—failuretoconsideralterna<veexplana<ons• Misrepresented—dataclaimedtobemorerobustthantheyactuallyare,
CHERRYPICKING!• Sloppy—failuretoaccountfor(orevenconsider)sourcesoferror,pooruse
ofsta<s<cs,poorcontrols
Thebiggestproblemisn’tfraudAlthoughthelionshareofretrac<onsarisefrommisconduct,poorreproducibilitydoesnot
5
So,whatarethesolu%ons?
6
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
Partofthesolu%oniseduca%onWemustn’ttakeitforgrantedthateveryoneknowswhatgoodresearchis
Casadevall,A.andFang,F.C.RigorousScience:aHow-ToGuide.mBio7,e01902-16(2016);doi:10.1128/mBio.01902-16
APentateuchforScien%ficRigour
7
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
Upda%ngimage
Toupdatethebackgroundimage,clickonthepictureplaceholdericon
1. WindowsExplorerorFinderwillopen
2. Findyourrequiredimageandclick‘Insert’
3. Whencoverimageinserted,right-clickinthegreyareadirectlybelowthistextboxandselect‘Resetslide’
NOTE:
Ifanimageisalreadyinplaceandyouwanttochangeit:
1. Selecttheimage,andhityourdeletekey
2. Followsteps1-4above
Encouragecommunitystandards
• Rapidimprovementintheefficiencyofperovskitesolarcellsleadtoasurgeinclaimsofeverhigherperformance.
• Extraordinaryclaimsbasedonpoorexperimentaldesignleadtodemandsfromrefereesthatallclaimsbesubmiiedtothirdpartyverifica<onbeforepeerreview.
• Independentverifica<onbeforepeerreview,enforcedbyjournals,isnowstandardprac<ce.
Casestudy:perovskitesolarcells
NaturePhotonics8,665(2014);doi:10.1038/nphoton.2014.212
8
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
Don’tciteanythingyouhaven’tread!Greenberg,S.A.Howcita<ondistor<onscreateunfoundedauthority:analysisofacita<onnetwork.BMJ339,b2680(2009);doi:10.1136/bmj.b2680
“Fig.5—ConversionofhypothesistofactthroughcitaKonalone.CitaKonsonstatementthataccumulaKonofβamyloid“precedes”otherabnormaliKesininclusionbodymyosiKsmuscle.StatementasfactissupportedthroughcitaKontopapersthatonlystateitashypothesis(forexample,references5to80,91to80,134to74)orsomeKmessupportedbycitaKontopapersthatcontainnostatementsaddressingit(forexample,references91to72,251to75;deadendcitaKons).ThisphenomenonmightbecalledcitaKontransmutaKon.”
©2009 by British Medical Journal Publishing Group
9
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
BeTermethodssec%ons,longermethodssec%ons
Eliminatelengthlimitsformethodssec<onsextendedmethodsonlineEncouragedtosubmit
MethodstoProtocolExchange
overepor<ngstandardschecklist
Increaserigourofsta<s<csSta<s<caladvisorsta<s<calconsultantsEduca<on–NatureMethodscolumn‘PointsofSignificance’
Datasharingreaffirmdatasharingpolicy–condiKonofpublica<ontowardsdatacita<onsimplementedsourcedataScien<ficData–publica<onofdatadescriptorstofacilitatereuseandcredit
Bi-direc<onallinksbetweenProtocolsandAr<cles
Neteffect:• Lengthincreasedbyaverage25%upto50%atsomejournals• Methodsfullyintegratedwithar<cle
online–htmlandPDFNomore“supplementarymethods”
10
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
BeTermethodssec%ons,longermethodssec%ons
CellPresship://www.cell.com/star-methods
11
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.ThereforeyoumayneedtocheckandmanuallyupdateotherslidesIregularlytellyoungresearchersthatonewaytodecidewhether
somethingitethicalornotistoaskthemselves,
“IfthisbehaviourorpracKceweretobecomepublicknowledge,wouldIhaveanyreasontofeeluncomfortableorashamedbyit?”
Andiftheansweris‘yes’oreven‘maybe’,don’tdoit!
Themainpointhereisthatpoorprac<cecanonlythriveinthedark.Andgoodprac%cecomesbydoingeverythingintheopen.
Thean%sep%cpowerofsunlight
12
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
Moretransparentrepor%nghip://www.nature.com/authors/policies/checklist.pdf
13
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
Encouragemoresharingofdata
14
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
SharingplaWormsneedn’tjustbeaboutsharing(oratall)Ifyourdataisn’tsavedinasecure,persistentrepositoryitmayaswellnotexist
A cloud based research data management system where you can:
Manage your research outputs privately and securely
Make your research outputs citable & discoverable
15
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides• Atleast28researchfundersgloballyhave
policiesormandatesrequiringarchivingofdataasacondi<onofgrants,including:• Na<onalScienceFounda<on(NSF)• Na<onalIns<tutesofHealth(NIH)• WellcomeTrust• BillandMelindaGatesFounda<on
• Someoftheserequiredatatobelinkedtopublica<onsincluding:• ResearchCouncilsUK(aspartofopenaccesspolicy)• EngineeringandPhysicalSciencesResearchCouncil(EPSRC)
Source:Hahnel,M:Globalfunderswhorequiredataarchivingasacondi<onofgrants.Figshare.hips://dx.doi.org/10.6084/m9.figshare.1281141.v1(2015)
Whysharedata?Becausefundersincreasinglyrequireit
16
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
16
Whysharedata?
“Publishingresearchwithoutdataisadver5sing,notscience!”
—CameronNeylon,GrahamSteel,andothers.
Becausethat’swhatscienceshouldbeabout
17
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
18
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
Upda%ngimage
Toupdatethebackgroundimage,clickonthepictureplaceholdericon
1. WindowsExplorerorFinderwillopen
2. Findyourrequiredimageandclick‘Insert’
3. Whencoverimageinserted,right-clickinthegreyareadirectlybelowthistextboxandselect‘Resetslide’
NOTE:
Ifanimageisalreadyinplaceandyouwanttochangeit:
1. Selecttheimage,andhityourdeletekey
2. Followsteps1-4above
OpendatainChina
“TheChinesegovernmenthasinvestednearly200million
yuan(US$30million)since2006onaplacormforsharing
corrosiondatafrom30field-tesKngstaKonscoveringstandardmaterialsin
environments(air,soilandwater)typicalofdifferentparts
ofthecountry.”—Li,X.G.etal.Nature527,441–442(2015).
Casestudy:China’sopencorrosiondataplaqorm
19
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
19
Butistheworldreadytoopenuptheirdata?
HasopendataarrivedattheBri5shMedicalJournal(BMJ)?Anobserva%onalstudyObjec%veToquan<fydatasharingtrendsanddatasharingpolicycomplianceattheBriKshMedicalJournal(BMJ)byanalysingtherateofdatasharingprac<ces,andinves<gateartudesandexaminebarrierstowardsdatasharing.Results…only7/157researchar<clessharedtheirdatasets,4.5%…ConclusionsDespitetheBMJ'sstrongdatasharingpolicy,sharingratesarelow.Possibleexplana<onsforlowdatasharingratescouldbe:thewordingoftheBMJdatasharingpolicy…requests[ending]upinresearchersspamfolders;andthatresearchersarenotrewardedforsharingtheirdata.—AnisaRowhani-Farid&AdrianGBarneiBMJOpen6,e011784(2016).
Orevenshareitonrequest?
20
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.ThereforeyoumayneedtocheckandmanuallyupdateotherslidesInMarch2016weranapilotrequiringauthors
ofpapersatfiveNaturejournalstodeclarehowreaderscanobtainthe‘minimaldataset’neededtointerpret,replicateandbuildonthefindingstheyreport.Bytheendof2016werolledthisouttoalljournalsintheNaturefamily.Aswellasbeinganinherentgood,wehopeto:• Learnmoreabouthowdifferentdisciplines
sharedatadifferently.• Learnhowwecanhelpthemmeetthe
challengesthey.• Raiseawarenessandencouragemoreresearchintotheopen.
EncouragemoresharingofdataNaturenowrequiresauthorstotellreadershowtheycangetholdoftheirdata
Nature536,138(20016).Tolearnmoregotohip://go.nature.com/2bf4vqn
21
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
Andnotjustdata—materialsarejustasimportant!
“AcondiKonofpublicaKoninaNaturejournalisthatauthorsarerequiredtomakeuniquematerialspromptlyavailabletootherswithoutunduequalificaKons.”hTp://www.nature.com/authors/policies/availability.html
“A>erpublicaKon,alldataandmaterialsnecessarytounderstand,assess,andextendtheconclusionsofthemanuscriptmustbeavailabletoanyreaderofScience.”hTp://www.sciencemag.org/authors/science-editorial-policies?_ga=1.200322094.1663650950.1484202254
“OneofthetermsandcondiKonsofpublishingwithCellPressisthatauthorsbewillingtodistributeanymaterialsandprotocolsusedinthepublishedexperimentstoqualifiedresearchersfortheirownuse.”hTp://www.cell.com/cell/authors
22
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
WhatisPreregistra5on?“PreregistraKonseparateshypothesistesKngfromhypothesisgeneraKngresearch.Bothareimportant.However,thesamedatacannotbeusedtogenerateandtestahypothesis,whicho>enhappensunintenKonally.WithpreregistraKon,confirmatoryanalysesareplannedinadvanceinordertoretainthevalidityoftheirstaKsKcalinferences,andexploratoryanalysesarereportedasposthocinvesKgaKonsthatmightinspireconfirmatorytestsinfuturestudies.”—fromhips://cos.io/prereg/
Andfinally… pre-registra%on
23
AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017
Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdatesmaynotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides
23 Upda%ngfooterToupdatethefooter:• Clickintothetextboxontheslide
masterpageandupdatetheinforma<on
Checkingupdates• Thetextupdateswillnotflow
throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtomanuallyupdateothertextboxessuchasthedividerslidesetc
Thankyou!谢谢!
EdGerstner印格致 —[email protected]
Furtherreading• ‘TurningPoint’whitepaperonChineseresearchlandscape(2015)—
hip://www.nature.com/press_releases/turning_point.pdf• JaneQiu,SafeguardingresearchintegrityinChina.NatlSci.Rev.2,
122–125(2015);doi:10.1093/nsr/nwv002• Munafò,M.R.etal.Amanifestoforreproduciblescience.Nature
HumanBehaviour1,0021(2017);doi:10.1038/s41562-016-002
“Good Research Practice”
February 20, 2017 Jun Fudano, Ph.D.
(Tokyo Institute of Technology)
Three senses of “Goodness”
2
Traditional Theories of Ethics
3
Agent → Conduct → Consequences
Virtue Ethics
Deontological Ethics
Utilitarian Ethics
Line Drawing/Demarcation Problem
4
“How good is good enough for what purpose, for whom?”
“How good is good enough?”
The same line for every discipline?
Well-being Model 1
The first and fundamental cannon of engineering ethics
“Hold paramount the safety, health,
and welfare of the public”
What is welfare/well-being/happiness?
Two aspects of Ethics
Aspire to do Ought not to do
Aspirational Ethics Preventive Ethics
Positive Negative
External orientation Internal orientation
Tend to be uplifting Tend to be intimidating
7
Contact
Jun Fudano, Ph.D. Professor Institute for Liberal Arts Tokyo Institute of Technology W-9 63 2-12-1 Ookayama Meguro-ku, Tokyo 152-8552, JAPAN Tel.:+81-3-5734-2370 Fax:+81-3-5734-2844 E-mail: [email protected]
APRI, Prof. Mai Har Sham, The University of Hong Kong
Quantity, Quality and Quality
• Publish or Perish - Quantity • Impact factors and citations - Quality • Research impact - Excellent Quality • Promotions and Ranking
Academic Excellence: Quality Research is Responsible Research
APRI, Prof. Mai Har Sham, The University of Hong Kong
Research Data Management
Researcher: • Data management plan • Reliable data collection, usage and
interpretation • Practice good data recording • Data sharing
Institution: • Research data management policy • Data storage and archive