The Good, the Bad, and the Ugly: What Retractions Tell Us About Scientific

Transparency

2011 CrossRef Annual Member MeetingNovember 15, 2011

Ivan Oransky, MDExecutive Editor, Reuters HealthCo-Founder, Retraction Watch

http://retractionwatch.com

Retractions on the Rise

-The Wall Street Journal

Retractions on the Rise

Retractions on the Rise

-Nature

Retractions on the Rise

-Neil Saunders

The Unofficial Record Holder

Which Journals Retract?

-Infection and Immunity 2011

Which Journals Retract?

Why Do Journals Retract?

-Journal of Medical Ethics 2010

Why Do Journals Retract?

• Error is more common than fraud• 73.5% of papers were retracted for error (or an

undisclosed reason) vs 26.6% for fraud• Most common reason for retraction: a scientific

mistake (234 papers; 31.5%)• Fabrication (including data plagiarism) more

common than text plagiarism • Multiple reasons for retraction cited for 67

papers (9.0%), but 134 papers (18.1%) were retracted for ambiguous reasons

-Journal of Medical Ethics 2010

Duplication

Publisher Error

Plagiarism

Legal Reasons

Lack of IRB Approval

Authorship Issues

Fraud: Image Manipulation

Fraud: Faked Data

Not Reproducible

What Happens to Retracted Papers?

-Assn of College & Research Libraries 2011

What Happens to Retracted Papers?

Budd et al, 1999: • Retracted articles received more than 2,000 post-

retraction citations; less than 8% of citations acknowledged the retraction

• Preliminary study of the present data shows that continued citation remains a problem

• Of 391 citations analyzed, only 6% acknowledge the retraction

This is Transparency?

This is Transparency?

This is Transparency?• ‘important irregularities’ Well, if they’re

important irregularities, why don’t you tell us what they are?

• ‘the authors ‘no longer stand by their results’ Are they standing somewhere else in the lab? C’mon, tell us why they can’t stand by the results anymore.

• ‘incorrect data were found to have been included on the study Case Report Forms’ Paging Dr. Kafka.

This is Transparency?• ‘figure withdrawn due to lack of supporting data’

“Someone seems to have made this up.”

• ‘Retraction…is being done for legal reasons based on the advice of counsel’ We’d comment on this, but we’d probably get sued.

• ‘Numerous errors in the text and references… were not discovered until after publication, although neither novel ideas nor data were misappropriated’ As journalism error maven Craig Silverman would say on RegretTheError.com, “Rest is fine.”

This is Transparency?

This is Transparency?In this Letter we made errors in representative image choice, including mislabelling of images or choosing an image from the inappropriate genotype. In all cases, choice of images was completely independent of the data analysis and so none of the conclusions in our original Letter are affected. We apologise for any confusion these errors may have caused.

This is Transparency?In this Letter we made errors in representative image choice, including mislabelling of images or choosing an image from the inappropriate genotype. In all cases, choice of images was completely independent of the data analysis and so none of the conclusions in our original Letter are affected. We apologise for any confusion these errors may have caused.

Figure 1a depicts a Tbr1 staining of the adult mouse cortex for four different genotypes. In the process of choosing representative pictures that reflect the results of our analysis shown in Fig. 1b, cropped images from original pictures were inadvertently mislabelled and used incorrectly. We provide below a corrected version of Fig. 1a with new representative images for the following genotypes: WT and Reln1/1;Efnb32/2. A new high-magnification picture for WT is also shown in the two rightmost panels. Original images for every genotype and additional examples are shown in the Supplementary Information of this Corrigendum.

This is Transparency?In this Letter we made errors in representative image choice, including mislabelling of images or choosing an image from the inappropriate genotype. In all cases, choice of images was completely independent of the data analysis and so none of the conclusions in our original Letter are affected. We apologise for any confusion these errors may have caused.

Figure 1a depicts a Tbr1 staining of the adult mouse cortex for four different genotypes. In the process of choosing representative pictures that reflect the results of our analysis shown in Fig. 1b, cropped images from original pictures were inadvertently mislabelled and used incorrectly. We provide below a corrected version of Fig. 1a with new representative images for the following genotypes: WT and Reln1/1;Efnb32/2. A new high-magnification picture for WT is also shown in the two rightmost panels. Original images for every genotype and additional examples are shown in the Supplementary Information of this Corrigendum.

Figure 1c depicts a Brn1 staining of the E17.5 mouse cortex for five different genotypes. In the process of figure assembly cropped images from original pictures were inadvertently mislabelled and used incorrectly. We provide below a corrected Fig. 1c with a new image for Reln1/1; Efnb3–/–. In the ephrinB3 compound mice (Reln1/2; Efnb32/2) Brn11 cells aberrantly accumulate in the lower layers of the cortex and do not migrate to the upper layers, resembling the Reeler (Reln2/2) phenotype. Original pictures and additional examples are shown in the Supplementary Information of this Corrigendum, where arrows indicate the distribution of Brn11 cells. We have also included results from a new, reproduced experiment recently performed with an additional cohort of animals that shows exactly the same results.

This is Transparency?In this Letter we made errors in representative image choice, including mislabelling of images or choosing an image from the inappropriate genotype. In all cases, choice of images was completely independent of the data analysis and so none of the conclusions in our original Letter are affected. We apologise for any confusion these errors may have caused.

Figure 1a depicts a Tbr1 staining of the adult mouse cortex for four different genotypes. In the process of choosing representative pictures that reflect the results of our analysis shown in Fig. 1b, cropped images from original pictures were inadvertently mislabelled and used incorrectly. We provide below a corrected version of Fig. 1a with new representative images for the following genotypes: WT and Reln1/1;Efnb32/2. A new high-magnification picture for WT is also shown in the two rightmost panels. Original images for every genotype and additional examples are shown in the Supplementary Information of this Corrigendum.

Figure 1c depicts a Brn1 staining of the E17.5 mouse cortex for five different genotypes. In the process of figure assembly cropped images from original pictures were inadvertently mislabelled and used incorrectly. We provide below a corrected Fig. 1c with a new image for Reln1/1; Efnb3–/–. In the ephrinB3 compound mice (Reln1/2; Efnb32/2) Brn11 cells aberrantly accumulate in the lower layers of the cortex and do not migrate to the upper layers, resembling the Reeler (Reln2/2) phenotype. Original pictures and additional examples are shown in the Supplementary Information of this Corrigendum, where arrows indicate the distribution of Brn11 cells. We have also included results from a new, reproduced experiment recently performed with an additional cohort of animals that shows exactly the same results.

In Fig. 1d, the second panel, labelled ‘Reln1/1;Efnb3–/–’ should instead be labelled ‘Reln1/2’. In the Methods summary section ‘Stimulation of neurons’, ‘‘Cortical neurons from E14.5 were grown….’’ should instead read ‘‘Cortical neurons from E15.5 were grown….’’.

(There were mistakes in the supplementary online material, too.)

This is Transparency?

This is Transparency?

This is Transparency?

This is Transparency?“The authors declare that key experiments presented in the majority of these figures were recently reproduced and that the results confirmed the experimental data and the conclusions drawn from them.”

EMBO Journal editor Bernd Pulverer:

“We did not formally investigate this case at the journal and we have not seen this data, as it does not affect the retraction.”

Model Organisms – and Retractions

Model Organisms – and Retractions

Model Retractions

Model Retractions

The Way Forward• Use systems to detect image manipulation and

plagiarism

• Require authors to disclose prior retractions and investigations

• Trust anonymous whistleblowers more

• Demand more of institutions

• Move more quickly to correct and retract

• Make retraction notices clearer - and -

The Way Forward• Make them freely available

Acknowledgements/Contact Info

• Nancy Lapid, Reuters Health• Adam Marcus, Retraction Watch• http://retractionwatch.com• ivan-oransky@erols.com• Twitter: @ivanoransky