How anonymous post-publication peer review
uncovers bad science
Leonid Schneider,science journalist with Laborjournal
[email protected]: @schneiderleonid
Junior scientists are often told by their advisors:
- If you can deliver this result, you will publish a nice paper and have a job
- If you don’t deliver this result, you will not publish any paper and have no job
Is bad scienceindividual or systemic failure?
$$$
Authors and institutions have little incentive to produce reliable quality science
Paper-to-funding convertion
Funding used for research…
Journals and funding agencies prefer simplistic, but sensationalist “break-through” science
• Cancer cure!
• Stem cells /”Reprogramming”
• One-Gene-Phenotype models
• Translational/Commercial potential
Biological systems are very complicated,
but in biological papers simplicity rules!
Scientists occasionally help data to fit their theoretical model for a publication
• Selective data acquisition and evaluation (very common)
• “Adjustments” or manipulation of data (less common)
• Data falsification / fraud (very rare)
Peer review weeds out bad science. Really?
• Data is submitted on trust as being honest/reliable
• Peer Reviewers are scientist colleagues, not data specialists
• Peer Reviewers only analyse science, not data integrity
• Peer review is not always done diligently enough
How did this pass peer review????
$$$
Traditional peer
review
Traditional peer review is anything but transparent
Years and years of research…
- Journal Editors- Decide on Quality,
Novelty, Impact- Appoint peer
reviewers- Make final decisions
- Peer Reviewers- 1-4 people- Unknown to authors
or readers- Potential COI,
personal animosities, lack of competence…
$$$
Too many financial and personal interests involved
Years and years of research…
Convincing peer reviewers is by far the most important task of a scientist
A peer-reviewed paper is a badge of honour
Things surely changed for him since he published in Nature…
• Publications are public evidence of success
• They are to be admired and not questioned
• Often not the content counts, but where it is published
Scientists waste time, money and their careers trying to reproduce unreliable or manipulated results
• Poor reproducibility in combination with high competition undermines productivity, but also work moral, trust and motivation
• It leads to even more data manipulation and fraud in science
What do you do if you spot data irregularities or irreproducibility in a published paper?
1. Write to authors
2. Write to journal
3. Write to authors’ institution
What happens if a published paper is reported to be wrong or even contain manipulated data?
1. Correction (rare)
2. Retraction (even rarer)
3. Nothing (most common)
Your paper is wrong, professor!
See you at the exam…
Individual criticisms are unwelcome and dangerous
• Financial interests behind publications prevent institutional investigations
• Institutions often refuse to react to anonymous hints
• Whistle-blowers are often punished or dismissed as incompetent or malicious
Solution: make valid criticisms public, but anonymously!
• Publicly available valid criticisms are much more difficult to be ignored
• Whistle-blowers are protected by the anonymity under which they are free to post concerns
More on this: http://www.laborjournal.de/editorials/424_11.lasso
Adam Marcus
Ivan Oransky
Retraction Watch takes whistle-blowers seriously
PubPeer allows anonymous post-publication peer review, including evidence
On PubPeer you could ask critical questions anonymously
Or, you can post evidence of data irregularities on PubPeer, also anonymously
I have some issues with your paper, Sir!
Pros and cons of anonymous commenting (aka witch-hunts)
• Protects whistle-blowers
• Only objective evidence and arguments matter, not who has raised them or why or where
• Unsubstantiated claims, personal insults
• Sock-puppeting (also by authors!)
Against:
For:
PubPeer protects the anonymity, even when the criticized scientists go to court
Blog post at Laborjournal about this interesting and useful experience: http://www.laborjournal.de/blog/?p=8281
Autors also reply to PubPeer criticisms, often
constructively
STAP: one of the biggest fraud scandals uncovered, thanks to post-publication peer review on PubPeer
Photo credit: Maigrot/REA
The case Olivier Voinnet
• PhD with Sir David Baulcombe at The Sainsbury Laboratory, Norwich
• Research leader at CNRS institute in Strasbourg (age 33)
• Professor at ETH Zürich (since 2010)
Photo credit: Maigrot/REA
The case Olivier Voinnet
• EMBO Gold Medal (2009)
• EMBO Member
• EMBO Young Investigator grant
• ERC start-up grant
• Max-Rössler-Prize (ETH Zürich, 2013)
It started with people finding irregularities in David Baulcombe’s papers
Olivier Voinnet publications flagged on PubPeer
Olivier Voinnet and other involved reply on PubPeer and promise to investigate
Vicki VanceProfessor of Botany, University of South Carolina
Vicki Vance: the key Whistle-Blower in Voinnet case
To be continued?
Top Related