Cullen: Why most published research is wrong
-
Upload
oliflower -
Category
Health & Medicine
-
view
653 -
download
1
description
Transcript of Cullen: Why most published research is wrong
![Page 1: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/1.jpg)
“Why most published
research is wrong.”
Louise Cullen(Clinician researcher)
![Page 2: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/2.jpg)
Disclosure Information
![Page 3: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/3.jpg)
“It is everyone’s responsibility to find out how to
ask questions systematically, find answers from searching the
literature, critically appraise the literature
and apply the results to practice.”
Rinaldo Bellomo
![Page 4: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/4.jpg)
“It is everyone’s responsibility to find out how to
ask questions systematically, find answers from searching the
literature, critically appraise the literature
and apply the results to practice.”
Rinaldo Bellomo
![Page 5: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/5.jpg)
![Page 6: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/6.jpg)
![Page 7: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/7.jpg)
40 ingredients associate with cancer
Most single studies showed implausibly large effects.
![Page 8: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/8.jpg)
The p value
![Page 9: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/9.jpg)
The p value
Observed size of Effect
![Page 10: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/10.jpg)
p=0.01
![Page 11: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/11.jpg)
p=0.01
There is a 1% chance of results as extreme as these would occur when there is really no difference occurring in the experiment.
![Page 12: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/12.jpg)
1000 hypotheses
![Page 13: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/13.jpg)
![Page 14: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/14.jpg)
![Page 15: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/15.jpg)
![Page 16: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/16.jpg)
Replication of studies
![Page 17: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/17.jpg)
Replication of studies
![Page 18: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/18.jpg)
Problems with the study itself.
![Page 19: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/19.jpg)
Wrong question
![Page 20: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/20.jpg)
Wrong Theory
![Page 21: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/21.jpg)
Wrong population studied
![Page 22: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/22.jpg)
2
ACS: Trial and community populations
Circulation. 115(19):2549-69, 2007 May 15.
![Page 23: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/23.jpg)
n=2
![Page 24: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/24.jpg)
Wrong design
![Page 25: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/25.jpg)
![Page 26: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/26.jpg)
• Greater the flexibility in– designs – definitions – outcomes– analytical modes
![Page 27: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/27.jpg)
• Greater the flexibility in– designs – definitions – outcomes– analytical modes
• Hotter a scientific field with more teams involved.
![Page 28: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/28.jpg)
Wrong Endpoints
![Page 29: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/29.jpg)
Ad and high dose Ad
Ca++ in cardiac arrest
COX-2 inhibitors
Milrinone
![Page 30: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/30.jpg)
![Page 31: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/31.jpg)
Methodology
Statistical hypothesis inference testing
![Page 32: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/32.jpg)
Problems with reporting
![Page 33: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/33.jpg)
Interpretation
![Page 34: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/34.jpg)
![Page 35: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/35.jpg)
![Page 36: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/36.jpg)
![Page 37: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/37.jpg)
![Page 38: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/38.jpg)
![Page 39: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/39.jpg)
• “a little significance”
• “a definite trend is evident”
• “a clear tendency”
• “almost achieved significance”
![Page 40: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/40.jpg)
• “a little significance”
• “a definite trend is evident”
• “a clear tendency”
• “almost achieved significance”
The data is practically
meaningless
![Page 41: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/41.jpg)
![Page 42: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/42.jpg)
• “In my experience”• “In case after case”• “In a series of cases”
• “It is generally believed that..”
• “A highly significant area for exploratory study”
• Once• Twice• Three times
• A couple of others think so too
• A totally useless topic in my underpowered study…….
![Page 43: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/43.jpg)
Omitting facts deliberately
….!
![Page 44: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/44.jpg)
Why?
Incentives
![Page 45: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/45.jpg)
Pharma
![Page 46: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/46.jpg)
Pharma
![Page 47: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/47.jpg)
![Page 48: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/48.jpg)
Why?
Ethical practice of researchers
![Page 49: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/49.jpg)
Problems with publishing
![Page 50: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/50.jpg)
Don’t believe in the review process
![Page 51: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/51.jpg)
Journal publishing practices
![Page 52: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/52.jpg)
• 2004 “original articles” in NEJM– 363 tested an established therapy– 146 (40%) reversed that practice– 138 (38%) reaffirmed it
![Page 53: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/53.jpg)
What can you do about it?
![Page 54: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/54.jpg)
Read more than the title!
![Page 55: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/55.jpg)
Reporting Framework
CONSORT (http://bit.ly/14qUNEF)– Standards for reporting of trials
STARD– Standards for the Reporting of diagnostic accuracy
studies
![Page 56: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/56.jpg)
Biases
![Page 57: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/57.jpg)
![Page 58: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/58.jpg)
![Page 59: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/59.jpg)
Be sceptical!
![Page 60: Cullen: Why most published research is wrong](https://reader036.fdocuments.in/reader036/viewer/2022062418/554ae615b4c905852a8b4c10/html5/thumbnails/60.jpg)
Thank you
@louiseacullen