Evidence-Based Funding: Supporting Biomedical Research with Austere Budgets

Michael S Lauer, MD Deputy Director for Extramural Research

National Institutes of Health

Massachusetts General Hospital Research Institute Thursday, November 3, 2016

Massachusetts General Hospital, Simches 3110, Boston, MA Disclosures: None

1

A Problem?

2 Shapiro D, Vrana K. PNAS 2015;112:9496-7

What Should Our Metrics Be?

3 Shapiro D, Vrana K. PNAS 2015;112:9496-7

Better Finish Line

4 Moses H et al. JAMA 2015;313:174-189

Funding

5

Moses H et al. JAMA 2015;313:174-189

Funding …

6 Moses H et al. JAMA 2015;313:174-189

Annual Growth 2004 – 2012: Pharma -0.6% Device +6.2% Biotech +4.6% NIH -1.8%

Workforce

7

Moses H et al. JAMA 2015;313:174-189

Root Causes…

8

“We identified two core problems: • Too many researchers vying for too few dollars. • Too many postdocs competing for too few positions. Most other issues can be viewed as symptoms.”

A Different Metric

9

Changing our funding metric “A question that at first glance may seem trivial but is, I believe, a significant one is whether our key metric for how… we invest in … research should be the number of grants we award or the number of investigators we support.”

Lorsch JR. Mol Biol Cell 2015;26:1578-82

10

Thanks to OER DPEA and SARB

11

Thanks to OER DPEA and SARB

Distribution of Dollars to Awardees …

12

Alberts B et al. PNAS. 2014;111:5773-7

“Agencies should be sensitive to the total numbers of dollars granted to individual laboratories…—although different research activities have different costs—at some point, returns per dollar diminish. We applaud the recent decision by the NIH to examine grant portfolios carefully before increasing direct research support for a laboratory beyond $1M per year.”

13

Thanks to OER DPEA and SARB

14

Thanks to OER DPEA and SARB

15

An Ongoing Conversation

16

Xie Y. Science 2014;344:809-810

Publication and Citations?

17 Shapiro D, Vrana K. PNAS 2015;112:9496-7

So How Do We Measure Research Impact?

• P = productivity – Publish trial results – Highly-cited papers: field-normalized

• Q = quality • R = replication • S = sharing • T = translation

18 Ioannidis JP, Khoury MJ. JAMA 2014 (June 9)

Relative Citation Ratio

19

NIH Office of Portfolio Analysis PLoS Biology (September 6, 2016)

What It Means

20

NIH Office of Portfolio Analysis PLoS Biology (September 6, 2016)

How Do We Know Whether It Means Anything?

21

NIH Office of Portfolio Analysis PLoS Biology (September 6, 2016)

2193 R01 papers 430 HHMI/NIH papers 290 R01 papers

Sample

• All PubMed papers – Published between 1995 and 2014 – With known RCR values (only 0.09% missing)

• Categorized by funding source (SPIRES) – NIH-MGH funded papers: N=24,659 (<1%) – Other NIH funded papers: N=1,415,696 (11%) – Non-NIH papers: N=11,922,575 (89%)

Thanks to OER DPEA and SARB Thanks to OER Data Systems