R&D Progression Bias and Rational Optimism
-
Upload
dennis-lendrem -
Category
Health & Medicine
-
view
356 -
download
4
Transcript of R&D Progression Bias and Rational Optimism
Lendrem D.W., Senn S.J., Lendrem B.C., Isaacs J.D. (2015),
R&D productivity rides again?, Pharmaceut. Statist., 14, 1–3, doi: 10.1002/pst.1653
Lendrem D., Senn S.J., Lendrem B.C., and Isaacs J.D. (2015), R&D productivity rides
again?, Pharmaceut. Statist., 14, 1–3, doi: 10.1002/pst.1653
Rational Optimism
• By placing tasks in parallel to reduce cycle time we
inadvertently:
• Increased R&D costs
• Increased late-stage attrition rates
• Increased expected time to market
Rational Optimism
• By placing tasks in parallel to reduce cycle time we
inadvertently:
• Increased R&D costs
• Increased late-stage attrition rates
• Increased expected time to market
Lendrem DW, Lendrem BC Torching the Haystack: Modelling fast-fail strategies in drug
development. Drug Discov Today. 2013 Apr;18(7-8):331-6.
http://dx.doi.org/10.1016/j.drudis.2012.11.011
Lendrem DW, Lendrem BC The Development Speed Paradox: Can increasing development
speed reduce R&D productivity? Drug Discov Today. 2014 Mar;9(3):209-214
http://dx.doi.org/10.1016/j.drudis.2013.09.002
Rational Optimism
• It is more important to do the right science than to do
the wrong science quickly.
• Build opportunities to kill projects earlier in the
development cycle.
• Focus on tests that prevent projects from entering the
development process.
• Focus on tests that allow elimination of projects early
in the development process.
Rational Optimism
• Progression Bias
• terminations are viewed as ‘losses’ and project teams are loss-
averse
• sunk-costs fallacy – terminating a project risks losing investment
that has already been lost
• marginal cost associated with a decision to continue development
may seem trivial compared to the losses associated with
terminating a potential NTD
• organizational and market incentives to load the pipeline with
‘promising’ candidates
• Progression bias leads to continued development even in
the light of strong evidence supporting termination.
Rational Optimism
• Rational Optimism
• Continued pursuit of a line of research in the knowledge that it is
likely to be successful.
Rational Optimism
• Rational Optimism
• Continued pursuit of a line of research in the knowledge that it is
likely to be successful.
• Mindless Optimism
• Continued pursuit of a line of research in the hope that it will be
successful.
Rational Optimism
• Opportunity Costs:
Loss due to false negatives = B
NTD
p1-β A
β B
1-pα C
1-α D
Success
Miss
False Discovery
Success
Rational Optimism
• Opportunity Costs:
Loss due to false negatives = B
Loss due to false positives = C
NTD
p1-β A
β B
1-pα C
1-α D
Success
Miss
False Discovery
Success
Rational Optimism
• Opportunity Costs:
Loss due to false negatives = B
Loss due to false positives = CxA
NTD
p1-β A
β B
1-pα C
1-α D
Success
Miss
False Discovery
Success
Opportunity
Costs
(%)
Probability (%)
20
15
10
5
0
25 -
-
-
-
-
-0 20 40 60 80 100I I I I I I
Cost of False Positives
when α = 67%
β =10%
mindless
optimism
rational
optimismvs
Opportunity
Costs
(%)
Probability (%)
20
15
10
5
0
25 -
-
-
-
-
-0 20 40 60 80 100I I I I I I
β =15%Cost of False Positives
when α = 67%
mindless
optimism
rational
optimismvs
Opportunity
Costs
(%)
Probability (%)
20
15
10
5
0
25 -
-
-
-
-
-0 20 40 60 80 100I I I I I I
mindless
optimism
rational
optimism β =20%
Cost of False Positives
when α = 67%
vs
Rational Optimism
• So?
Opportunity
Costs
(%)
Probability (%)
20
15
10
5
0
25 -
-
-
-
-
-0 20 40 60 80 100I I I I I I
Cost of False Positives
when α = 67%
Cost of False Negatives
when β =10%
mindless
optimism
rational
optimismvs
Opportunity
Costs
(%)
Probability (%)
20
15
10
5
0
25 -
-
-
-
-
-0 20 40 60 80 100I I I I I I
mindless
optimism
rational
optimism Cost of False Negatives
when β =20%
Cost of False Positives
when α = 67%
vs
The false negative rate has to be rather high before it outweighs
the cost of false positives!
Opportunity
Costs
(%)
Probability (%)
20
15
10
5
0
25 -
-
-
-
-
-0 20 40 60 80 100I I I I I I
β =20%
Cost of False Positives
when α = 67%
mindless
optimism
rational
optimismvs
Opportunity
Costs
(%)
Probability (%)
20
15
10
5
0
25 -
-
-
-
-
-0 20 40 60 80 100I I I I I I
β =20%
mindless
optimism
rational
optimismvs
Opportunity
Costs
(%)
Probability (%)
20
15
10
5
0
25 -
-
-
-
-
-0 20 40 60 80 100I I I I I I
β =20%
mindless
optimism
rational
optimismvs
Rational Optimism
• So?
Opportunity
Costs
(%)
Probability (%)
20
15
10
5
0
25 -
-
-
-
-
-0 20 40 60 80 100
I I I I I I
Cost of False Negatives
when β =20%
Cost of False Positives
when α = 67%
mindless
optimism
rational
optimismvs
Opportunity
Costs
(%)
Probability (%)
20
15
10
5
0
25 -
-
-
-
-
-0 20 40 60 80 100
I I I I I I
Cost of False Negatives
when β =20%
mindless
optimism
rational
optimismvs
Early Development Late Development
Rational Optimism
• So?
• We fear false negatives, but the opportunity costs of false positives
are huge.
• Terminating projects frees up R&D resources to pursue other
prospects.
• Thinking about opportunity costs allows project teams to reframe
termination decisions as prospective future gains.
Rational Optimism
Lendrem D.W., Lendrem B.C., Peck R.W., Senn S.J., Day, S., Isaacs J.D. (2015),
Progression Bias and Rational Optimism in Research & Development,
Nat. Rev. Drug Discov., In press
NTD
p1-β1
1-β2
1-β3
1-β4
1-β5
1-β6
1-β7
1-β8
β8β7
β6β5
β4β3
β2β1
1-pα1
α2
α3
α4
α5
α6
α7
α8
1-α81-α7
1-α61-α5
1-α41-α3
1-α21-α1
Rational Optimism
• Richard Peck (Roche)
• Stephen Senn (CRP Santé)
• Simon Day (MHRA)
• John Isaacs (MRG)
• John Loughlin (MRG)
NIHR Newcastle BRC
Rational Optimism
• Richard Peck (Roche)
• Stephen Senn (CRP Santé)
• Simon Day (MHRA)
• John Isaacs (MRG)
• John Loughlin (MRG)
• Clare Lendrem (AHG)
NIHR Newcastle BRC