TIA Consulting March 2002 Estimating the Contribution of a Government Program Rosalie T. Ruegg TIA...
-
Upload
verity-barton -
Category
Documents
-
view
215 -
download
1
Transcript of TIA Consulting March 2002 Estimating the Contribution of a Government Program Rosalie T. Ruegg TIA...
TIA ConsultingMarch 2002
Estimating the Contribution of a Government Program
Rosalie T. RueggTIA Consulting
Conference on Estimating the Benefits of Government-Sponsored Energy R&D
Crystal City, VAMarch 5, 2002
TIA ConsultingMarch 2002
Attributing Benefits to a Government Program
inputs outputs outcomes impacts
• What would have happened without the gov’t effort?[counterfactual studies]
• What happened in similar cases where there was no gov’t intervention?
[control group and comparison studies]
It’s not enough to measure these effects; evaluation seeks to determine how much is attributable (caused by) the gov’t intervention.
TIA ConsultingMarch 2002
Overall Net Benefits to Innovation X
Government +Industry R&D Investment
NPV Benefits computed against a baseline
TIA ConsultingMarch 2002
To estimate the return on government investment, we need to isolate its effect
Benefits w/o gov’tmeasured against the
baseline
Private Actions w/o Gov’t Intervention
TIA ConsultingMarch 2002
A Gov’t Program Might Affect Benefits By:
• Accelerating development & deployment- start earlier- progress faster- pursue what otherwise would not be done
• Increasing scale raising likelihood of success
• Increasing scope widening applications
• Building new innovation networks/pathways
• Other effects
(It might also substitute for private investment)
TIA ConsultingMarch 2002
Impact of Acceleration on Social Returns
Source: RTI, NIST GCR 97-737, 1998, fig. 2-3
TIA ConsultingMarch 2002
Illustrative Approaches to Estimating Gov’t Impact
(examples from ATP)
Purposes:1. To support specific project benefit-cost case studies2. To provide program-wide evidence of attribution
Study Examples:• Survey Method e.g., Silber, Powell, Laidlaw –counterfactual analysis Feldman/Kelley--control group study
• Case Study Method e.g., Gompers/Lerner, RTI, Pelsoci, 50 completed Projects, etc.
• Econometric/Sociometric Methodse.g., Branstetter/Sakakibara, Zucker/Darby, Fogarty/Sinha/Jaffe, etc.
TIA ConsultingMarch 2002
Acceleration: Laidlaw Survey of 28 ATP Projects
Applied research cycle time Reductions:
Median reported values- 50% reduction- 3 year cut
Range of reported values- % reduction: 25% to infinity- years cut: 1-2 to unbounded
TIA ConsultingMarch 2002
Other Relevant Effects: Laidlaw Survey
• 86% expected flow-thru of time cuts to later stages
• 79% gave “ballpark estimates” of the value of cutting time by 1 year
• Respondents listed ways the program cut time- upfront planning and mangt requirements- achieving a critical funding mass- enhancing ability to attract additional funding- increasing project stability- encouraging collaboration
• 86% reported carry-over of acceleration to other projects
TIA ConsultingMarch 2002
Feldman/Kelley Non-Winner “Comparison Group” Analysis as Evidence that Gov’t Program (ATP) Makes a Difference
Extent of Follow-up Activity Percentage
Did not proceed with the project at all
61.4
Began project on a much smaller scale
16.4
Began project on a somewhat smaller scale
11.7
Began project at about the same scale as proposed to ATP
5.3
Began project at somewhat larger scale
2.9
Began project on a much larger scale
0.6
Refused to answer/don’t know
1.8
Source: Feldman & Kelley, NRC Report on ATP, 2001, p. 134.
TIA ConsultingMarch 2002
Discussion Topic: Use of a 5-Year Acceleration Rule for DOE Program Evaluation
• NRC basis for?
• Is it “conservative”?
• Is it generalizable?
• Is it feasible to determine specific time lags on a program-by-program basis or should it be on a project-by-project basis?
(ATP’s experience suggests no fixed Rule)