Moving Toward Moving Toward Quantitative Evidence-Quantitative Evidence-based Sciencebased SciencePolicy: Science of Science Policy: Science of Science Policy Developmental Policy Developmental Efforts In Theory, Efforts In Theory, Evaluation Methods, and Evaluation Methods, and Data InfrastructureData Infrastructure

Kei KoizumiKei KoizumiKei_Koizumi@ostp.eop.govKei_Koizumi@ostp.eop.gov

November 12, 2009November 12, 2009

for the American Evaluation Associationfor the American Evaluation Association

An interagency collaboration (16 agencies or offices) with three guiding themes and ten key questions

33

The SOSP RoadmapThe SOSP Roadmap

Science Questions

1. What Are The Behavioral Foundations Of Innovation?

2. What Explains Technology Development, Adoption And Diffusion?

3. How And Why Do Communities Of Science And Innovation Form And Evolve?

4. What Is The Value Of The Nation’s Public Investment In Science?

5. Is It Possible To “Predict Discovery”?

6. Is It Possible To Describe The Impact Of Discovery On Innovation? 7. What Are The Determinants Of Investment Effectiveness?

8. What Impact Does Science Have On Innovation And Competitiveness?

9. How Competitive Is The U.S. Scientific Workforce?

10. What Is The Relative Importance Of Different Policy Instruments In Science Policy?

The National Imperative

Theme 3: Using the Science of Science

Policy to Address National Priorities

Theme 2: Investing in Science

and Innovation

Theme 1: Understanding Science

and Innovation

Recommendations

Findings

Themes

Science of Science Policy in the Science of Science Policy in the Obama AdministrationObama Administration

There is a new charter for the Interagency Task Group There is a new charter for the Interagency Task Group to carry through the SoSP roadmap. to carry through the SoSP roadmap.

Science and technology are high priorities in the ARRA, Science and technology are high priorities in the ARRA, the 2010 Budget, and in planning for the 2011 budget. the 2010 Budget, and in planning for the 2011 budget.

Openness and transparency are high priorities: Openness and transparency are high priorities: data.gov, open.gov and recovery.gov are examples. data.gov, open.gov and recovery.gov are examples.

Evidence-based policy and evaluation are also Evidence-based policy and evaluation are also priorities: OMB Director Orszag has issued numerous priorities: OMB Director Orszag has issued numerous statements, including an October 7 Evaluation memo. statements, including an October 7 Evaluation memo.

Agencies are collaborating to build a new data Agencies are collaborating to build a new data infrastructure to be able to evaluate R&D investments infrastructure to be able to evaluate R&D investments (STAR pilot). (STAR pilot).

See scienceofsciencepolicy.net

A Science of Science Policy A Science of Science Policy Community website, with a new Community website, with a new version coming soonversion coming soon

OMB / OSTP “R&D Priorities Memo” for FY 2011 Budget (see www.ostp.gov)

OMB / OSTP “R&D Priorities Memo” OMB / OSTP “R&D Priorities Memo” for FY 2011 Budgetfor FY 2011 Budget

““Agencies should describe the expected outcomes from their Agencies should describe the expected outcomes from their research in relation to… practical challenges and cross-research in relation to… practical challenges and cross-cutting areas, providing quantitative metrics where possible.”cutting areas, providing quantitative metrics where possible.”

““Federal agencies… should develop datasets to better Federal agencies… should develop datasets to better document Federal science and technology investments and document Federal science and technology investments and to make these data open to the public in accessible, useful to make these data open to the public in accessible, useful formats.”formats.”

““Agencies should develop outcome-oriented goals for their Agencies should develop outcome-oriented goals for their science and technology activities, establish procedures and science and technology activities, establish procedures and timelines for evaluating the performance of these timelines for evaluating the performance of these activities…”activities…”

““Agencies should develop ‘science of science policy’ tools Agencies should develop ‘science of science policy’ tools that can improve management of their [R&D] portfolios and that can improve management of their [R&D] portfolios and better assess the impact of their [S&T] investments.” better assess the impact of their [S&T] investments.”

Next StepsNext Steps

Feedback from the December 2008 Workshop shaped a Feedback from the December 2008 Workshop shaped a set of interagency research priorities addressing the set of interagency research priorities addressing the scientific challenges confronting the unique science scientific challenges confronting the unique science policy analysis needs of Federal agencies: Developing a policy analysis needs of Federal agencies: Developing a Data Infrastructure for Science and Innovation Policy; Data Infrastructure for Science and Innovation Policy; Modeling; Creating an Innovation Framework; Informing Modeling; Creating an Innovation Framework; Informing and Assessing R&D Investments (workshop in October and Assessing R&D Investments (workshop in October 2009); and Conducting Outreach to Underrepresented 2009); and Conducting Outreach to Underrepresented Populations. Populations.

The next workshop in 2010 will be for NSF Science of The next workshop in 2010 will be for NSF Science of Science & Innovation Policy (SciSIP) Grantees: Science & Innovation Policy (SciSIP) Grantees: Presentations and Findings from the current Pis; Presentations and Findings from the current Pis; Keynotes and discussions from practitioner communityKeynotes and discussions from practitioner community