Federal Demonstration Partnership January 13, 2015 - Home | National...
Transcript of Federal Demonstration Partnership January 13, 2015 - Home | National...
![Page 1: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/1.jpg)
Federal Demonstration Partnership
January 13, 2015
STAR METRICS is a registered trademark of HHS. The logo STAR METRICS & Design is a service mark of HHS.
![Page 2: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/2.jpg)
Purpose
• Trans-federal agency program with participation from NIH, NSF, USDA, OSTP, and EPA
• Goal is to provide better empirical basis for science policy by providing an open and automated infrastructure to document and analyze the impact of federal investments in science
2
![Page 3: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/3.jpg)
Two Levels of Activity
• Level I: Developing uniform, auditable and standardized measures of the impact of science spending (ARRA and non-ARRA) on job creation, using data from research institutions' existing database records. No personally identifiable information (PII) is collected in Level I.
3
![Page 4: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/4.jpg)
Two Levels of Activity
• Level II (Federal RePORTER): Develop an open automated data infrastructure and tools that will enable the documentation and analysis of a subset of the inputs, outputs, and outcomes resulting from federal investments in science
4
![Page 5: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/5.jpg)
FederalRePORTER.NIH.gov
• Hard launch on September 23rd, 2014 – Coupled with an overhaul of the STAR METRICS®
website
• Public-facing access from a single web portal
• Funded data from 14 agencies or agency sub-components for FY 2004 – FY 2013 • Almost 600,000 awards records
5
![Page 6: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/6.jpg)
FederalRePORTER.NIH.gov
• Roughly 85,000 pageviews from over 8,000 unique visitors
• The initial focus is on a small set of core data elements, which includes funding amounts, titles, and project abstracts
6
![Page 7: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/7.jpg)
FederalRePORTER.NIH.gov
7
![Page 8: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/8.jpg)
8
Search Titles, Abstracts, Terms
PTSD
![Page 9: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/9.jpg)
9
Search for ‘PTSD’
![Page 10: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/10.jpg)
10
Project Description
![Page 11: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/11.jpg)
Results
11
![Page 12: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/12.jpg)
12
Similar Projects
![Page 13: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/13.jpg)
13
Charts with Drill-Down
![Page 14: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/14.jpg)
14
Text Mining
![Page 15: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/15.jpg)
15
Text Mining
![Page 16: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/16.jpg)
16
Text Mining
![Page 17: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/17.jpg)
FederalRePORTER.NIH.gov
• Data infrastructure that can be used to describe the broader results of science investments beyond jobs data
• Allows for portfolio analysis within and across agencies to assess the impact of science, technology, and innovation investments
17
![Page 18: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/18.jpg)
Participants To-Date • Department of Health and Human Services
– National Institutes of Health
– Administration for Children and Families – Agency for Health Care Research and Quality – Centers for Disease Control and Prevention – Food and Drug Administration
• Department of Agriculture – Agricultural Research Service – Forest Service – National Institute of Food and Agriculture
• Department of Defense – Center for Neuroscience and Regenerative Medicine – Congressionally Directed Medical Research Programs
• Department of Education – National Institute on Disability and Rehabilitation Research*
• Department of Veterans Affairs • Environmental Protection Agency • National Aeronautics and Space Administration • National Science Foundation
18
![Page 19: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/19.jpg)
19
Potential Future
Capabilities
![Page 20: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/20.jpg)
Topic Modeling RePORTER “Food Safety” Matches
20
![Page 21: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/21.jpg)
Topic Modeling Drill Down
21
![Page 22: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/22.jpg)
PI Profile Links
22
![Page 23: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/23.jpg)
Federal RePORTER Future
• Investigator disambiguation • Add links to SciENcv profiles
• Expand agency participation • Standardization of performing organization
information • Automating the data feeds • Incorporate research results
• Publications • Patents • RPPR products • SciENcv data
23
![Page 24: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/24.jpg)
Level I Activities
and Future Directions
![Page 25: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/25.jpg)
Level I Activities
• Over 100 campuses have signed MOUs with the STAR METRICS® program
• Institutional reports from FY 2014 represent >25% of NIH extramural funding
• Quarterly reports generation includes jobs calculations, vendor information, geographic and historical data
25
![Page 26: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/26.jpg)
Level I Activities
• Develop a common infrastructure across institutions – Through the partnership with FDP, STAR
METRICS® has been able to collaborate on common data elements and mapping techniques
– Standardized a modular report format and data file templates
26
![Page 27: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/27.jpg)
Level I Activities
• Focus on adjusting administrative datasets to connect investments with outcomes – Automated generation of workforce measures
using existing database records • Direct economic impacts of R&D spending
• Develop the academic field – UMetrics and other initiatives, researchers
27
![Page 28: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/28.jpg)
Current Status
• Successful proof of concept for using administrative data
• Created the foundation for a comprehensive dataset that can connect investments with outcomes
28
![Page 29: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/29.jpg)
National Academy of Sciences Report
29
• Recommendations: • Accessibility of data • Linkages to other
datasets • Limited number of data
elements • Institutional
participation
![Page 30: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/30.jpg)
Implementing Level I on a Broader Scale
30
• Level I is ready to be taken to the next level • Community-driven efforts such as UMETRICS provide an
attractive pathway for expansion: – The expertise resides in the science-of-science and economics
research communities – More flexibility in data collection and wider access can be provided
in a community-driven effort – Results are more likely to be tailored to the specific needs of
institutions and researchers
• The STAR METRICS® consortium wants to support such efforts and is currently exploring ways to do that
![Page 31: Federal Demonstration Partnership January 13, 2015 - Home | National …sites.nationalacademies.org/cs/groups/pgasite/documents/... · 2020. 4. 14. · • Hard launch on September](https://reader035.fdocuments.in/reader035/viewer/2022071212/60267c71ce6119202c69f7b2/html5/thumbnails/31.jpg)
Questions
31