Reconsidering the Scientists and Engineers Statistical ...SESTAT • The NSF’s Scientists and...
Transcript of Reconsidering the Scientists and Engineers Statistical ...SESTAT • The NSF’s Scientists and...
Reconsidering the Scientists and Engineers
Statistical Data System
(SESTAT)
John Finamore
CGS Graduate Education Research & Policy Forum
January 18, 2012
National Science Foundation
National Center for Science and Engineering Statistics
www.nsf.gov/statistics/
Overview
• What is SESTAT?
• The Development of SESTAT
• Reconsidering SESTAT
• Data Needs and Analytical Interests
1
What is SESTAT?
SESTAT
• The NSF’s Scientists and Engineers Statistical Data System
(SESTAT) is a unique source of cross-sectional and
longitudinal information on the college-educated U.S.
science and engineering workforce
• The SESTAT data are collected through three biennial
surveys:
- National Survey of College Graduates (NSCG)
- National Survey of Recent College Graduates (NSRCG)
- Survey of Doctorate Recipients (SDR)
3
SESTAT Survey Questionnaires
• The core questionnaire items are similar across the three
SESTAT surveys
• All surveys have similar questionnaire sections:
- Education background
- Employment situation (Current and Past)
- Other work-related experiences
- Demographic information
• SESTAT provides information on topics such as
educational attainment, employment status, work
activities, salary, and continuing educational experiences 4
SESTAT Questionnaire Items
• Question modules rotate in and out of the surveys
throughout the decade
• For the 2010 survey cycle, new questionnaire items
include:
- Importance of various job attributes
- Available benefits in principal job
• For the 2010 survey cycle, past items rotated in
included:
- Satisfaction with various job attributes
- Membership in professional meetings
- Immigration module 5
The Development of SESTAT
NSF’s Scientists and Engineers Data
• For over 60 years, the National Science Foundation (NSF)
has been responsible for providing information about the
nation’s scientists and engineers
• In response to a 1989 request to review the NSF’s system
on scientists and engineers data, the Committee on National
Statistics (CNSTAT) made the following recommendations
7
1989 CNSTAT Recommendations
Recommendation 5.6.
NSF should conduct a large postcensal survey based on the
decennial census long form that provides baseline
information on college graduates
Recommendation 5.7.
NSF should conduct a panel survey that includes a sample
for each new graduating class of students at higher
educational institutions
Recommendation 5.8.
NSF should continue the Survey of Doctorate Recipients as
a major source of information on science and engineering
personnel at the Ph.D. level 8
Implementation of Recommendations
NSF implemented the 1989 CNSTAT recommendations
through the coordinated sponsorship of three surveys:
• National Survey of College Graduates (NSCG)
• National Survey of Recent College Graduates (NSRCG)
• Survey of Doctorate Recipients (SDR)
The information from these three surveys were combined to
form a data system called the Scientists and Engineers
Statistical Data System (SESTAT)
9
SESTAT Target Population
Each SESTAT survey was designed to target a unique
population
• The NSCG: all college graduates residing in the
U.S. as of the decennial census reference date
• The NSRCG: recent college graduates with a
bachelor’s or master’s S&E degree earned from a
U.S educational institution
• The SDR: all U.S.-earned S&E doctoral degree
recipients including those earned during the
previous two academic years 10
Type of Degree Holder Date Degree Earned SESTAT Survey
Foreign-Earned S&E
Degree Holders
Prior to April 2000 NSCG
After April 2000
U.S. Earned S&E
Bachelor's and Master's
Degree Holders
Prior to April 2000 NSCG
2001-2002
2003-2005
2006-2007
2008-2009
U.S. Earned S&E Doctoral
Degree Holders
Prior to April 2000 NSCG
2001-2002
2003-2005
2006-2007
2008-2009
11
Note: SESTAT also provides coverage of the population of U.S. residents with a non-S&E degree
working in an S&E occupation. In the 2010 decade, SESTAT will continue to cover this population,
and will also provide coverage of the non-S&E degree holders working in a non-S&E occupation.
SESTAT Coverage During the 2000 Decade
Type of Degree Holder Date Degree Earned SESTAT Survey
Foreign-Earned S&E
Degree Holders
Prior to April 2000 NSCG
After April 2000
U.S. Earned S&E
Bachelor's and Master's
Degree Holders
Prior to April 2000 NSCG
2001-2002 RCG (2003)
2003-2005
2006-2007
2008-2009
U.S. Earned S&E Doctoral
Degree Holders
Prior to April 2000 NSCG
2001-2002
2003-2005
2006-2007
2008-2009
12
SESTAT Coverage During the 2000 Decade
Type of Degree Holder Date Degree Earned SESTAT Survey
Foreign-Earned S&E
Degree Holders
Prior to April 2000 NSCG
After April 2000
U.S. Earned S&E
Bachelor's and Master's
Degree Holders
Prior to April 2000 NSCG
2001-2002 RCG (2003)
2003-2005 RCG (2006)
2006-2007 RCG (2008)
2008-2009 RCG (2010)
U.S. Earned S&E Doctoral
Degree Holders
Prior to April 2000 NSCG
2001-2002
2003-2005
2006-2007
2008-2009
13
SESTAT Coverage During the 2000 Decade
Type of Degree Holder Date Degree Earned SESTAT Survey
Foreign-Earned S&E
Degree Holders
Prior to April 2000 NSCG
After April 2000
U.S. Earned S&E
Bachelor's and Master's
Degree Holders
Prior to April 2000 NSCG
2001-2002 RCG (2003)
2003-2005 RCG (2006)
2006-2007 RCG (2008)
2008-2009 RCG (2010)
U.S. Earned S&E Doctoral
Degree Holders
Prior to April 2000 NSCG & SDR
2001-2002 SDR (2003)
2003-2005
2006-2007
2008-2009
14
SESTAT Coverage During the 2000 Decade
SESTAT Coverage During the 2000 Decade
Type of Degree Holder Date Degree Earned SESTAT Survey
Foreign-Earned S&E
Degree Holders
Prior to April 2000 NSCG
After April 2000
U.S. Earned S&E
Bachelor's and Master's
Degree Holders
Prior to April 2000 NSCG
2001-2002 RCG (2003)
2003-2005 RCG (2006)
2006-2007 RCG (2008)
2008-2009 RCG (2010)
U.S. Earned S&E Doctoral
Degree Holders
Prior to April 2000 NSCG & SDR
2001-2002 SDR (2003)
2003-2005 SDR (2006)
2006-2007 SDR (2008)
2008-2009 SDR (2010)
15
The Importance of the NSRCG
• In the 1990 and 2000 decades, the NSCG did not
capture the inflow of two groups into the S&E workforce:
- Recent college graduates over the decade
- Immigrants into the U.S. over the decade
• The goal of NSRCG is to improve the coverage of
NSCG. Without the NSRCG, SESTAT would not provide
data for recent college graduates with a bachelor’s or
master’s S&E degree
16
NSRCG
• Cross-sectional survey of individuals who recently received
bachelor’s or master’s degrees in S&E fields from U.S.
academic institutions – the inflow of new graduates through
the decade
• Collected using a two-stage sample design
Stage 1 – U.S. educational institutions
Stage 2 – Graduates within sampled institutions
• Most expensive of the SESTAT surveys
- two-stage sample design (institutions and graduates)
- hard to reach, mobile population 17
Reconsidering SESTAT
Examining the Potential of the ACS
• The long form was the basis of the “once per decade”
sample for the NSCG in the 1990 and 2000 decades
• After the 2000 decennial census, the Census Bureau
discontinued the use of the long form and introduced the
American Community Survey (ACS)
• The ACS is an on-going survey that provides national,
state, and metropolitan level estimates of the
characteristics of the U.S. population
19
2008 CNSTAT Recommendation
• In light of this change from the long form to the ACS as
the NSCG sampling frame, CNSTAT was charged with
assessing the benefits of the ACS for the NSF
• In response, CNSTAT provided the following
recommendation
Recommendation 7.5.
The NSF should use the opportunity afforded by the
introduction of the ACS as a sampling frame to reconsider
the design of the SESTAT Program and the content of its
component surveys
20
Motivation for Reconsidering SESTAT
• Coverage
- The American Community Survey (ACS) is being
used to draw sample for the NSCG on a biennial
basis
- The ACS-based rotating panel design of the
NSCG now provides coverage of the inflow of
recent college graduates
21
Motivation for Reconsidering SESTAT
(Continued)
• Timeliness
- Over the years increasing complexity has been
built into the SESTAT design
- There is a need to improve the timeliness of the
data releases
• Cost
- The reality of budgetary pressures now and into
the future
- The rapidly increasing costs of surveys
22
SESTAT Redesign Possibility
• One possibility is to redesign SESTAT to include only
the SDR and NSCG
• In this design, we would include more young college
graduates into our NSCG sample as a replacement for
the recent college graduates sample from the previous
SESTAT design
23
SESTAT Coverage During the 2000 Decade
Current Design
24
Note: SESTAT also provides coverage of the population of U.S. residents with a non-S&E degree
working in an S&E occupation. In the 2010 decade, SESTAT will continue to cover this population,
and will also provide coverage of the non-S&E degree holders working in a non-S&E occupation.
Type of Degree Holder Date Degree Earned SESTAT Survey
Foreign-Earned S&E
Degree Holders
Prior to April 2000 NSCG
After April 2000
U.S. Earned S&E
Bachelor's and Master's
Degree Holders
Prior to April 2000 NSCG
2001-2002 RCG (2003)
2003-2005 RCG (2006)
2006-2007 RCG (2008)
2008-2009 RCG (2010)
U.S. Earned S&E Doctoral
Degree Holders
Prior to April 2000 NSCG & SDR
2001-2002 SDR (2003)
2003-2005 SDR (2006)
2006-2007 SDR (2008)
2008-2009 SDR (2010)
SESTAT Coverage During the 2010 Decade
Potential Design
Type of Degree Holder Date Degree Earned SESTAT Survey
Foreign-Earned S&E
Degree Holders
Prior to January 2009 NSCG
After January 2009 NSCG (2012-2018)
U.S. Earned S&E
Bachelor's and Master's
Degree Holders
Prior to January 2009 NSCG
2009-2011 NSCG (2012)
2012-2013 NSCG (2014)
2014-2015 NSCG (2016)
2016--2017 NSCG (2018)
U.S. Earned S&E Doctoral
Degree Holders
Prior to January 2009 NSCG & SDR
2009-2011 NSCG & SDR (2012)
2012-2013 NSCG & SDR (2014)
2014-2015 NSCG & SDR (2016)
2016--2017 NSCG & SDR (2018)
25
Note: In the 2010 decade, SESTAT will provide coverage of all non-S&E degree holders regardless of
occupation.
Different SESTAT Data Focus –
Recent vs. Young
• The potential design would continue to cover the
recent college graduates population, but at a lower
precision level
• Under the potential design, the focus would change
from an emphasis on estimating recent college
graduates to estimating young college graduates
• To help us evaluate this focus change, we need to
understand how the current data is used and explore
the potential uses of data from the potential redesign
26
Exploration Question
• Can we leverage our use of the ACS-based NSCG to
derive NSCG estimates similar to those presently
provided by the NSRCG?
• As part of this exploration, we identified seven issues
that need detailed evaluation
27
Seven Issues to be Considered
1. The impact of the change in estimation concepts
(i.e., young graduates vs. recent graduates)
• “Recent graduates” refers to individuals earning a college
degree within the past two academic years whereas “young
graduates” refers to individuals below a particular age cutoff
without regard for the number of years since the graduation date.
• The ACS does not collect graduation date, so it is not possible to
target “recent graduates” in the ACS-based NSCG sample. The
ACS, on the other hand, does collect age, so it is possible to
target “young graduates” in the ACS-based NSCG sample
selection effort.
28
Seven Issues to be Considered
(continued)
2. Whether there is enough ACS sample to make this
switch
3. Accuracy of NSRCG vs. ACS-based estimates
4. Outreach to S&E and broader user community
5. Cost implications of NSRCG vs. ACS-based design
6. Impact on other NCSES surveys and data products such
as Women, Minorities and Persons with Disabilities and
Science and Engineering Indicators
7. Completion of evaluation summary report
29
Tentative Schedule Critical Evaluation Completion Date: March 2012
Ongoing Tasks
• Examine the sample availability
• Submission of Federal Register Notice
• Internal and external outreach to NSRCG community
• Examination of the cost implications
• Evaluate the precision of ACS-based estimates
Upcoming Tasks
• Continue outreach to NSRCG community to determine potential
impact on users
• Synthesize evaluation information and develop a summary
evaluation memorandum
• Review by CNSTAT panel 30
Impact on Users
• There will be no significant change in the coverage
of the S&E workforce from SESTAT under the
proposed design
• The SESTAT race and diversity questions will not
change, so the proposed changes will have NO
impact on the race and diversity data that will be
produced from SESTAT
• Long term, the proposed changes will allow
NCSES the ability to consider estimating the
Associate’s degree population
31
Assumptions
• The possibility of transitioning to an ACS-based
NSCG is based on the assumption of continued
funding of the ACS by the U.S. Census Bureau
• In addition, it is based on the assumption of the
continued availability of the ACS as a sampling
frame
• Any change to the funding or availability of the ACS
has severe implications on the feasibility of any
proposed SESTAT design
32
Data Needs and
Analytical Interests
Topic #1 – Recent vs. Young
• What survey is more appealing from a research
and/or policy perspective:
- A survey with a large number of recent
college graduates of all ages
- A survey with a large number of young adults
who are college graduates
34
Topic #2 – Research and Policy
Questions
• What are some of the important research and
policy questions we are answering through the use
of education/occupation statistics?
• What are some of the important research and
policy questions related to education/occupation
statistics for which no data source is available?
• What steps could be taken to address this
unavailable data source limitation?
35
Topic #3 – Emerging Issues
• What are some of the emerging estimation issues
in education statistics?
• Do you consider any of the following issues to be
of analytical interest? - Non-traditional graduates
- Professional master’s degree recipients
- The recent increase of master degree recipients
- The reasons for pursuing a master’s degree
- Community college enrollment
- Pathways ( or )
- S&E pipeline
36
Contact Information
37
John Finamore
National Center for Science and Engineering Statistics
National Science Foundation
(703) 292-2258