Post on 06-Aug-2020
4/29/2015
1
Products Showcase
11
April 29, 2015
Facilitators
Melissa MackWDQI Project Manager
Social Policy Research Associates
222
Vinz KollerWDQI Director
Social Policy Research Associates
It’s getting HOT in here…
Avoiding the fire drill:
• Just use your phone
• Deactivate microphones on all devices
• Don’t place the line on hold
33
• Phones will be muted (To unmute, press *6)
3
4/29/2015
2
444
Illinois Presenters
Patti SchnoorWDQI Project Director
IL Dept of Commerce & Economic OpportunityPatricia Schnoor@illinois gov
555
Patricia.Schnoor@illinois.gov
Dr. George PutnamChief Economic Analyst
Illinois Department of Employment Securitygeorge.putnam@illinois.gov
Illinois Data Quality
April 2015
ULINE:ULINE:
4/29/2015
3
Illinois Data Quality
April 2015
ULINE:ULINE:
Share
Clean + Change
ExploreShape
Customer
Universal Portal
Integration: Continuous Process
Longitudinal Data System
EnrichCombine
MIS Data
Customer Vault
Merging and sharing data between agencies is an ongoing process. 8
Agencies Programs Services
WorkforceWorkforce
Illinois Department of Commerce and Economic Opportunity (DCEO)
• WIOA Title I• Trade
• Career Services• Employment Services• Training
Illinois Department of Employment Security (IDES)
• Trade• Unemployment Insurance • Labor Market Information (LMI)
• Unemployment benefits• LMI
HumanHuman ServicesServices
WDQI & LDS Data Sources
MIS Data
Illinois Department of Human Services (IDHS)
• TANF• Rehabilitation Services
• Public assistance benefits• Disability services
EducationEducation
Illinois State Board of Education (ISBE)
• Pre‐Kindergarten –Grade 12
• Education program/course approvals
Illinois Community College Board (ICCB)
• Adult Education• Community Colleges
• Education program/course approvals
Illinois Student Assistance Commission (ISAC)
• Financial Assistance • Financial assistance
Illinois Board of Higher Education (IBHE)
• Private Postsecondary and Public Universities
• Education program/course approvals
9
4/29/2015
4
Overview of Workforce Narratives
Conventional Narrative Enhanced Narrative
Long‐term Gains Sustainable Results
Clean + Change
Quantitative Workforce Measures
DCEO IDES
Qualitative Workforce Measures
Career Placement Job Attachment Return on
Investment Gap Analysis Competitive
Advantages
ShareExplore
Enrich
Shape
Combine
10
g
Workforce Outcomes: Policy Implications Reference Group and Trend Comparisons
Employment and Training Program Exit Reasons Counts for Program Year 2012 Quarter 2 “Q2”
Education/Credential/Military 469 7.4%
Entered Employment 3,073 48.2%ntered mployment 3,073 48. %
Client Terminate 1,069 16.8%
Other 1,759 27.6%
Total 6,369 100.0%
11
Workforce NarrativesConventional Narrative Enhanced Narrative
•Job‐based measures•Unit of analysis‐ pairing SSN and Employer UI Account
•Workforce outcomes
•Individual‐based measures•Unit of analysis‐ SSN
•Workforce outcomes
•Outcome comparisons consistent with labor market status•Time‐series and cross‐sectional perspectives on employment and earnings by job attachment and hiring behavior
•Outcome comparisons divorced from labor market status•Cross‐sectional perspective on employment and earnings can be misleading
12
4/29/2015
5
Workforce NarrativesConventional Narrative Enhanced Narrative
•Analytical Themes•Job Hiring•Job attachment•Monthly earnings•Near‐term measure development‐
l l b d l b h ld
•Analytical Themes•Employment counts•Monthly earnings
multiple jobs vs dual job holding, primary job, turnover rate …
•Policy Implications of Workforce Outcomes
•Detailed reference group comparison: http://qwiexplorer.ces.census.gov/
•Policy Implications of Workforce Outcomes
•Constrained reference group comparisons
13
Conventional Narrative: Employment Counts
Employment count of participants that Exited Employment and Training Program Program Year 2012, Quarter 2
with Employm
ent
14
Count of Participants w
Quarterly Incremental Reporting
Exiters that Entered Employment
Conventional Narrative: Monthly Earnings
Participants
Earnings estimates are confounded by partial employment and multiple jobs in a single quarter
Earnings of participants that Exited Employment and Training Program Program Year 2012, Quarter 2
15
Earnings of P
Exiters that Entered Employment – Earnings from each of exiter’s employment records distinct.
Exiters that Entered Employment – Earnings from all of exiter’s employment records combined.
Quarterly Incremental Reporting
4/29/2015
6
Enhanced Narrative: Employment vs. Job
Combining data allows Decision Makers and Stakeholders the ability to see the big picture and trends in outcomes.Approximately 15% ‐ 20% of jobs are held by individuals with more than one job.
loym
ent/Job Combine
16
Pre‐exit and Post‐exit Em
p
Quarterly Incremental Reporting
Exiter’s Employment Exiter’s Jobs
Enhanced Narrative: Hiring Activity
Prior to exiting, trainees tend to hold multiple jobs and did not maintain stable employment. After exiting, trainee hiring activity decreases as they find stable jobs. Post‐exit trainees are more likely than other IL workers to be hired by an employer for whom they did not work for previously.
of Jobs
17Quarterly Incremental Reporting
Percentage o
Exiter Hires –.New EmployerExiter HiresIllinois Hires – New EmployerIllinois Hires
Enhanced Narrative: Job Attachment for
2 Consecutive Quarters
Exiters exhibit 2Q job attachment similar to IL workers within 2 quarters of exit
mploym
ent
18
Percentage of Em
Illinois Jobs Exiter Jobs
Quarterly Incremental Reporting
4/29/2015
7
Enhanced Narrative: Job Attachment for
3 Consecutive Quarters
Exiters exhibit 3Q job attachment similar to IL workers within 4 quarters of exit
mploym
ent
19
Illinois Jobs Exiter Jobs
Percentage of Em
Quarterly Incremental Reporting
Enhanced Narrative:Average Monthly Earnings by Job Attachment
Enrich
Stable earnings measure (3Q) reduces the impact of estimation problems due to partial employment and multiple jobs in a single quarter
ngs
20
Participant Earnin
Exiters that Entered Employment – Earnings from each of exiter’s employment records distinct.
Exiters that Entered Employment – Earnings from all of exiter’s employment records combined.
Quarterly Incremental Reporting
Exiters ‐ Earnings from jobs held for 3 Quarters
Enhanced Narrative:Average Monthly Earnings (3Q Job Attachment)
Earnings reflect a distinct seasonal pattern for IL workers that confounds point‐in‐time comparisons; over‐the‐year earnings gains are higher for Exiters (12%) than IL Workers (2%)
ngs
21
Participant Earnin
Quarterly Incremental Reporting
Explore
4/29/2015
8
Enhanced Narrative:Average Monthly Earnings by Industry
Earnings reflect industry‐specific seasonal pattern; earnings disadvantage of health jobs is more severe among Exiters (‐20%) than IL Workers (‐14%)
ngs
22
Participant Earnin
Quarterly Incremental Reporting
Enhanced Narrative:Average Monthly Earnings by Age within Industry
Older Exiters earn less (‐8%) than other Exiters in Health jobs; earnings disadvantage may be growing
mploym
ent
23
Percentage of Em
Quarterly Incremental Reporting
Enhanced Narrative:Average Monthly Earnings by Age within Industry
Older Exiters earn less (‐8%) than other Exiters in Health jobs; Older IL Workers earn more (16%) than other workers in Health jobs
mploym
ent
24
Percentage of Em
Quarterly Incremental Reporting
4/29/2015
9
The Vision of the Future
siness Value
ReportingWhat
happened?
AnalyzingWhy did it happen?
PredictingWhat will happen?
ActivatingMake it happen.
OperationalizingWhat is happening
now?
Automated k
Insights
Action
Bus
Degree of Intelligence
Batch Reports
happened?
Ad Hoc
Predictive Models
Link to Operational Systems
Linkages
Analytics Maturity Model
25
George PutnamGeorge.putnam@illinois.gov
26
Patti SchnoorPatricia.schnoor@illinois.gov
272727
4/29/2015
10
Indiana Presenters
Carol O. Rogers
282828
Deputy Director Indiana University Business Research Center,
Indiana Universityrogersc@iupui.edu
Indiana Presenters
Allison LeeuwDirector, Research and Analysis Division
Indiana Department of Workforce DevelopmentALeeuw@dwd IN gov
292929
ALeeuw@dwd.IN.gov
Thea EvansWorkforce & Education Analyst Indiana Business Research Center
tevans2@iu.edu
4/29/2015
1
A Tranche Approach to “Closing the OccupationClosing the Occupation Data Gap”Indiana Department of Workforce Development and
the IBRC at Indiana University
April 29, 2015
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 1
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 2
From Frustration, A Plan• Through direct and indirect (predictive) means, we will apply the dimension of occupation titles/codes to wage records.
• This will allow the tracking of career pathways and the intersection of WIA training and other education along that pathway and measureWIA training and other education along that pathway and measure outcomes.
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 3
4/29/2015
2
Current State• Of the 13 million Indiana workforce person‐records, only 13.7% have an occupation code (ranging from 2 digit to 6 digit) and that code may have been applied prior to 2010. These come from unemployment claims records, TrackOne and Indiana Career Connect.
• First‐time indicates the first instance of an occupational code for that person for the following summary metrics.
• 77.8% with first‐time code prior to 2010
• 17.1% with first‐time code post 2010
• 5.1% with no date, but a code
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 4
Phase I• Link self‐reported occupation from Unemployment forms to detailed SOC codes. Occupation data comes from Uplink claimant self‐service center, Indiana Career Connect (ICC) and TrackOne customer service center.
• Unemployment claims from 2014 and later contain 6‐digit SOC codes
• Unemployment claims prior to 2014 only contain 2‐digit SOC codes
• ICC work history contains 6‐digit SOC codes
• TrackOne work history contains a mixture of 6‐digit SOC codes and free‐form occupation title inputs
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 5
Phase I• Bridge licenses (PLA) to occupation and wage records. Applying for and obtaining a state license or certificate does not necessarily mean that the license holder is actually working in that occupation, so care will be taken to build license to appropriate‐NAICS (at the 5‐digit level) to first determine if the license holder is working in that occupationif the license holder is working in that occupation.
• License data available from circa 1940 to present
• Approximately 240 licenses for Indiana (including temporary and limited licenses)
• Database includes the status and issue and expiration dates of the license
• DWD provided a crosswalk to link license type and SOC codes
• OES staffing patterns will be used to link the SOC codes by license type to the associated NAICS codes
• The corresponding SOC code to will be assigned to the wage record of the license holder if there is a match between the license type and NAICS code from the wage record
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 6
4/29/2015
3
• Complete the public employment (approx. 750,000 records with secular job titles) bridge to occupation titles/codes and add to the warehouse records
• Data available for 2012 ‐ 2014
Phase I
• Data will be cleaned and standardized by occupation grouping (managers, assistants, custodians, drivers, etc.)
• SOC codes will be assigned based upon the occupation title and the department of employment
• If a detailed occupation code isn’t able to be determined based upon the information, the record will be assigned to the “All Other” group (e.g. 25‐1199: Postsecondary Teachers, All Other)
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 7
• Compile data to be used for predictive modeling• OES NAICS staffing patterns for Indiana, Economic Growth Regions, and counties. Compiled by DWD.
• Currently possible for 5‐digit NAICS
• Data available from 2009 – 2014
Phase 2
Data available from 2009 2014
• Includes wage distributions for each occupation
• National OES staffing patterns
• Data available for 2 to 4‐digit NAICS
• Data available from 1988‐2014
• Includes major industries of employment and wage distributions for each occupation
• O*NET data for occupations• Includes KSAs, educational requirements, and projected growth for each occupation
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 8
Phase 2• Build a master crosswalk to link data from all sources
• Evaluate the Perkins IV (Carl D. Perkins Career and Technical Education Act of 2006) series of crosswalks (http://www.careertech.org/crosswalks) that link SOC 2000 d NAICS d (2 3 4 d 5 di it ) CIP d d th 16SOC 2000 codes, NAICS codes (2, 3, 4, and 5 digits), CIP codes, and the 16 Career Cluster codes.
• Many of these tables were created prior to 2010 and do not contain updated SOC or NAICS codes. 4 Tables were updated with a grant from the National Research Center for Career and Technical Education (NRCCTE) in 2012, and they produced a report on the updating and validation process of the tables.
• Need to determine whether these sources can be used as a base.
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 9
4/29/2015
4
Phase 3• Constructing Career Pathways
• Tackle the temporal construct for the “career pathway”. Determine methods of measuring multiple occupation codes on a temporal basis so that we can measure occupational changes over time based on industry, occupation and wages.
• We would then be able to look at the education dimension based on the attainment (from a public college) of a certificate or degree (by level).
• Test whether the efficacy of using the Classification of Instructional Programs (CIP) taxonomy to industry enhances accuracy
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 10
Tranche MethodOne part or division of a larger unit, from the French tranche “to cut, to slice”; a term often used in finance. We use it here because we are taking slices or slivers of many sources to create a modeled approach that combines direct and indirect assignments of occupations.
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 11
The Tranche Method
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 12
4/29/2015
5
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 13
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 14
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 15
4/29/2015
6
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 16
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 17
4/29/2015 IDWD and IBRC Partnership for WDQI Round 3 18
4/29/2015
7
Thank You!
1919