PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally&...
Transcript of PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally&...
![Page 1: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/1.jpg)
April 10, 2014
PISA for Development | Technical Workshop
Tania Rajadel World Bank
![Page 2: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/2.jpg)
Outline
q Mo.va.on
q Objec.ve
q Coverage
q Implementa.on
q Survey instrument
q Preliminary findings
![Page 3: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/3.jpg)
MoAvaAon | Improving InformaAon on Skills
q What is the skills profile of the labor force?
q What skills maHer for employment and produc.vity?
q What is the nature and size of skills gaps and mismatches?
q What interven.ons may be considered to improve employability and produc.vity?
To answer these QuesAons about Skills
![Page 4: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/4.jpg)
Collect interna.onally comparable data on different types of skills to inform policy and skill development strategies
Survey of Individuals (HH survey) Supply of skills
• Sample size: 2,000-‐3,500 • Length: 120-‐150 minutes • Representa.ve of urban areas • Popula.on aged 15-‐64
ObjecAve | STEP Skills Measurement Program
4
Survey of Employers (Firms) Demand for skills
• Sample size: 300-‐500 enterprises • Length: 45-‐60 minutes • Formal and informal sectors • Geographic or economic sector based
![Page 5: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/5.jpg)
Colombia
Bolivia
Sri Lanka Vietnam
Lao PDR
Ukraine
Ghana
China (Yunnan Province) Armenia,
Azerbaijan, Georgia
Macedonia
Wave 1 – Started 2011
Wave 2 – Started 2012
Kenya
Countries | Global Scope
Two Waves
![Page 6: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/6.jpg)
STEP Household Survey | ImplementaAon
q Design of the survey instruments q Developed with the support of a mul.-‐disciplinary panel of experts (psychology,
skills assessment, educa.on, and labor market specialists) q Qualita.ve tes.ng and pilots were undertaken to test the instruments in different
cultural se[ngs
q A standardized implementa.on relying on strict technical standards q Adapta.on & transla.on of the survey instruments
q Fieldwork processes q Booklet scoring q Centralized coordina.on and supervision
q One survey methodologist to ensure consistency across country methodologies
q Sampling strategy q Weigh.ng
![Page 7: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/7.jpg)
STEP Household Survey | Key Features
q Target popula.on q Urban popula.on aged 15-‐64
q Household-‐based survey q The Background Ques.onnaire provides a rich array of data, which can
be used independently from the Reading Literacy Assessment data
q Interview sequence
Background QuesAonnaire (80 minutes)
Reading literacy assessment (20 minutes)
• Reading components
• Core assessment
Extended assessment (45 minutes)
• For respondents having passed the Core assessment
![Page 8: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/8.jpg)
Respondent Aged 15-‐64
Background
Health Educa.on & Training
Employ-‐ment History
Family & Language
Skills
Cogni.ve Skills
Self-‐reported numeracy, literacy and wri.ng
Direct Reading
Assessment 1 -‐ Core 2 -‐ Reading components
3 -‐ Booklets
Job-‐relevant skills
Socio-‐emo.onal Skills
Personality traits Behavior Preferences
STEP Household Survey | Survey Instrument
Household InformaAon
Household Roster Dwelling Characteris.cs Random selec.on criteria to select respondent
![Page 9: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/9.jpg)
STEP Background QuesAonnaire | Household InformaAon
q The household roster provides informa.on on respondents’ current living situaAon
q Lists all household members
q Includes their educa.onal aHainment, or whether they can read and write, and their labor market status
q The Family & Language module collects data on family background
q Parents’ educa.onal aHainments
q Siblings, including age, at the age of 12
q Nega.ve shocks experienced by the household at the age of 12
q Languages used
![Page 10: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/10.jpg)
STEP Background QuesAonnaire | Household InformaAon
10
q Dwelling characteris.cs q Dwelling characteris.cs, including ownership
q Water & sanita.on, cooking & ligh.ng energy
q Assets, including books, and animals owned
q Transfers
ü Adapted to country-‐context, as assets correlated with wealth vary across countries
ü Use of asset indices
![Page 11: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/11.jpg)
STEP Background QuesAonnaire | EducaAon & Training Module
q Extensive informa.on collected on educa.on and training q Early Childhood Educa.on q Formal educa.on
q Start age q AHainment and field of study q School (type, loca.on)
q Other forms of training / learning q Literacy program q Professional cer.ficate, work-‐related training q Appren.ceship
q Dropped out and/or interrupted studies and reasons q Reasons for having never aHended school q School-‐to-‐work transi.on q Parental involvement in primary school
![Page 12: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/12.jpg)
STEP Background QuesAonnaire | Employment Module
12
q Provides informa.on on current employment status q Labor market par.cipa.on, including reasons for inac.vity q Employment status q Types of occupa.on and tasks q Earnings q Aspira.ons
q TransiAons in the labor market q First job and previous job
q Informa.on on previous wage jobs for self-‐employed (voluntary/involuntary self-‐employment)
q Labor market success of the self-‐employed q Start-‐up capital
q Earnings, sales
q Business expansion, employees
![Page 13: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/13.jpg)
STEP Background QuesAonnaire | Skills Modules
13
§ Direct assessment of reading literacy based on the Survey of Adults Skills instruments
§ Indirect assessment on individuals’ use of reading, wri.ng and numeracy skills at work and/or in daily life
§ Qualifica.ons required for the job and job learning .mes § Indirect assessment of skills used at work
Job tasks Data—cogni.ve, mental power People—interpersonal Things—physical, muscle power Technology use
§ Personality traits (Big Five and Grit) § Behavior (Hos.le AHribu.on Bias and Decision Making) § Risk and Time Preference
Cogni.ve Skills
Job-‐specific Skills
Socio-‐emo.onal Skills
![Page 14: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/14.jpg)
99% 95% 86% 84%
67%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0 1 2 3 4 5 6 7 8 Score Points
YUN
VNM
LKA (Urban Only)
BOL
LAO (Urban Only)
Preliminary Findings | ETS Literacy Assessment
Results from CORE Assessment: CumulaAve DistribuAon By Country Passing Score = 3 out of 8 ques.ons
50%
Ghana (preliminary March 2014)
67%
ü Adapt ques.onnaires to account for possible low reading literacy rates, in par.cular parent ques.onnaire
Kenya (preliminary March 2014)
![Page 15: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/15.jpg)
Preliminary Findings | Household InformaAon
15
q And adults who dropped out of school are likely to score lower on reading proficiency than those who did not
q Adults whose mothers completed primary school or less display higher school drop-‐out rates than adults with more educated mothers…
![Page 16: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/16.jpg)
Preliminary Findings | Household InformaAon
q A family’s past socioeconomic status can affect how a child develops socio-‐emo.onal skills : adults who come from a higher SES tend to exhibit more grit
q Experiencing more than one household shock before the age of 15 is associated with lower reading proficiency levels
ü Key role of family background Informa.on collected directly from parents would improve data quality ü Recalled informa.on retains an explanatory power
![Page 17: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/17.jpg)
Preliminary Findings | EducaAon
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Ghana Sri Lanka Lao PDR Vietnam Georgia Yunnan Bolivia Armenia
Primary or less Lower Secondary Voca.onal
Lower Secondary General Upper Secondary Voca.onal
Upper Secondary General Post-‐secondary & Higher educa.on
Highest level completed & age at which leq formal educa.on -‐ Respondents aged 20 to 29 -‐
18 16 16 19 19 19 20 19
19 Average age at which leq school
q Respondents aged 15 may be close to ending their formal educa.on
19
ü Ques.onnaire to adapt to contexts in which a majority of 15-‐year olds may be about to leave school
![Page 18: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/18.jpg)
Preliminary Findings | EducaAon
18
Dropout rates – respondents aged 15-‐24 q In most countries, over 20% of respondents aged 15 to 24 dropped out of the highest level they were aHending
q A major reason for dropping out was the lack of money to pay for fees, uniforms, and/or school materials
q Other reasons included having to work or help at home, and pregnancy (Bolivia and Colombia)
q Lack of interest in school is usually reported in lower propor.ons than the above reasons
0%#
5%#
10%#
15%#
20%#
25%#
30%#
35%#
40%#
45%#
Bolivia# Colombia# Lao#PDR# Sri#Lanka# Vietnam# Yunnan# Armenia# Georgia# Ghana#
ü Uncertainty regarding how much further they will study might affect students’ outlook
![Page 19: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/19.jpg)
Preliminary Findings | EducaAon
q More likely to pass the core reading literacy assessment than those who did not
q More likely to have started primary educa.on at the compulsory age (6-‐7 years old)
q Less likely to perceive hos.le intent in others (low hos.lity bias score)
Adults who par.cipated in ECE programs are…
ü ECE maHers in low and middle-‐income countries § In STEP captured by an
indicator variable § More informa.on could
improve insight
![Page 20: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/20.jpg)
Preliminary Findings | CogniAve Skills
q Respondents’ self-‐reported informa.on on how they use their reading skills is different from their actual reading ability…
q And is a very weak indicator of actual reading proficiency
How do self-‐reported and direct assessments of reading skills compare?
![Page 21: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/21.jpg)
Preliminary Findings | Socio-‐emoAonal Skills
q Educa.onal aHainments and socio-‐emo.onal skills are correlated
q Adults with more educa.on tend to score higher in terms of grit
2
2.2
2.4
2.6
2.8
3
3.2
3.4
Armenia Ghana
Pred
icted level of G
RIT (scale 1 to
4)
q All skills maHer for labor market success, including socio-‐emo.onal skills
q Wage workers in Vietnam may increase their earnings by more than 15 percent with an increase in socio-‐emo.onal skills
0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0
One additional year of education Increase in reading literacy Increase in socio-emotional skills Increase in computer skills Increase autonomy skills
Bolivia Vietnam
![Page 22: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/22.jpg)
Preliminary Findings | Job-‐relevant Skills
q Learning outcomes are related to the use of job-‐relevant skills
q For instance, reading proficiency level is associated with higher computer use at work
q Similarly, a worker’s socio-‐emo.onal skills are associated with the use of job-‐relevant skills
q The greater a person’s openness to experience, the greater his or her ability to solve complex problems
20%
30%
40%
50%
60%
70%
80%
Level 1 and Below Level 2 Level 3 Level 4 ǀ 5
Colombia
Vietnam
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
Armenia Bolivia Colombia Ghana Laos Sri Lanka Vietnam Yunnan
Results presented are sta.s.cally significant
Correla.
ons
![Page 23: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/23.jpg)
Main Conclusions
q Family background
q Key role, but might take different forms than in higher income countries
q Parental literacy rates are likely to be lower than their children’s
q A significant propor.on of 15-‐year olds is likely to be on its way out of school
q ECE maHers both for the acquisi.on of cogni.ve and socio-‐emo.onal skills
q Socio-‐emo.onal skills and educa.onal outcomes are correlated
![Page 24: PISA Technical Workshop - STEP Skills Measurement - April ... Skills... · Collectinternaonally& comparable&dataon& differenttypes&of&skills& to&inform&policy&and& skill&development](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f03a9837e708231d40a2786/html5/thumbnails/24.jpg)
Thank you
HDN co-‐Task Team Leaders:
CORE STEP team: Tania Rajadel
[email protected] Sebas.an Monroy Taborda
24
Alexandria Valerio (HDNED) [email protected]
Maria Laura Sanchez Puerta (HDNSP) [email protected]