RTI International SYMPOSIUM: FOUNDATIONAL FIRST FIVE NEED, MEASUREMENT, RESPONSE IN EARLY CHILDHOOD...
-
Upload
leslie-patrick -
Category
Documents
-
view
215 -
download
1
Transcript of RTI International SYMPOSIUM: FOUNDATIONAL FIRST FIVE NEED, MEASUREMENT, RESPONSE IN EARLY CHILDHOOD...
RTI International
SYMPOSIUM:
FOUNDATIONAL FIRST FIVE NEED, MEASUREMENT, RESPONSE IN EARLY CHILDHOOD
CHAIR: RACHEL HINTON, DFID
PAPERS:
LUIS CROUCH, RTI INTERNATIONAL
ABBIE RAIKES, TECHNICAL LEAD, MELQO, UNICEF
MARTIN WOODHEAD, OXFORD UNIVERSITY
UKFIET CONFERENCE, OXFORD, UK, 15-17 SEPTEMBER 2015
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
Stumbling at the First Step: A Triple Crisis at the Foundations of Education Systems
UKFIET Conference
Oxford, UK
Luis Crouch
20 August 2015
Note: author realizes it will be difficult to get through this in 15 minutes. I will skip some slides. Also, some of the slides go quickly as they build on each other. Finally, it is hoped, due to early publication, that some will read ahead of time.
RTI International
Summary of claims
► “Learning crisis” starts with the “Foundational First Five” (FFF)
► Addressing it without resolving the FFF will not work
► Much of the cost is already being paid for
► We know (more or less) what to do – enough to get started
RTI International
Outline
1. Clues as to the problem– Bulge in the early grades– Cross-sectional evidence on correlates with provision of
early childhood care– Reading inability by grade 3, 4
2. Cost considerations
RTI International
What we don’t do
There is vast literature (Heckman, etc.) on the returns to investment in the early years
This is great but…
The paper takes its cue from “planning” sorts of information that MoE planners and policy-makers may be familiar with
“We sensed what you are talking about, but we had never looked at the statistics in this way…”
RTI International
Clue 1: The bulge in the early grades
0.25
0.5
0.75
1
1.25
1.5
1.75
2
2.25
2.5
2.75
Enro
llmen
t Gra
de fo
r Age
Age for Grade and the Early Grades Bulge
RTI International
Results for key four indicators
Medians of key ratios in countries with ratio of Grade 1 enrollment to population > 1.3Ratio or indicator Median in
“high bulge” countries
“Ideal”
Grade 1 enrollment to population aged 7 1.50 ≈ 1 to 1.1
Grade 2 enrollment to Grade 1 enrollment
0.82 ≈ 1 or so
Grade 2 enrollment to population aged 8 1.28 ≈ 1 to 1.1
Gross intake ratio into primary schooling 1.27 ≈ 1 to 1.1
Pre-primary gross enrollment ratio 0.24 ≈ 1.0
RTI International
P1
P2
P3
P4
P5
P6
P7
0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000
Enrollment Pyramid
Ugandan case: What do the most basic numbers tell us?
RTI International
P1
P2
P3
P4
P5
P6
P7
0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000
Enrollment Pyramid
What do the most basic numbers tell us?
What is the size of the job?
RTI International
P1
P2
P3
P4
P5
P6
P7
0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000
Enrollment Pyramid
What do the most basic numbers tell us?
What is the size of the job?
This big?
RTI International
Trick question:
Common claim in Uganda: Survival rate to Grade 5 is only 60% Can this be?
RTI International
P1
P2
P3
P4
P5
P6
P7
0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000
Enrollment Pyramid
What happens if we overlay the population?
RTI International
P1
P2
P3
P4
P5
P6
P7
0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000
Enrollment with Population Overlay
Population of Appropriate Age Enrollment
What happens if we overlay the population?
What do we notice?
Compare population and enrollment carefully.
RTI International
P1
P2
P3
P4
P5
P6
P7
0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000
Enrollment with Population Overlay
Population of Appropriate Age Enrollment
What is the size of the job now?
This small?
What happens if we overlay the population?
RTI International
Can the bulge, and the 120% gross intake ratios be due to over- and under-age intake?
15
Most likely no
Not if it has been going on for a decade or more
One a child is enrolled as new, the child is enrolled, full stop…
You cannot be “new” twice
So the 40% of over-enrolment would have to be new and over-aged or under-aged… But that is not possible…
RTI International
2001 2002 2003 2004 2005 2006 2007 2008 2009 20100
20
40
60
80
100
120
140
160
180
200
Gross Intake Ratio into Grade 1: One Decade
Uganda case… take a look at the intake ratio for a decade…
16
RTI International
2001 2002 2003 2004 2005 2006 2007 2008 2009 20100
20
40
60
80
100
120
140
160
180
200
Gross Intake Ratio into Grade 1: One Decade
Now, what is the “normal intake”? 100%
17
RTI International
Now, what is the “normal intake”? 100%
182001 2002 2003 2004 2005 2006 2007 2008 2009 2010
0
20
40
60
80
100
120
140
160
180
200
Gross Intake Ratio into Grade 1: One Decade
Now suppose you start out with half the kids out of school in 2000…
RTI International
Now, what is the “normal intake”? 100%
192001 2002 2003 2004 2005 2006 2007 2008 2009 20100
20
40
60
80
100
120
140
160
180
200
Gross Intake Ratio into Grade 1: One Decade
… Even with 50% OOS in 2000, in just a few years the over-age due to that would have been taken in, the top triangle compensating for the bottom one, because the system was over-intaking by 60% to 80%.
And then you have all that rectangle to spare: impossible.
RTI International
20Source: DHS Surveys
Yet we know that even in 2000 the total enrollment in all of primary was already > 100% of the population!
RTI International
Perhaps the numbers are exaggerated? Or bad data?
There may be some exaggeration in the numbers
In 2000 and 2011, the official statistics were: 130% and 110% Gross Enrollment Ratio in Primary
We see from DHS estimates of 120% and 100% respectively. Only a 10 point difference in each case.– This is an independent survey and the numerator and
denominator are the same source.– So, unlikely to be exaggerated or bad data.
So if there is some exaggeration, it is relatively minor.
21
RTI International
An interesting other take
Look at aging in school
23Note: am not sure when these data are for, but probably reasonably recent; problem may be smaller today but probably persists
Data from IDS Sussex’s (Keith Lewin)“CREATE” project
24
Look at the peak for grade 1:Very peaked and right at 6 (green dotted lines)
Then by the time you get to grade 3, the peak is for age 9, and curve is much flatter (non-peaked)
It is the system that is over-aging the kids, not late enrollment
Data from IDS Sussex’s (Keith Lewin)“CREATE” project
RTI International
Which means that it is the system that is over-aging the children
It is not late entry that is aging them actively, though there is some late entry
The children age 1.7 years in grade 1 in Moz, 1.25 in Uganda
And age again 1.15 years in grade 2, 1.25 in Uganda
By the time they get to grade 3-4 they are often 1 full year over-age even if they entered at normal age
So the children are over-aging while in school
25
RTI International
Clue 2: All this correlates strongly with lack of ECD The “over-enrollment,” apparent dropout, permanent
“fake” intake of 125%, are all correlated with lack of ECD, lack of preparedness
Bottom Quintile 2nd Quintile Middle Quintile 4th Quintile Top Quintile0
20
40
60
80
100
120
140
160
G1to7 G2toG1 GERPRE INTAKE
GDP per capita groupings
Pre-primary enrollment ratio
RTI International
Internal correlation between key "early years" variables
Grade 1 Enrollment to Population of Age 7
Grade 2 Enrollment to Grade 1 Enrollment
Pre-primary Gross Enrollment Ratio
Gross Intake Ratio into Grade 1
Grade 1 Enrollment to Population of Age 7 1
Grade 2 Enrollment Grade 1 Enrollment -0.64 1
Pre-primary Gross Enrollment Ratio -0.27 0.47 1
Gross Intake into Grade 1 0.86 -0.55 -0.16 1
Note: in the paper these correlations are “controlled” for the effect of overall development as proxied by GDP per capita. The results don’t change much.
RTI International
Clue 3: “Reading crisis”Testing in some 53 language/country combinations: about 50% of kids unableto read any words in the language of instruction
RTI International
Efficiency implications… Countries “with bulge” spend 46% of ed budget on
primary Countries that control the bulge spend 36% of ed budget
on primary Bulge in P1 and P2 in “worst 40” countries = 1.2 years of
education Spend it on ECD! Spending / pupil explains only 9% of variance in
completion rates The “foundational factors” explain 34% of variance in
completion rates Control the bulge, provide ECD, get better completion Fiscally neutral? Maybe. Socially efficient, yes!!!
RTI International
Optional technical note
A regression of the primary completion rate against public expenditure per pupil as a share of GDP per capita finds has R2 of only 0.09.
A regression of completion on the “foundational factor” has R2 of 0.34: the foundational factors are the “efficiency explanation”
Principal Component Coherence of Foundational Factors
Proportion of variance in Primary School Completion Ratio accounted for by the first component
0.67
FactorsCoefficients in Principal Component
“Excess” of Grade 1 Enrollment to Population of Age 7 0.55
Grade 2 Enrollment to Grade 1 Enrollment -0.53Pre-primary Gross Enrollment Ratio -0.36“Excess” of Gross Intake into Grade 1 0.54
RTI International
Suggested discussion questions
For those with field experience in the classroom (especially early childhood):– Does “bulge in grade 1” (due to too-young kids), drop-off to grade
2, etc., jibe with your experience of not-enough ECD and grade 1 being used as substitute?
For those with an interest in data: could the data be wrong?– Have checked against MICS, other ways to confirm?
For those with interest in policy: – Can arguments such as “over-enrollment in grades 1 and 2” =
1.2 “wasted” years of schoolingdo ECD be useful?
RTI International
More Information
Luis CrouchVP and Chief Technical Officer
International Development Group
RTI
+1 202 728 2058
Global Measurement in
Early Childhood Abbie Raikes
Measuring Early Learning Quality and Outcomes (MELQO)UNICEF
UKFIET, Oxford, UK, September 2015
Important Era for Early Childhood Development
• Increasing awareness of the importance of investments in early childhood development
• Many new program models and growing evidence of the importance of ECD through randomized trials and other research studies
• Measurement as one important element to track trends, investments at the population level
Sustainable Development Agenda, Proposed Goal 4, Target 4.2
4.2 by 2030 ensure that all girls and boys have access to quality early childhood development, care and pre-primary education so that they are ready for primary education
Realizing Promise of ECD Requires Good Measurement
Measuring Education in the Sustainable Development Era
• Shift in direction of national monitoring • Emphasis on measurement of learning, not just access• Expansive definitions of learning that include many undefined
concepts• Resources invested in education overall, let alone measurement, are
limited
Data must serve multiple users – globally comparable is one factor among many, and must be able to use the data for improvement
Measurement Framework for Children Birth to 8 Years
Raikes, Britto & Dua, 2014. A Measurement Framework for Early Childhood: Birth to 8 Years of Age, Institutes of Medicine
What is MELQO?
• Consortium following from Learning Metrics Task Force, WHO Birth to 8 Measurement Framework
• Build on existing assessments: Great work to date
• Develop prototypes for measurement of quality and child development/learning between ages 4 and 6, with emphasis on low-income countries
• “Core Team” led by UNICEF, UNESCO, World Bank and Brookings Institution
Vision of what we will achieve …
• Goal of free, open-source items to allow easy use and adaptation • Clear application to decision-making in policy and practice
Flexible, feasible measures to provide on-going information about children’s development and learning and quality of learning environments
Field-testing in 2015 followed by full validation in upcoming years
What do the MELQO tools include?
• Child development and learning (teacher/parent report and direct assessment):
• Pre-academic skills: Math and literacy • Social/emotional skills• Executive Function
• Quality of Learning Environments: 7 Domains• Observation of classroom settings• Teacher characteristics
Framework for Child Development/Learning
Child Development & Learning
InteractionsMaterials
Inclusiveness
Professional DevelopmentTeacher Motivation
ResourcesSchool Leadership
Policy structure, including M&E Framework
Child counts objects
Child Names
NumbersCounts up
to 20
Teacher has training on
math instruction for young children and receives
regular feedback
Teacher works with small groups of
children to teach each child
at individual level
Child has access to counting
materials like blocks and other
small objectsCurriculum with age-appropriate expectations for math
Adequate funding for
preschools to buy materials,
ensure reasonable class sizes
Talk about numbers – open-ended questions and dialogue
If the child is to achieve these learning outcomes …
Classrooms should have these characteristics …
And programs must ensure this happens...
Through policies and funding at the national level
Existence of a strong, effective monitoring and evaluation system with reliable data and incentives to improve
Phases for Testing Measures
• Do items work well? Are they feasible to use and train?
• Can we find a common core, or do items need to be generated everywhere?
Feasible
• Can we link child development/quality in a meaningful way in the item design?
• Can we establish validity for both instruments?
Valid • Are measures used for improvement?
• How do measures link to longitudinal achievement in early grades?
Useful and Predictive
Data Collection to Date
• Partnerships with RTI, Save the Children, USAID and many experts
• Child Development and Learning • Tested in Kenya, Sudan, Bangladesh, Laos, Mongolia, Madagascar• Beginning Tanzania this fall for validation study, followed by nationally-
representative sample in 2016
• Quality• Observational measure and teacher/supervisor interviews developed• Refining and testing in Tanzania in 2016
Interpreting Data
• Cultural influence vs. ability – how best to pull these apart
• Hard to get a normal distribution – many of the tasks tend to be more binary; children either get them or not
• Estimating how sensitive to age – best to have information on context
• Construct validity – are the measures measuring the same construct in all countries?
Kenya: Forward Digit Span by Age: % Of Children Answering Items Correctly at Each Age
Age 4 Age 5 Age 6 70
5
10
15
20
25
30
35
40
45
50
0 1 2 3 4
Sudan: Forward Digit Span: % Of Children Answering Items Correctly at Each Age
Age 5 Age 6 Age 70
10
20
30
40
50
60
0 1 2 3 4
Where to go from here? MELQO Phase 2• On-going technical development
• Defining what is comparable and not• Testing quality instrument and drawing connections with M&E system
• Clarifying workable use for population-based tools• Program evaluations • Occasional use by governments for monitoring • On-going use by governments or other entities as part of continuous quality
improvement
• Getting to “scale”: Strong institutional backing (UNICEF) coupled with clear plans for implementation and alignment with purpose for tools
• Estimating cost
Conclusions
• Demand for ECD measurement is strong
• Important to create tools that are efficient and culturally relevant
• Ensuring use for improvement is a central question
Opportunity for great improvement in coordination and accuracy of ECD data in upcoming years
Placeholder for Martin Woodhead’s PPT