Un Buen Comienzo (A Good Start) · ntent 1. The Chilean context and the intersectorial creation of...
Transcript of Un Buen Comienzo (A Good Start) · ntent 1. The Chilean context and the intersectorial creation of...
Un Buen Comienzo (A Good Start)Evolution, results and lessons learned
Andrea Rolla & Marcela Marzolo
Co
nte
nt
1. The Chilean context and the
intersectorial creation of the
Un Buen Comienzo program
2. The evolution of the
intervention and the
evaluation of the program
3. Lessons learned
Share the experiences, results, and lessons
learned from the professional development
program Un Buen Comienzo
Non-profit organization in education
aiming to create and increase learning
opportunities and outcomes for children
from at-risk communities in Chile.
Who we are
6
7
1.The national and
international
context and the
intersectorial
creation of the Un
Buen Comienzo
(UBC) program
2006: The Government Emphasized Early Childhood Education
and Early Childhood in General
International Evidence on the Importance of Early Childhood Education
Need to improve process
quality, beyond structural
quality and access
Why was UBC created?
Harvard University,
Chilean Ministry of
Education, and Fundación
Educacional Oportunidad
lead an Interinstitutional
Technical Council (Mesa
Técnica Interinstitucional),
which defines the
characteristics of the
intervention
How it was created?
An initiative to improve the
quality of early childhood
education in Chile through
professional development
with a focus on classroom
coaching
UBC is defined as:
2.
The Evolution of
the Intervention
and the Evaluation
of the Program
UBC Experimental Trial 2008-2011
• 6 municipalities in the
Metropolitan Region (Santiago)
• 32 schools
• 3,200 children
The CLASS instrument (Pianta, La Paro, & Hamre,
2012), that measures the quality of teacher-child
interactions, was appropriate for the Chilean context.
Quality of Interactions Classroom
Observation Instrument
Beginning of
PreKindergarten
End of
PreKindergarten
End of
Kindergarten
Leyva et. al. ( 2015)
Quality of Interactions Classroom
Observation Instrument
Results: UBC 2008-2011 Positive Impact
on Quality Teacher-Child Interactions
0,3760,432
0,056
0
0,2
0,4
0,6
0,8
1
EmotionalSupport
ClassroomOrganization
InstructionalSupport
0,814
0,459 0,438
0
0,2
0,4
0,6
0,8
1
EmotionalSupport
ClassroomOrganization
InstructionalSupport
Eff
ect
siz
e
***
* ~*
*
~p<.10, *p<.05, ***p<.001
Yoshikawa et al. (2015)
UBC Impact Year 1 UBC Impact Year 2
Results: UBC 2008-2011 Positive Impact
on Quality Teacher-Child Interactions
UBC Teachers Taught More Minutes of High
Quality Language Instruction, with Small but
Significant Impacts on Emergent Literacy at
the End of Kindergarten
UBC Experimental Trial Results
3.8
7.38.9
4.3
12.3* 13.0~
0
2
4
6
8
10
12
14
16
18
20
22
24
0 1 2 3
Nu
mb
er
of
min
ute
s p
er
day
Point of Observation
Comparison Full UBC
Baseline End of Year
1
End of Year
2
Statistical significance levels are indicated as ~ = .10, * = 0.05, ** = 0.01, *** = 0.001
Mendive, Weiland, Yoshikawa, & Snow (2015)
UBC Experimental Trial Results
Statistical significance levels are indicated as ~ = .10, * = 0.05, ** = 0.01, *** = 0.001
Impact for the
Entire Sample
Impacts for Children Who Were
More Likely to Attend School
-0,07 0,09 -0,03 -0,060,002
0,181 0,214
-0,11
-1
-0,8
-0,6
-0,4
-0,2
0
0,2
0,4
0,6
0,8
1
Eff
ect
Siz
es * *
Arbour et al. (2015)
UBC Experimental Trial: Small-Moderate Impact on
Emergent Literacy in Children Who Attend School More
Results and Lessons Learned
from UBC Experimental Trial
Impacts
Teachers:
• In quality of teacher-child interactions and quantity
and quality of language instruction
Children:
• In attention, impulse control, and reduction of
aggressive behavior
• In those who attended school more, better results in
emergent literacy
Lessons Learned
• High percentage of absenteeism in Prekindergarten
and Kindergarten
• Low Instructional Support in classrooms
• Low dosage in terms of language instruction
• Low levels of oral comprehension in children
And much more…
Results and Lessons Learned
from UBC Experimental Trial
Evolution of Un Buen Comienzo
2008-2011 2011-2013
+ School leadership
+ Continuous quality
improvement
+ Attendance
+ Language,
socioemotional
and health
+ Coaching
Evolution of Un Buen Comienzo
2008-2011
+ Language,
socioemotional
and health
+ Coaching
+ School leadership
+ Continuous quality
improvement
+ Attendance
2014-2017
+ Instructional
time
+ Effective
interactions
+ Attendance
+ Coaching +
Video feedback
Evolution of Un Buen Comienzo
2011-2013
2008-2011
+ Instructional
time
+ Effective
interactions
+ Attendance
+ Coaching +
Video feedback
2018
+ Language,
socioemotional
and health
+ Coaching
+ Creation of a
sustainable
Network
Improvement
Community (NIC)
13 municipalities
60 schools
150 teachers
120 school leaders
+ School leadership
+ Continuous quality
improvement
+ Attendance
Evolution of Un Buen Comienzo
2014-20172011-2013
2008-2011
+ Instructional
time
+ Effective
interactions
+ Attendance
+ Coaching +
Video feedback
+ Language,
socioemotional
and health
+ Coaching
+ Creation of a
sustainable
Network
Improvement
Community (NIC)
23 municipalities
153 schools
500 teachers
12.400 students
+ School leadership
+ Continuous quality
improvement
+ Attendance
Evolution of Un Buen Comienzo
13 municipalities
60 schools
150 teachers
120 school leaders
20182014-20172011-2013
What is UBC today?
A professional development program for
teachers and municipal and school
leadership teams that aims to improve the
quality of preschool education through
supporting schools to enhance pedagogical
and leadership practices for greater
language, socioemotional and executive
function development in at-risk children.
ActorsStrategies for
capacity
development
Content and
Methodologies
What is UBC today?
Actors
+ Classroom teams
• Teachers
• Aids
+ Leadership teams:
• Principals
• Pedagogical
leaders
+ Families
+ Municipal
counterparts
What is UBC today?
Strategies for
capacity development
+ Training/strategies
and methodologies
+ Coaching + video
feedback =
pedagogical reflection
+ Collaborative
instances: Learning
sessions, municipal
meetings, visits among
schools.
What is UBC today?
Content and
Methodologies
+ Overall
• CQI (IHI)
• DataWise (Harvard)
+ Specifics
• Effective interactions
(CLASS)
• Instructional time
(Stallings)
• Language and
socioemotional
instructional
strategies
What is UBC today?
36
Learning
Session 1
Learning
Session 3
Learning
Session 2
Model of Network Sharing
Improvement theoryInterested municipalities
and schools
Action periods+ PDSA cycles
+ Coaching + Video
feedback
+ Data analisys
+ Municipal meetings
+ Visits among schools
+ Virtual and onsite training
Working as a
network
promotes
improvements
in ALL
schools
Action periods+ PDSA cycles
+ Coaching + Video
feedback
+ Data analisys
+ Municipal meetings
+ Visits among schools
+ Virtual and onsite training
80% of PK
children and
90% of K
children reach
the “advanced”
category in the
language
evaluation at the
end of the year.
None are in the
“initial” category
Maximization of
instructional time
Promote effective
classroom interactions
Aim Primary Drivers
Improve school
attendance
Theory of improvement
Implementation of integrated
pedagogical plans
Time management in classroom
Use of CLASS (emotional
support, classroom organization,
and instructional support)
Coaching with
video- feedback
Establishment of
Attendance Committees
(one-on-one support)
Universal strategies
(for all students)
Secondary Drivers
80% of PK
children and
90% of K
children reach
the “advanced”
category in the
language
evaluation at the
end of the year.
None are in the
“initial” category
Maximization of
instructional time
Promote effective
classroom interactions
Aim Primary Drivers
Improve school
attendance
Theory of improvement
Implementation of integrated
pedagogical plans
Time management in classroom
Use of CLASS (emotional
support, classroom organization,
and instructional support)
Coaching with
video- feedback
Establishment of
Attendance Committees
(one-on-one support)
Universal strategies
(for all students)
Secondary Drivers
40
Learning
Session 1
Learning
Session 3
Learning
Session 2
Model of Network Sharing
Action periods+ PDSA cycles
+ Coaching + Video
feedback
+ Data analisys
+ Municipal meetings
+ Visits among schools
+ Virtual and onsite training
Improvement theoryInterested municipalities
and schools
Working as a
network
promotes
improvements
in ALL
schools
Action periods+ PDSA cycles
+ Coaching + Video
feedback
+ Data analisys
+ Municipal meetings
+ Visits among schools
+ Virtual and onsite training
42
Learning
Session 1
Learning
Session 3
Learning
Session 2
Model of Network Sharing
Action periods+ PDSA cycles
+ Coaching + Video
feedback
+ Data analisys
+ Municipal meetings
+ Visits among schools
+ Virtual and onsite training
Improvement theoryInterested municipalities
and schools
Working as a
network
promotes
improvements
in ALL
schools
Action periods+ PDSA cycles
+ Coaching + Video
feedback
+ Data analisys
+ Municipal meetings
+ Visits among schools
+ Virtual and onsite training
UBC Model- Coaching and meetings with
school teams
Coaching session with video
feedback focused on improving
quality of interactions and
instructional time
Virtual coaching focused on
following the ideas tested
Session to review and
provide feedback on data
44
Learning
Session 1
Learning
Session 3
Learning
Session 2
Model of Network Sharing
Working as a
network
promotes
improvements
in ALL
schools
Action periods+ PDSA cycles
+ Coaching + Video
feedback
+ Data analisys
+ Municipal meetings
+ Visits among schools
+ Virtual and onsite training
Improvement theoryInterested municipalities
and schools
Action periods+ PDSA cycles
+ Coaching + Video
feedback
+ Data analisys
+ Municipal meetings
+ Visits among schools
+ Virtual and onsite training
UBC Network
Improvement Community
These changes did make a difference
What results have we obtained?
Improvements in pedagogical practices
in teachers and in language, literacy
and socioemotional development and
executive function in children
Learning and
Development
Pedagogical Practices
Language
Executive Function
Socioemotional
Development
Instructional Time
Effective
Interactions
Results Children: Language and Literacy
Development (Woodcock-Muñoz)
0,21
0,3
0,18
0,38
0,27
0,2
0
0,05
0,1
0,15
0,2
0,25
0,3
0,35
0,4
UBC experimental 2008-2011
UBC experimental 20%2008-2011
UBC VI 2011-2013 UBC VI 2014-2015
Eff
ec
t S
izes in
Sta
nd
ard
Devia
tio
ns
Versión UBC
Escritura emergente
Identificación de letras ypalabras
Comprensión de textos
UBC Children
Comparison
Group Children Emergent
ReadingEmergent Writing Emergent Reading
Comprehension
Differences in Emergent LiteracyDevelopment in UBC vs. ComparisonGroup Children
5 years 10 months 6 years 6 months 5 years 3 months 6 years 5 months 5 years 7 months 6 years 11 months
Child Impacts: Executive Function
and Socioemotional Development
Executive function consists of the early skills
that are important in the integration of
cognitive and affective development in
children, due to the fact that they test mental
abilities that help the brain to organize and act
according to the information it receives,
permitting it, then, to plan, organize, pay
attention, prioritize, etc.
0,3
0,2
0,54
0,43
0,49
0,161
0,28
0,48 0,48
0
0,1
0,2
0,3
0,4
0,5
0,6
UBC experimental 2008-2011
UBC VI 2011-2013 UBC VI 2014-2015
Eff
ec
t S
ize iin
Sta
nd
ard
Devia
tio
ns
Version UBC
Cuerda Floja (Control inhibitorio,autorregulación motora)
Comportamiento externalizante
Comportamiento internalizante
Comportamiento pro-social
Atención*
Control de impulsos
Child Impacts: Executive Function
and Socioemotional Development
Improvement in the Quality of Teaching:
CLASS, Effective Interactions, Instructional Support
Threshold of teaching
quality to produce
significant impacts in
child development (3.25)
(Burchinal et al., 2010)2,1
2,3
3,79
1
2
3
4
5
6
7
UBC experimental2008-2011
UBC VI 2011-2013 UBC VI 2014-2015
Instr
uctu
ion
al
Su
pp
ort
Version of UBCCalidad de la enseñanzaal final del proyecto
Example of Effective Interactions: Positive Climate
-16%-13%
-4% -4%
1% 2% 5% 7% 8% 10% 13%17% 19% 19%
22% 22% 22% 24%
38%44% 46%
50%
-40%
-20%
0%
20%
40%
60%
80%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Variation Percentage Instructional Time between April-May and November 2017
Improved their Instructional Time
Reduced their Instructional Time
of UBC teachers improved their
percentages of Instructional time81%
3.Lessons
learned
+ Flexibility
+ Willingness to improve
+ Collaboration with all
stakeholders
+ Permanent evaluation
+ Practice based on
research and experience
on the ground
+ Alignment with
public policy
What elements have
been essential?
And above all:
+ AUTONOMY
for teachers and leaders
to learn, think about, and
propose ideas to improve.
+ COLLABORATION
Joint work in search of
a common goal.
What elements have
been essential?
Thank you!