Scalable Strategies to Improve Preschool Quality: Experimental … · 2019-05-15 · Quality...

Scalable Strategies to Improve Preschool Quality: Experimental Impacts of the

‘Quality Preschool for Ghana’ Interventions

Sharon Wolf, University of Pennsylvania

Acknowledgements

Partners: Innovations for Poverty Action, National Nursery Teacher Training Center, Ghana Education Service, Sabre Education

Funders: UBS Optimus Foundation, World Bank Strategic Impact Evaluation Fund, Early Learning Partnership, British Academy.

Collaborators: J. Lawrence Aber (NYU), Jere Behrman (Penn)

Teachers, head teachers, caregivers and children who participated in this study.

Introduction Methods Results Conclusions

ECE participation is on the rise globally, and Ghana is a leader in these trends

Sub-Saharan Africa region: 20%

Ghana: 74.5%

2004: National Early Childhood Care and Development Policy; KG curriculum developed.

2007: Expansion of 2 years of pre-primary education (KG1 and KG2) as part of free, compulsory and basic education (fCUBE).


Quality Preschool for Ghana (QP4G)

• In partnership with Ghana Education Service, National Nursery Teacher Training Center, and Innovations for Poverty Action:

• Develop and test a nationally scalable model for teachers and parents with the goal of improving KG quality and children’s school readiness.

• Two parts – Teacher Training and Coaching and Parental Awareness Interventions.


Intervention I – Teacher Training Program

In-service teacher training

• Led by the NNTTC trainers

• 5-day training at the start of the school year

• Refresher trainings 4 and 8 months later

In-classroom monitoring and coaching

• Led by the district education coordinators

• Classroom visits


Intervention II – Parental Awareness Program

3 sessions of parental awareness meetings (1/term)

implemented through PTA

meetings

Video screening

Discussion with focus on:

(1) play-based learning,

(2) parents’ role in child learning, and

(3) encouraging parent-teacher and parent-

school communication



Summer of 2015

QP4G: Research design

240 KG schools (108 public and 132 private)

79(35 public, 44 private)

Control group

Randomization


T1

Teacher training and coaching program


T2

Teacher training and coaching program

Parental awareness about KG learning

6 disadvantaged districts in the Greater Accra Region

Stratification

Total N = 444 teachers, 3,435 children

Teacher training/

Coaching support

Interventions Classroom-level mediators Child outcomes

Teacher

professional

well-being

QP4G Theory of Change

Teacher training/

Coaching support

+

Parental-awareness

meetings

School

readiness

Classroom

Quality


Measures

1. Classroom quality – implementation and teacher-child interactions Video-taped classroom observations

2. Professional well-being Teacher surveys; school administrative records

3. Child school readinessDirect assessments

Impacts are assessed:

• End of implementation year (June 2016)

• One year later (June 2017)

• Two years later (June 2018; preliminary)


Teacher training/

Coaching support

Interventions Classroom-level mediators Child outcomes

Teacher

professional

well-being

Teacher training/

Coaching support

+

Parental-awareness

meetings

School

readiness

Classroom

Quality

Implementation: Are teachers integrating practices from the training in their classroom?

Teacher praises children for positive behavior

Teacher threatens children with or uses a cane on children at least once

Teacher explicitly reminds children of the class rules

Teacher uses a “Circle Time” activity during the lesson (i.e., all children seated together participating in unison)

The lesson consists of a game that facilitated the lesson objectives

Teacher asks students at least two open-ended questions during the class

Teacher incorporates found items as Learning Materials (e.g., bottle caps, milk cartons)

Checklist with 15 teaching practices that were in the training. For example:


Emotional support & behavior management

• Positive climate• Negative climate• Teacher sensitivity/tone• Behavior management• Consistent Routine

Supporting student expression

• Student ideas considered• Reasoning/problem solve• Connections to life• Language modeling

Facilitating deeper learning

• Scaffolding (concept development)

• Quality of feedback • Objectives explicit

Using the Teacher Instructional Practices and Processes System (TIPPS; Seidman et al., 2013; Wolf et al., 2018)

Developed based exploratory and confirmatory factor analysis, we assess impacts on three dimensions of classroom quality.

Classroom quality: Does QP4G improve the quality of teacher-child interactions?


Emotional support & behavior management

• Positive climate• Negative climate• Teacher sensitivity/tone• Behavior management• Consistent Routine

Supporting student expression

• Student ideas considered• Reasoning/problem solve• Connections to life• Language modeling

Facilitating deeper learning

• Scaffolding (concept development)

• Quality of feedback • Objectives explicit

Using the Teacher Instructional Practices and Processes System (TIPPS; Seidman et al., 2013; Wolf et al., 2018)

Developed based exploratory and confirmatory factor analysis, we assess impacts on three dimensions of classroom quality.

Impacts on Implementation and Classroom Quality


0.56***

-0.11

0.65**0.52*

0.61***

-0.16

0.66**

0.09

-1.00

-0.80

-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60

0.80

1.00

Implementationchecklist

Facilitating deeperlearning

Emotionalsupport/behavior

management

Supporting studentexpression

Effe

ct S

ize

(dw

t)

TT TTPA


Teacher professional well-being: Does QP4G improve teacher well-being?

Construct Measures Sample items

Motivation(5 items, α = .64)

Adapted from Bennell & Akyeampong (2007)

I’m highly motivated to:• …help children learn to read and write• …help children develop well socially.

Burnout (11 items, α = .0.85)

Maslach Burnout Inventory (Maslach et al., 1996)

• How often have felt mentally drained from your work.

• How often do you feel fatigued when you wake up in the morning.

Job satisfaction (4 items, α = .72)


• I want to transfer to another school (rev)• I want to leave the teaching• Profession (rev)

Developed based on both new and previously validated survey measures, and analyzed using factor analysis:

Teacher professional well-being: Does QP4G improve teacher well-being?


Motivation(5 items, α = .64)


I’m highly motivated to:• …help children learn to read and write• …help children develop well socially.

Burnout (11 items, α = .0.85)

Maslach Burnout Inventory (Maslach et al., 1996)

• How often have felt mentally drained from your work.

• How often do you feel fatigued when you wake up in the morning.

Job satisfaction (4 items, α = .72)


• I want to transfer to another school (rev)• I want to leave the teaching• Profession (rev)

Developed based on both new and previously validated survey measures, and analyzed using factor analysis:

0.35

-0.40*

0.33

0.11

-0.59***

0.11

-1.00

-0.80

-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60

0.80

1.00

Motivation Burnout Job Satisfaction

Effe

ct S

ize

(dw

t)

TT TTPA

Teacher attrition: QP4G reduced the likelihood that teachers’ leave the school mid-year

The effects were larger in the private sector.

The probability of a teacher leaving the school in the private sector was reduced by 45%.

0.407

0.223**

0.268*

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Pred

icte

d p

rob

abili

ty o

f att

riti

on

Control TT TTPA

Private Schools


QP4G improves children’s school readiness, including academic and social-emotional development


Overall school readiness(composite score)

International Development and Early Learning Assessment (IDELA); Save the Children, 2015.

Early literacy, Early numeracy, Social-emotional, and Executive Function skills

Early literacy(6 item sets, α = .73)

IDELA Print awareness Oral comprehensionLetter identification Phonological awareness

Early Numeracy(6 item sets, α = .71)

IDELA Shape identification One to one correspondenceNumber identification Size / length differentiation

Social-emotional(5 item sets, α = .69)

IDELA Emotion identification Conflict resolutionEmpathy Personal awareness

Executive function(2 item sets, r = .27)

IDELA Working memoryInhibitory control

0.16*

0.11* 0.11*

0.18*

0.11

0.03

-0.03 -0.03

0.13

0.04

-0.25

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

School readiness Early literacy Early numeracy Social-emotional Executive function

Effe

ct s

ize

(dw

t)

TT TTPA

One year later: Impacts on social-emotional outcomes sustained

0.11+

0.04

0.09

0.14*0.11+

-0.060 -0.06-0.04

0.01

-0.060

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

School readiness Early numeracy Early literacy Social-emotional Executive function

TT TTPA


Two years later: Sustained impacts on some domains of development [preliminary]

0.00

0.11+0.08 0.09+

0.15**

-0.10*

-0.04

0.05 0.06

0.14*

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Early numeracy Early literacy Social-emotional Executive function Behavior regulation

TT TTPA

Sustained impacts on academic outcomes depend on the quality of the primary classroom and teacher burnout level.

Impacts on Literacy in Year 3 by Teacher Burnout

0.23*

0.10

-0.04

-0.30

-0.20

-0.10

0.00

0.10

0.20

0.30

Low Med High

Effe

ct s

ize

Primary School Teacher Burnout


“Global goods” for ECE: Implications

• Brief, affordable in-service, teacher training can be successful if teachers are given ongoing support and messages are reinforced throughout the year.

• Teachers continue to use some training activities one year later, but most impacts fade out.

• Cost estimates are $479.65 and $19.19 per teacher and child, respectively.

• First causal evidence that activity-based, emotionally supportive classrooms with opportunities for play improve preschoolers’ development in this context.

• Persistent impacts on academic outcomes depend on subsequent school quality; persistent impacts on behavior regulation for all children.

• Positive impacts should not be assumed when engaging parents; a more intensive approach may be needed.


Thank [email protected]

mailto:[email protected]

Additional Slides

Do experimentally induced changes to quality explain changes in child outcomes?

Teacher training

Intervention

Developmentally appropriate

activities

School readiness

Supporting Student Expression

Emotional Support & Behavior

Management

.034(.022)

.367***(.056)

.100+(.057)

-.011(.022)

.189***(.058)

.066***(.020)

Notes. Standardized coefficients. Model includes covariates.

.057*(.027)

Teacher training

Intervention

School readiness

Supporting Student Expression

Emotional Support & Behavior

Management

Indirect effects: B = .018*; 17% of total effects

Indirect effects: B = .021*; 19% of total effects

.034(.022)

.367***(.056)

.100+(.057)

-.011(.022)

.189***(.058)

.066***(.020)

.057*(.027)Developmentally

appropriate activities

Do experimentally induced changes to quality explain changes in child outcomes?

Teacher perspectives on the parental awareness program

“When the topics were discussed, some of them did not agree. They were like ‘I gave birth to my child so why shouldn’t I beat him if the child is misbehaving?’” ~Teacher A

“Like getting the child some learning materials so that as they are playing they can be learning at the same time. And sometimes they have to stop using the cane but a parent voiced out and said that her kids are stubborn so without the cane... So I told her that with some parents they said the kids are always happy whenever they are around so they should avoid the cane and they will be fine.”

~Teacher B

Some evidence that parental resistance was limited to parents with low education

Impacts of TTPA on children’s school readiness by male household head literacy status

* p < .05, + p < .10.

0.659

0.6440.639

0.6300.638

0.643

0.55

0.57

0.59

0.61

0.63

0.65

0.67

0.69

Male head not literate Male head literate No male head

Control TTPA

+

*

A framework for examining persistence and fadeout in interventions (Bailey, Duncan, Odgers & Yu, 2017)

“Trifecta” skills

1. Malleable

2. Fundamental

3. Would not have developed eventually in the absence of the intervention

Analytic strategy: Impact analysis

• Multi-level models (3-level for child outcomes, 2-level for teacher outcomes with children nested classrooms, classrooms nested in schools).

• Covariates: within sample mobility dummies; district dummies; baseline scores; child gender, child age, grade (KG1 vs. KG2), public/private sector.

• Attrition: Missing data imputed using a large range of covariates, outcome variables, and treatment status according to WWC standards (WWC, 2014).


One year later: Are teachers still using the practices from the training? Does improvement in classroom quality still persist?

0.21*

-0.30

0.36

-0.34

0.25**

-0.42

0.14

-0.68*

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Fidelity checklist Facilitating deeperlearning

Emotionalsupport/behavior

management

Supporting studentexpression

TT TTPA

YESTeachers are still using some of the training

practices

NOImprovements in

classroom quality have faded out or become

negative

Scalable Strategies to Improve Preschool Quality: Experimental … · 2019-05-15 · Quality...

Documents

Transcript of Scalable Strategies to Improve Preschool Quality: Experimental … · 2019-05-15 · Quality...