Long-Term Educational Outcomes of Children in the Positive Education Program’s Early Childhood...

Long-Term Educational Outcomes of Children in the

Positive Education Program’sEarly Childhood Centers

Long-Term Educational Outcomes of Children in the

Positive Education Program’sEarly Childhood Centers

Kimberly T. KendzioraAmerican Institutes for Research

Presented September 28, 2005 to the Advisory Board for the

Long Term Outcomes for Children Receiving Preschool Intervention for Behavioral and Developmental Concerns

Project Cleveland, OH

Kimberly T. KendzioraAmerican Institutes for Research

Presented September 28, 2005 to the Advisory Board for the

Long Term Outcomes for Children Receiving Preschool Intervention for Behavioral and Developmental Concerns

Project Cleveland, OH

AcknowledgementsAcknowledgements

Funded by a grant from the U.S. Department of Education’s Office for Special Education Programs

Performed in collaboration with Early Childhood staff from the Positive Education Program

David Osher, Principal Investigator Elizabeth Spier, Bridget Goosby, Yen Chau,

and Fiona Helsel, Co-Investigators

Funded by a grant from the U.S. Department of Education’s Office for Special Education Programs

Performed in collaboration with Early Childhood staff from the Positive Education Program

David Osher, Principal Investigator Elizabeth Spier, Bridget Goosby, Yen Chau,

and Fiona Helsel, Co-Investigators

BackgroundBackground

Behavior plays an important role in the education of children with disabilities

Young children with undercontrolled, challenging behavior are at risk for poor long-term outcomes

Emerging behavior problems can be successfully addressed before school entry

Behavior plays an important role in the education of children with disabilities

Young children with undercontrolled, challenging behavior are at risk for poor long-term outcomes

Emerging behavior problems can be successfully addressed before school entry

The Regional Intervention Program Study

The Regional Intervention Program Study

This was previously the only long-term study of children who received early intervention for behavioral concerns at a 3-9 year follow-up, RIP children worked,

complied, and played at school very much like their typical peers, and complied 82% of the time at home

This was previously the only long-term study of children who received early intervention for behavioral concerns at a 3-9 year follow-up, RIP children worked,

complied, and played at school very much like their typical peers, and complied 82% of the time at home

The RIP Study, continuedThe RIP Study, continued

At a 25 year follow-up: 39 out of 40 had graduated high school None received special education services for

emotional or behavioral disturbances 39 out of 40 were employed

Source: Strain, Steele, Ellis, & Timm, 1981; Strain & Timm, 2001

At a 25 year follow-up: 39 out of 40 had graduated high school None received special education services for

emotional or behavioral disturbances 39 out of 40 were employed

Source: Strain, Steele, Ellis, & Timm, 1981; Strain & Timm, 2001

About the Early Childhood Centers About the Early Childhood Centers

Previously known as the “EIC” program Founded in 1976, the ECCs offer a

professionally administered, parent-operated therapeutic program for children

Parents receive an experiential, coached training program to improve their relationships with their children and to enhance their children’s growth and development

Parents are trained to teach and work with their own children, and later work with new families entering the program

Previously known as the “EIC” program Founded in 1976, the ECCs offer a

professionally administered, parent-operated therapeutic program for children

Parents receive an experiential, coached training program to improve their relationships with their children and to enhance their children’s growth and development

Parents are trained to teach and work with their own children, and later work with new families entering the program

ECC PopulationECC Population

Two thirds of the children are preschool children with a disability (PCD)

Of the remaining third, half of these are under 3 years of age and eligible for an IFSP

The other half of the remaining third, or 1/6 of the children are age 3 or older and “at risk”• May have significant problems at home or in the

community but do not qualify as PCD in the schools• Parenting concerns (DCFS referrals)

Two thirds of the children are preschool children with a disability (PCD)

Of the remaining third, half of these are under 3 years of age and eligible for an IFSP

The other half of the remaining third, or 1/6 of the children are age 3 or older and “at risk”• May have significant problems at home or in the

community but do not qualify as PCD in the schools• Parenting concerns (DCFS referrals)

ECC Essential ComponentsECC Essential Components

Daily classroom/small group experience Daily parent-child play/training sessions Development of home programming Consultation with other care/service providers Weekly theory classes Support from parents who have received

services at ECC Strength based, ecological focus Team based staff with both education and

mental health orientation

Daily classroom/small group experience Daily parent-child play/training sessions Development of home programming Consultation with other care/service providers Weekly theory classes Support from parents who have received

services at ECC Strength based, ecological focus Team based staff with both education and

mental health orientation

ECC “Critical Attitudes”ECC “Critical Attitudes”

Parent implementationNo rejectRe-EDucation principlesCelebrate strengths and small stepsFocus on “What do we do now?” not

“Why?”Appropriate referral to other servicesLifetime commitment

Parent implementationNo rejectRe-EDucation principlesCelebrate strengths and small stepsFocus on “What do we do now?” not

“Why?”Appropriate referral to other servicesLifetime commitment

Research QuestionsResearch Questions

What important school outcomes, such as grades, achievement test scores, attendance, grade retention, disciplinary removals, special education service use, and high school graduation, occur within four groups of children matched at first grade:

What important school outcomes, such as grades, achievement test scores, attendance, grade retention, disciplinary removals, special education service use, and high school graduation, occur within four groups of children matched at first grade:

Matched Child DesignMatched Child DesignFour Children in Each SetFour Children in Each Set

Child 1: Enrolled in the PEP EIC program

Child 4: Typical school mate

Child 3: Began receiving special education services for emotional/ behavioral problems in first grade

Child 2: Enrolled in a school district-basedearly intervention program Question: What are local

norms for academic development for children like this?

Question: What are long-term academic outcomes for children receiving different amounts of a parent-driven preschool behavioral intervention?

Question: What are long-term academic outcomes for children receiving typical district-sponsored preschool intervention?

Question: Does the timing of intervention (preschool vs. first grade) matter?

Research Questions, continuedResearch Questions, continued

What were the observed effects of the Early Childhood Center program, and how were these achieved?

How cost-effective is parent-driven preschool intervention? Data are still being collected for this analysis

What were the observed effects of the Early Childhood Center program, and how were these achieved?

How cost-effective is parent-driven preschool intervention? Data are still being collected for this analysis

Research ParticipantsResearch Participants All children ever enrolled in an ECC were eligible to participate –

approximately 2,400 We were able to locate about 800 families, and of these more than half

returned consent forms. Our consented sample of 389 was submitted to the Cleveland Municipal

School District for matching 103 students had at least some matches, resulting in a Cleveland

sample of 249 students Of the 249, 241 had elementary school data; 180 had middle school

data; 118 had high school data

All children ever enrolled in an ECC were eligible to participate – approximately 2,400

We were able to locate about 800 families, and of these more than half returned consent forms.

Our consented sample of 389 was submitted to the Cleveland Municipal School District for matching

103 students had at least some matches, resulting in a Cleveland sample of 249 students

Of the 249, 241 had elementary school data; 180 had middle school data; 118 had high school data

Limitations of the Research DesignLimitations of the Research Design

Intervening variables: what matters more for long-term outcome: a long-ago intervention, or the accumulation of experiences since that intervention?

Biased group membership: how are ECC families different from those in the other matched groups?

Intervening variables: what matters more for long-term outcome: a long-ago intervention, or the accumulation of experiences since that intervention?

Biased group membership: how are ECC families different from those in the other matched groups?

ECC Service DataECC Service Data

Children with information about their services at ECC

Children with information about their services at ECC

Variable N Average Std Dev Range

Age at enrollment 369 4.0 1.1 1.6 – 6.7

Months through Phase I

42 11.4 8.9 1 – 36.6

Months through Phase II

137 12.0 5.8 2.8 – 29.3

Year of Birth of Study Children

05

1015202530354045

72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

Year of birth

Co

un

t

Gender of Study Children

Male78%

Female22%

Race/Ethnicity ofStudy Children

83%

13%

2%

0%

1%

1%

0%

WhiteAfrican AmericanHispanicAsian AmericanMultiracialOtherUndeclared

EIC Center Where Study Children Were Served

EIC East38%

EIC West62%

Treatment Outcome Ratinggot worse

0%

no change9%

some improvement

60%

significant improvement

31%

Attendance, Truancy, and Suspensions Across

Study Groups

Attendance, Truancy, and Suspensions Across

Study Groups

ProcedureProcedure

Only Cleveland Municipal School District had the capacity to identify matches for the comparison study. Privacy, consent issues proved overwhelming when

only paper student records were available

District-level attendance records and school transcripts were obtained and reviewed for participants and their matched peers.

Only Cleveland Municipal School District had the capacity to identify matches for the comparison study. Privacy, consent issues proved overwhelming when

only paper student records were available

District-level attendance records and school transcripts were obtained and reviewed for participants and their matched peers.

Who Are the Children in this Analysis?Who Are the Children in this Analysis?Gender: Group 1

Boys

Girls

Gender: Group 2

Boys

Girls

Gender: Group 3

Boys

Girls

Gender: Group 4

Boys

Girls

Who Are the Children in this Analysis?Who Are the Children in this Analysis?

Race/Ethnicity: Group 2

White

African American

Latino


White

African American

Latino


White

African American

Latino


White

African American

Latino

Who Are the Children in this Analysis?Who Are the Children in this Analysis?Free/Reduced Price Lunch: Group 1

Free/Reduced PriceLunch

Full Price Lunch

Free/Reduced Price Lunch: Group 2


Full Price Lunch



Full Price Lunch



Full Price Lunch

Importance of AttendanceImportance of AttendanceOn average, the higher a student’s attendance rate

in grades 1–9: the more high school credits they attempted each year

r = .23, p < .01 the more high school credits they completed each year

r = .72, p < .001 the better their yearly high school GPA

r = .73, p < .001

On average, the higher a student’s attendance rate in grades 1–9: the more high school credits they attempted each year

r = .23, p < .01 the more high school credits they completed each year

r = .72, p < .001 the better their yearly high school GPA

r = .73, p < .001

Attendance ResultsAttendance Results

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10 11 12

1

2

3

4

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10 11 12

1

2

3

4

School Grade

Attendance ResultsAttendance ResultsAcross grades 1 through 9, ECC children had

significantly better attendance than all other groups, averaging 93%.

At 90%, children receiving district-based preschool services had significantly higher attendance rates than those receiving initial services in first grade or typically developing peers

The latter two groups had the lowest attendance rates, at 84% and 87% respectively.

Across grades 1 through 9, ECC children had significantly better attendance than all other groups, averaging 93%.

At 90%, children receiving district-based preschool services had significantly higher attendance rates than those receiving initial services in first grade or typically developing peers

The latter two groups had the lowest attendance rates, at 84% and 87% respectively.

Importance of Early TruancyImportance of Early Truancy

Even in the first grade, students who later dropped out of high school had significantly more unexcused absences than those who went on to graduate

• t (42.90) = 3.75, p < .01

Even in the first grade, students who later dropped out of high school had significantly more unexcused absences than those who went on to graduate

• t (42.90) = 3.75, p < .01

Unexcused Absences/TruancyUnexcused Absences/Truancy

0%

5%

10%

15%

20%

1 2 3 4 5 6 7 8 9 10 11 12

1234

School Grade

Truancy ResultsTruancy ResultsECC children had significantly less truancy than the

other groups, averaging 2.1%Across grades 1 through 9, children initially receiving

services in first grade had significantly more truancy than the other groups, averaging 7.6%.

Children receiving district-based preschool and typically developing children had truancy rates at 4.0% and 4.9% respectively.

ECC children had significantly less truancy than the other groups, averaging 2.1%

Across grades 1 through 9, children initially receiving services in first grade had significantly more truancy than the other groups, averaging 7.6%.

Children receiving district-based preschool and typically developing children had truancy rates at 4.0% and 4.9% respectively.

SuspensionsSuspensions

0%

1%

2%

3%

4%

5%

1 2 3 4 5 6 7 8 9 10 11 12

1234

School Grade

Suspension ResultsSuspension ResultsAlthough all four groups had low levels of

suspension, children receiving any early intervention–either ECC or district-based– had the fewest average days suspended at 0.13% and 0.28%, respectively.

Children initially receiving services in first grade and typically developing peers spent significantly more days suspended, averaging 0.47% and 0.77% respectively.

Although all four groups had low levels of suspension, children receiving any early intervention–either ECC or district-based– had the fewest average days suspended at 0.13% and 0.28%, respectively.

Children initially receiving services in first grade and typically developing peers spent significantly more days suspended, averaging 0.47% and 0.77% respectively.

Attendance Study: DiscussionAttendance Study: DiscussionAttendance is about both a child’s willingness to

go to school and a caregiver’s ability to get them there.

Children who received ECC services had better attendance than matched children both with and without identified special needs.

Truancy is not just a high school problem; it can emerge early in elementary school.

Attendance is about both a child’s willingness to go to school and a caregiver’s ability to get them there.

Children who received ECC services had better attendance than matched children both with and without identified special needs.

Truancy is not just a high school problem; it can emerge early in elementary school.

Emerging Results: GPA StudyEmerging Results: GPA Study Same sample: analysis of high school GPA

over time (63 children had high school data) No differences in how GPAs change over

time, but significant differences in GPA by group: ECC children: 2.55 District Preschool children: 2.51 Service at 1st grade children: 1.83 Typically developing children: 1.96

ECC group’s GPAs are significantly higher than those in the latter two groups; not different than district-based preschool group

Same sample: analysis of high school GPA over time (63 children had high school data)

No differences in how GPAs change over time, but significant differences in GPA by group: ECC children: 2.55 District Preschool children: 2.51 Service at 1st grade children: 1.83 Typically developing children: 1.96

ECC group’s GPAs are significantly higher than those in the latter two groups; not different than district-based preschool group

Emerging Results: Dropout StudyEmerging Results: Dropout Study

Same sample as previousNo statistically significant results

Same sample as previousNo statistically significant results

N No Data Graduated Dropped out

ECC 85 61 19 5District Preschool 48 39 5 41st Grade Special Ed 31 25 4 2Typically Developing 85 61 16 8Total 249 186 44 19

Status

Effects of ECC “Dosage” on Educational Outcomes

Effects of ECC “Dosage” on Educational Outcomes

Dosage Study SampleDosage Study Sample

309 ECC children103 who attended Cleveland Municipal schools206 who attended any other district

“Dosage” = __actual days__ scheduled days

Median dosage = .73Sample split into “high” and “low” groups

309 ECC children103 who attended Cleveland Municipal schools206 who attended any other district

“Dosage” = __actual days__ scheduled days

Median dosage = .73Sample split into “high” and “low” groups

Ohio Proficiency Exam Scores: 4th gradeOhio Proficiency Exam Scores: 4th grade

19

75

27

77

25

75

27

90

29

80

0

20

40

60

80

100

Low High

Dosage Level

% P

ass

Exam

Math Reading Citizen Writing Science

Note: all group differences are statistically significant


8

50

15

62

8

57

25

71

8

64

010

203040

5060

7080

Low High

Dosage Level

% P

ass

Exam


8

50

15

62

8

57

25

71

8

64

010

203040

5060

7080

Low High

Dosage Level

% P

ass

Exam


Note: all group differences are statistically significant


36

6573

85

55

73697975

67

0

20

40

60

80

100

Low High

Dosage Level

% P

ass

Exam

Math Reading Citizenship Writing Science

Note: no group differences are statistically significant

Educational Outcomes: Elementary Educational Outcomes: Elementary Effect B Chi-Square R2

Logistic Regression Results

-Repeated Grade -0.711 2.859

-Special Ed. Services 0.539 5.138

-Restrictiveness of Special Ed. Services 1.553

- Fully mainstreamed -0.321

- Partially mainstreamed -0.255

OLS Regression Results

-Number of Days Attended 9.252** 0.036

-Number of Days Tardy -1.604** 0.028

-Number of Unexcused Absences -1.048* 0.024

-Number of Days Suspended -0.014 0.003

-Average Grade in English 0.580** 0.129

-Average Grade in Math 0.214 0.017

*p<.05, **p<.01; positive numbers mean higher values for the high dosage group; negative numbers mean higher values for the low dosage group

Educational Outcomes: Middle School Educational Outcomes: Middle School Effect B Chi-Square R2

• Logistic Regression Results

-Repeated Grade 1.227* 2.979

-Special Ed. Services 0.122 0.111



- Partially mainstreamed -0.318

• OLS Regression Results

-Number of Days Attended 3.904 0.005

-Number of Days Tardy 1.755 0.014

-Number of Unexcused Absences 1.203 0.001


- Average Grade in English 0.031 0.000

- Average Grade in Math -0.062 0.001


Educational Outcomes: High School Educational Outcomes: High School

Effect B Chi-Square R2

• Logistic Regression Results

-Repeated Grade 0.383 0.325

-Special Ed. Services -0.177 0.379



- Partially mainstreamed 0.214

• OLS Regression Results

-Number of Days Attended 14.295 0.044

-Number of Days Tardy -2.699* 0.061

-Number of Unexcused Absences -0.535 0.000


- Average Grade in English 0.47** 0.044

- Average Grade in Math 0.144 0.004


Dosage Study: DiscussionDosage Study: Discussion

Results of group comparison study for attendance are replicated with this alternative methodologyResults may be strongest in elementary school

Children of families who attended ECC more faithfully did better on 4th and 6th grade Ohio Proficiency Exams than low ECC attenders did

Results of group comparison study for attendance are replicated with this alternative methodologyResults may be strongest in elementary school

Children of families who attended ECC more faithfully did better on 4th and 6th grade Ohio Proficiency Exams than low ECC attenders did

Focus Study: How Does ECC Work?

Focus Study: How Does ECC Work?

Participants and ProceduresParticipants and Procedures

Conducted 90 minute focus groups with:10 Parents (2 groups)7 Professional staff2 Paraprofessionals

Conducted 30 minute interviews with 9 former participants

Participants were identified by ECC staff, who were asked to select a representative cross-section of individuals

Conducted 90 minute focus groups with:10 Parents (2 groups)7 Professional staff2 Paraprofessionals

Conducted 30 minute interviews with 9 former participants

Participants were identified by ECC staff, who were asked to select a representative cross-section of individuals

Effects of the ProgramEffects of the Program Improved child behavior and skills

“Getting your kids literacy level up, getting a kid an ability to communicate, getting them able to read, when they can’t function on a more basic a level, the social skill building that happens for the kids is enormous.”

Improved parenting skills “I was better able to deal with the kids and enjoyed

that time at home more.” “When my mom was coming over and giving

incorrect parenting I’d have to correct her. I’d have to share the skills with her and share the skills with the teacher in kindergarten.”

Improved child behavior and skills “Getting your kids literacy level up, getting a kid an

ability to communicate, getting them able to read, when they can’t function on a more basic a level, the social skill building that happens for the kids is enormous.”

Improved parenting skills “I was better able to deal with the kids and enjoyed

that time at home more.” “When my mom was coming over and giving

incorrect parenting I’d have to correct her. I’d have to share the skills with her and share the skills with the teacher in kindergarten.”

Effects of the ProgramEffects of the Program Sense of community

“In PEP everyone had the same problem and everyone was equal and they didn’t judge, they just helped.”

“The power of community, because people spent a significant amount of time there, relationships were formed that I think were tremendously supportive and healing. And I think these continue to this day. I think this is a healing community and an important dynamic.”

“They learn to advocate by meeting parents and families and making friendships. To have that supportive network is an essential component.”

“EIC is the first place where people come and aren’t told that they’re bad parents.”

Sense of community “In PEP everyone had the same problem and

everyone was equal and they didn’t judge, they just helped.”

“The power of community, because people spent a significant amount of time there, relationships were formed that I think were tremendously supportive and healing. And I think these continue to this day. I think this is a healing community and an important dynamic.”

“They learn to advocate by meeting parents and families and making friendships. To have that supportive network is an essential component.”

“EIC is the first place where people come and aren’t told that they’re bad parents.”

Why Does ECC Work?Why Does ECC Work? Knowledge and Skill

“I learned to give positives, you always will love him, but this he did and these are the consequences”

“Also the practice, the guided practice, they do it we give them feedback, and its not like a workshop. You go to a workshop and its not interactive, but at PEP it’s the instruction, the practice, the feedback about the practice, and a big piece too is the sense of confidence.”

Empowerment “I felt more empowered because staff was telling me that I

had skills….I was then able to go into my child’s other settings and be very assertive and aggressive…I was able to go in and advocate in a positive, professional way. I could demonstrate my skills and I could advocate and get changes made.”

“When you leave EIC you know what your rights are. I wasn’t afraid to ask for them because I knew I was entitled to them.”

Knowledge and Skill “I learned to give positives, you always will love him, but this

he did and these are the consequences” “Also the practice, the guided practice, they do it we give

them feedback, and its not like a workshop. You go to a workshop and its not interactive, but at PEP it’s the instruction, the practice, the feedback about the practice, and a big piece too is the sense of confidence.”

Empowerment “I felt more empowered because staff was telling me that I

had skills….I was then able to go into my child’s other settings and be very assertive and aggressive…I was able to go in and advocate in a positive, professional way. I could demonstrate my skills and I could advocate and get changes made.”

“When you leave EIC you know what your rights are. I wasn’t afraid to ask for them because I knew I was entitled to them.”

Why Does ECC Work?Why Does ECC Work? Demystifying Systems

“They get that practice, they’re a part of a team. They get to watch somebody who knows how to talk with a school district and just seeing that they absorb a model.”

Role of Paraprofessionals “They’re so credible. I can say one thing as a professional,

but I’m not going home with the kid. My saying, ‘Research shows…’ is undermined by the fact I don’t put him to bed at night. The paraprofessional says I do go home at night. The paraprofessional can say the tough things.”

Parent to Parent “The key to the success is the parent helping parent under

the guidance of professionals. The parent to parent component is the core of the program.”

Demystifying Systems “They get that practice, they’re a part of a team. They get to

watch somebody who knows how to talk with a school district and just seeing that they absorb a model.”

Role of Paraprofessionals “They’re so credible. I can say one thing as a professional,

but I’m not going home with the kid. My saying, ‘Research shows…’ is undermined by the fact I don’t put him to bed at night. The paraprofessional says I do go home at night. The paraprofessional can say the tough things.”

Parent to Parent “The key to the success is the parent helping parent under

the guidance of professionals. The parent to parent component is the core of the program.”

For Further InformationFor Further Information

Kimberly KendzioraAmerican Institutes for Research

[email protected]

(202) 403-5391

Kimberly KendzioraAmerican Institutes for Research

[email protected]

(202) 403-5391

Long-Term Educational Outcomes of Children in the Positive Education Program’s Early Childhood...

Documents

Transcript of Long-Term Educational Outcomes of Children in the Positive Education Program’s Early Childhood...