Effects of the My Teaching Partner Intervention in Secondary School Classrooms
description
Transcript of Effects of the My Teaching Partner Intervention in Secondary School Classrooms
![Page 1: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/1.jpg)
Effects of the My Teaching Partner Intervention in Secondary School Classrooms
Joseph P. AllenRobert C. Pianta
University of VirginiaCo-Collaborators:Amori MikamiAnne Gregory
Project Team:Chris HafenSharon DealJudith WassermanRachel BorenJanetta Lun
![Page 2: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/2.jpg)
ContextNumber of Secondary School Students in U.S.: 24
million
Number of Secondary School Classesbeing taught each week 6 million
% of 9th graders who won’t finish High school by the end of 12th grade 25%
Number of programs in ‘What Works’ Clearinghouse with demonstrated efficacy improving teaching quality enough to improve student achievement in these classrooms
0
![Page 3: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/3.jpg)
Key Questions• Can we identify teacher-student
interaction qualities that predict student engagement and achievement?
• Can we change these qualities?• Will changes lead to sustainable student
achievement gains?• What are the mechanisms of change?
![Page 4: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/4.jpg)
Key Questions• Can we identify teacher-student
interaction qualities that predict student engagement and achievement?
• Can we change these qualities?• Will changes lead to sustainable student
achievement gains?• What are the mechanisms of change?
![Page 5: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/5.jpg)
Classroom Learning Assessment & Scoring System- Secondary (CLASS-S)
Emotional Support
Positive ClimateTeacher Sensitivity
Regard for Adolescent Perspectives
Negative Climate
Instructional Support
Instructional Learning FormatContent
UnderstandingAnalysis & Problem
SolvingQuality of Feedback
Classroom Organization
Behavior ManagementProductivity
Student OutcomesStudent Engagement
![Page 6: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/6.jpg)
Evaluation Design• 43 teachers within 8 schools (640 students)
(The control condition in an RCT).
• 1 focal classroom selected per teacher
• Predicting Future Achievement after Covarying Baseline Achievement Test Scores
• Teacher Demographics:– 64% female– 83% White, 8% African-American; 6% Mixed-Ethnicity; 3% Other– 54 middle school, 34 high school– 35% BA degree; 65% at least a year of course work beyond BA– Average 8 years of teaching experience
![Page 7: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/7.jpg)
Classroom Characteristics• School type: 39% High school;
61% Middle School
• Subject: 52% Language/Social Studies; 48% Math/Science
• Average class size: 23 students
• Gender: 47% girls 53% boys
• Ethnicity: 23% African American2% Asian4% Hispanic70% European-American
![Page 8: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/8.jpg)
Observational Assessment of Classroom Environment
• Videotaped observations of a classroom– spread throughout course of year
• Two 20-minute segments per class session/tape– Each tape rated by 2 raters
• Coded Using CLASS-S System• High inter-rater reliabilities; ICC’s range from
– .73 - .82 for overarching domains– .50 -.78 for specific dimensions (all but one dimension > .64)
![Page 9: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/9.jpg)
Student Academic Success• Score on State “Standards of Learning” End of
Year Subject Test
• The measure by which schools/students are judged for accreditation/graduation.
• Extensive seven-year validation/standardization process.
![Page 10: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/10.jpg)
Analytic Approach• Multi-level modelling • All models covary:
– Student factors:• Grade level• Gender• Family poverty status
– Classroom factors:• Classroom size
– Teacher Factors• Teacher experience• Teacher education• Teacher gender and race
• Moderating effects of covariates were also examined.
![Page 11: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/11.jpg)
Predicting Student AchievementAchievement
Emotional Positive Climate .22**Support: Negative Climate -.04
Teacher Sensitivity .16*Regard for Adol. Perspectives .21**
Classroom Behavior Management .06Organization: Productivity .15
Instructional Content Understanding .12Support: Analysis & Prob. Solving .18*
Instructional Learning Formats .22**Quality of Feedback .09
Composite of Significant Dimensions Above .32***
![Page 12: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/12.jpg)
Key Questions
✔Can we identify teacher-student interaction qualities that predict student engagement and achievement?
• Can we change these qualities?• Will changes lead to sustainable student
achievement gains?• Why?
![Page 13: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/13.jpg)
MyTeachingPartner Overview• Consultant and teacher work together
using the CLASS-S in cultivating: – Observation– Reflection– Development of knowledge and expertise
Classroom Observation Teaching Practice
Knowledge Expertise Support
![Page 14: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/14.jpg)
The Steps of the Consultancy
![Page 15: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/15.jpg)
MTPS Website
www.mtpsecondary.net
![Page 16: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/16.jpg)
Detailed Video Examples
www.mtpsecondary.net
![Page 17: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/17.jpg)
Key Questions
✔Can we identify teacher-student interaction qualities that predict student engagement and achievement?
• Can we change these qualities?• Will changes lead to sustainable student
achievement gains?• What are the mechanisms of change?
![Page 18: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/18.jpg)
Evaluation Design88 classrooms
45 Tx. 43 Control(Classrooms Randomized within school)
2237 Students
Assessed Across 2 Years
![Page 19: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/19.jpg)
Evaluation DesignTreatment group:
Year 1:– Introductory Workshop (late summer)– Ongoing consultancy– ~ 2 days total in-service timeYear 2:– Booster Workshop (late summer) only + Web site
access• Control group:
Usual in-service practice.
![Page 20: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/20.jpg)
Intervention Effect on Change in Classroom Qualities
Intervention Target January MarchApril/May
Overall Teacher-Student Interaction Composite
ns
Student Engagement ns
![Page 21: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/21.jpg)
Intervention Effect on Change in Classroom Qualities
Intervention Target January MarchApril/May
Overall Teacher-Student Interaction Composite
ns ns
Student Engagement ns ns
![Page 22: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/22.jpg)
Intervention Effect on Change in Classroom Qualities
Intervention Target January MarchApril/May
Overall Teacher-Student Interaction Composite
ns ns .19*
Student Engagement ns ns .34*
![Page 23: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/23.jpg)
Year 1 Change in OverallTeacher-Student Interactions
Pre-Test Post-Test3.553.603.653.703.753.803.853.903.954.00
Control GroupMyTeachingPartner
Standardized Effects: Baseline = .45***Intervention = .19*
MTPS participation predicts higher quality teacher-student
interactions
![Page 24: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/24.jpg)
Key Questions
✔Can we identify teacher-student interaction qualities that predict student engagement and achievement?
✔Can we change these qualities?• Will changes lead to sustainable student
achievement gains?• What are the mechanisms of change?
![Page 25: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/25.jpg)
Year 1Intervention Effects on Achievement
• No relation of intervention to either baseline or exit achievement test scores in Year 1 (all p’s > .35).
• Why?– No evidence we changed the classroom until the
very end of the year when most teaching was past.
![Page 26: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/26.jpg)
Year 2Change in Achievement
Pre-Test Post-Test460.00465.00470.00475.00480.00485.00490.00495.00
Control GroupMyTeachingPartner
Standardized Effects: Pre-test = .54***Intervention = .22*
MTPS is predicting increases in End of Course Achievement
Tests
![Page 27: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/27.jpg)
Year 2Intervention Effects on Achievement
• Real-world effect size = .22 SD increment in Achievement Test scores • Average ‘Bump’ of students in MTP from 50th to 59th percentile in
achievement• If effect applies equally at all parts of achievement spectrum (as
appears to be the case): a .22 SD boost would reduce failure rates from:
14% without the intervention to
10% with it
Reducing the number of failing students each year by 29%
*** This occurs in the year AFTER the intervention year (i.e., sustainability), across diverse subject matter/content areas.
![Page 28: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/28.jpg)
Key Questions
✔Can we identify teacher-student interaction qualities that predict student engagement and achievement?
✔Can we change these qualities?
✔Will changes lead to sustainable student achievement gains?
• What are the mechanisms of change?
![Page 29: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/29.jpg)
A Preliminary Mediational Analysis
“My Teaching Partner”
Intervention
Intervention
Observed Change in Student
Achievement
*
Environmental Outcome
![Page 30: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/30.jpg)
A Preliminary Mediational Analysis
“My Teaching Partner”
Intervention
InterventionIntervention
Target
Observed Change in Student
Achievement
Environmental Outcome
??
![Page 31: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/31.jpg)
Mediational Analyses• Assessed via Multi-level Structural Equation
Modelling, followed up via parametric bootstrapping analysis (Preacher et al., 2010)
• Focus on target of intervention (Teacher-student interactions assessed via CLASS-S)
• Using Centered/Standardized data for ease of interpretation.
![Page 32: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/32.jpg)
“My Teaching Partner”
Intervention
InterventionIntervention
TargetStudent
Outcome
.37** .16**
Change in Student Achievement
MTP-S Effect as Mediated via Observed Interactions
Initial Model (Simple Direct Effects)
.12*
Observed Teacher-Student
Interactions
![Page 33: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/33.jpg)
“My Teaching Partner”
Intervention
InterventionIntervention
Target
.06* *
StudentOutcome
.37** .16**
Change in Student Achievement
MTP-S Effect as Mediated via Observed Interactions
.06 ns
Initial Model (Simple Direct Effects)Final Model (Including Mediated Effect)
.12*
Observed Teacher-Student
Interactions
![Page 34: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/34.jpg)
Limitations• Design only supports causal interpretations for outcomes, not
for mediating processes with analyses thus far.
• Some Attrition Took Place (though it was unrelated to the intervention in every possible way we could test).
• Modest statistical significance with small sample
![Page 35: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/35.jpg)
Conclusions• We CAN identify elements of the classroom environment that predict
student achievement.
• We CAN change these environmental factors.
• If we do, student achievement will change as well, eventually.
• Changes can be sustained over time and in new classrooms, post-intervention.
• We can identify potential mechanisms of change linked to the intervention.
• Which has implications for cost effectiveness…
![Page 36: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/36.jpg)
*BOE = Back of Envelope
Potential Significance – Costs vs. Benefits(BOE* Calculation)
Resources per classroom Estimated Cost
20 Teacher hours No additional cost to system (in lieu of Regular In-service)
1 Teacher-consultant per 20 teachers Maximum of $3,500 per teacher including benefits (Potentially offset by ongoing teacher supervision personnel costs)
Video equipment $200 per teacher
TOTALS: Maximum of $3,700/23 children = $160/child (i.e., < 2% of annual per pupil expenditures) **
BenefitsAverage ‘Bump’ in achievement of ALL students
from 50th to 59th percentileReduction of 1 course failure per classroom
(**Results may apply to multiple classrooms taught by a teacher)
![Page 37: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/37.jpg)
MyTeachingPartner Secondary
Replication is ongoing with the support of IES
Further information available at:www.myteachingpartner.net