Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa,...
-
Upload
christopher-peters -
Category
Documents
-
view
222 -
download
0
Transcript of Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa,...
![Page 1: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/1.jpg)
Coaching in Literacy Collaborative and Its Effects on Teachers and Students
Gina Biancarosa, University of OregonAnthony S. Bryk, Carnegie Foundation for the
Advancement of TeachingAllison Atteberry, Stanford UniversityHeather Hough, Stanford University
Annual Meeting of the Society for Research on Educational Effectiveness
March 2010
![Page 2: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/2.jpg)
Key Features of Literacy Collaborative
• Comprehensive school reform program designed to improve elementary children’s reading, writing, and language skills primarily through school-based coaching
• Used in over 700 elementary schools in 200 districts across 26 states
• Intensive professional development– Coaches trained over one year (Lesley University and the Ohio State
University)– Ongoing support from local and national network
• Coaches– In-school professional development courses– One-on-one coaching: viewed as the high leverage activity
![Page 3: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/3.jpg)
• Anatomy of a coaching session– Pre-briefing– Observation– Modeling– Debriefing
• Elements of literacy instruction– Interactive read aloud– Shared reading– Guided reading– Interactive writing– Writing workshop– Word study
Key Features of Literacy Collaborative
![Page 4: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/4.jpg)
Main Research Questions
• Does Literacy Collaborative improve the value-added to student literacy learning?
• Can any effects of Literacy Collaborative be indirectly attributed to coaching via teachers’ changing expertise implementing the instructional practices?
• Can any effects of Literacy Collaborative be directly attributed to coaching? – Does overall coaching activity in a school predict value-
added to student literacy learning?– Does individual teacher participation in coaching predict
value-added to student literacy learning?
![Page 5: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/5.jpg)
Student Data• Value-added analyses focused on grades exposed to LC
professional development (K-2)• Sample: 8576 children, 341 teachers, and 17 coaches in 17
public schools across 8 states in the Eastern U.S. • Children tested in fall and spring for 4 years to measure
change over time in students’ literacy learning using:– Dynamic Indicators of Basic Early Literacy Skills (DIBELS)– Terra Nova in spring
Low Income 46.0%
Race/Ethnicity African-American
LatinoOtherWhite
15.5%5.8%7.2%70.6%
Limited English Proficiency 4.0%
![Page 6: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/6.jpg)
Accelerated Longitudinal Cohort Design6 cohorts studied over 4 years
Year of Study
First Year Second Year Third Year Fourth Year
Fall Spring Fall Spring Fall Spring Fall Spring
K C C D D E E F F
1 B B C C D D E E
2 A A B B C C D DGra
de
Training yearYear 1 of
implementationYear 2 of
implementationYear 3 of
implementation
![Page 7: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/7.jpg)
Our early literacy scale
• Equal differences on scale imply equal differences on the trait measured at any level
• Reported in logits (which describe the probability of a student with a given ability level getting a particular item right or wrong)
• But what do they mean given the particular assessments used?
1
2
3
4
Mean at K entry Names about 30 letters in a minuteVery low phonemic awareness (PA)
Mean at K end & 1st grade entryAccurate and fast letter recognitionGood initial sound PALittle evidence of decoding
Mean at 1st grade end & 2nd grade entryAccurate (not fast) PAReads 50-60 wpmAnswers 1/3 of 1st grade comprehension questions correctly
Mean at 2nd grade endMastery of component skillsReads 90 wpmAnswers 2/3 of 1st grade comprehension questions correctly, 1/3 of 2nd grade questions correctly
![Page 8: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/8.jpg)
Additional Measures
Year of implementationParticipant Construct Instrument 1 2 3Teachers Background characteristics Survey ● ●
Coaching participation Coach logs ● ● ●Frequency of implementation
Coach report
● ●
Expertise of implementation
Coach report
● ● ●
Coaches Background characteristics Survey ● ●Schools Contextual characteristics Survey ● ●
Social-professional network
Survey ● ●
![Page 9: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/9.jpg)
Prior Findings:Coaching sessions per month (n=249)
Coefficient (se) Standardized effect size
Teacher-level moderatorsRole conception 0.048
(0.021)* 0.026
School commitment 0.092 (0.0242)
*** 0.049
10+ years prior teaching experience
-0.099 (0.046)
* -0.053
School-level moderatorsK-2 Staff Size -0.579
(0.110)*** -0.308
Perceived support for LC 0.152 (0.060)
* 0.081
Coach’s prior training 0.148 (0.057)
* 0.079
Teacher influence in school decisions
0.101 (0.064)
* 0.054
* p < .05; ** p < .01; *** p < .001
![Page 10: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/10.jpg)
Prior Findings:Frequency and expertise of implementation (n=249)
ExpertiseCoefficient (se) Standardized effect
sizeInitial status, Teacher-level moderators
School commitment 0.142(0.073)
** 0.252
Prior professional development 0.134(0.078)
* 0.238
≤ 3 years prior teaching experience
-0.534(0.199)
*** -0.951
Growth, Teacher-level moderators
Coaching participation 0.130(0.049)
*** 0.231
* p < .05; ** p < .01; *** p < .001
![Page 11: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/11.jpg)
Value-added Hierarchical Cross-classified Effects Modeling
• Four Levels – time : (students x teachers) : school
– Repeated measures on students (level 1) – Students (level 2) who cross Teachers (level 3) over time – All nested within Schools (level 4)
• The analysis model can be conceptualized as a joining of 2 separate multi-level models
– One two-level model for individual growth in achievement over time, and
– A second two-level model which represents the value-added that each teacher in a school contributes to student learning in that school in a particular year.
![Page 12: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/12.jpg)
Hierarchical Crossed Value-added Effects Model
)_3()_3_()_3(
)_2(
)_2()_2()_2(
)_1(
)_1()_1()_1(
)__(
)__2/1_()__2/1_(
)__1/_()__1/_(
)__()__()__(
)6___(
)5___(
)4___(
)7_()6_()5_()3_()2_(
)6_()5_()4_()3_()2_(
1201201200
1100
1001001000
900
8080800
70
60600
50500
4040400
300
200
100
0100090080070060
050040030020010
0000000
ValueAddedYeardValueAddedYearcValueAddedYear
ValueAddedSummer
ValueAddedYeardValueAddedYearcValueAddedYear
ValueAddedSummer
ValueAddedYeardValueAddedYearcValueAddedYear
ValueAddedBaseTeacherc
EffectSummerBasedEffectSummerBase
EffectSummerKBasedEffectSummerKBase
EffectAcademicYrBasedEffectAcademicYrBasebEffectAcademicYrBase
AdjustmentMetrictalDevelopmen
AdjustmentMetrictalDevelopmen
AdjustmentMetrictalDevelopmen
CohortCohortCohortCohortCohort
LocationLocationLocationLocationLocation
edbY
Individual growth parameters
overall value-added effects
teacher-level school-level value-added effects
![Page 13: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/13.jpg)
Value-added effects by year (prior to adding coaching as predictor)
Year 1 Year 2 Year 3
Average value-added (overall)
.164 .280 .327
Performance improvement
16% 28% 32%
Effect size .22 .37 .43
Ave. student learning growth is 1.02 per academic year
![Page 14: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/14.jpg)
School 95% plausible value-added range
±.23
Variability in school value-added, year 1Average student gain per academic year
No effect
Year 1 mean effect (.16)
High value-added schools
Low value-added schools
![Page 15: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/15.jpg)
School 95% plausible value-added range
±.23 ±.28
Variability in school value-added, year 2Average student gain per academic year
No effect
Year 1 mean effect (.16)
Year 2 mean effect (.28)
![Page 16: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/16.jpg)
School 95% plausible value-added range
±.23 ±.28 ±.37
Variability in school value-added, year 3Average student gain per academic year
No effect
Year 1 mean effect (.16)
Year 2 mean effect (.28)Year 3 mean effect (.33)
![Page 17: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/17.jpg)
Teacher 95% plausible value-added range
±.51
Variability in teacher value-added within schools, year 1
Average student gain per academic year
No effect
![Page 18: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/18.jpg)
Teacher 95% plausible value-added range
±.51 ±.71
Variability in teacher value-added within schools, year 2
Average student gain per academic year
No effect
![Page 19: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/19.jpg)
Teacher 95% plausible value-added range
±.51 ±.71 ±.91
Variability in teacher value-added within schools, year 3
Average student gain per academic year
No effect
![Page 20: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/20.jpg)
Explaining variability in value-added effects
• Tested models with cumulative number of coaching sessions per year (derived from coach logs)– Per teacher– Averaged across teachers at school-level
• Also tested a variety of controls thought to influence teachers’ openness to, participation in, and selection for coaching– Prior use of reform literacy practices– Role conception– School commitment– New to school
![Page 21: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/21.jpg)
Hierarchical Crossed Value-added Effects Model
)_3()_3_()_3(
)_2(
)_2()_2()_2(
)_1(
)_1()_1()_1(
)__(
)__2/1_()__2/1_(
)__1/_()__1/_(
)__()__()__(
)6___(
)5___(
)4___(
)7_()6_()5_()3_()2_(
)6_()5_()4_()3_()2_(
1201201200
1100
1001001000
900
8080800
70
60600
50500
4040400
300
200
100
0100090080070060
050040030020010
0000000
ValueAddedYeardValueAddedYearcValueAddedYear
ValueAddedSummer
ValueAddedYeardValueAddedYearcValueAddedYear
ValueAddedSummer
ValueAddedYeardValueAddedYearcValueAddedYear
ValueAddedBaseTeacherc
EffectSummerBasedEffectSummerBase
EffectSummerKBasedEffectSummerKBase
EffectAcademicYrBasedEffectAcademicYrBasebEffectAcademicYrBase
AdjustmentMetrictalDevelopmen
AdjustmentMetrictalDevelopmen
AdjustmentMetrictalDevelopmen
CohortCohortCohortCohortCohort
LocationLocationLocationLocationLocation
edbY
Individual growth parameters
overall value-added effects
teacher-level school-level value-added effects
Predictors added to baseline and LC value-added effects
![Page 22: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/22.jpg)
Summary of findings
• Only one teacher characteristic significant• Teacher expertise of implementation not
significant• Coaching at the school level not significant• Coaching at the teacher level significant
![Page 23: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/23.jpg)
Teachers’ role conception
• High scorers: Teachers who take an active stance in their professional role in terms of initiating contact and offering help to colleagues
• Higher value-added to student literacy learning in their schools in baseline and Y2
Baseline Year 1 Year 2 Year 3
.049** -.011ns .042* .009ns
![Page 24: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/24.jpg)
Average Value-added of Coaching by year
Year 1 Year 2 Year 3
Average value-added for teacher receiving NO coaching
0.26*** 0.17* 0.14ns
Role conception -.01ns .04* .01ns
Teacher expertise 0.02ns -0.03ns 0.03ns
Value-added per coaching session (cumulative)
-.026* .012* .012*
![Page 25: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/25.jpg)
Average Value-added of Coaching by yearYear 1 Year 2 Year 3
Value-added per coaching session (cumulative)
-.026* .012* .012*
Mean cumulative coaching sessions
2.60 8.96 15.70
Mean coaching value-added
-0.07 0.09 0.19
Unconditional average value-added (overall)
.164 .280 .327
Proportion accounted for by coaching
NA 0.32 0.57
Cumulative coaching sessions min-max
0-12 0-33 0-43
![Page 26: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/26.jpg)
0
10
20
30
40
Cu
mu
lativ
e N
umb
er
of
Se
ssio
ns
pe
r T
ea
che
r
JanY2FebY2
MarY2AprY2
MaJuY2AuSeY3
OctY3NovY3
DecY3JanY3
FebY3MarY3
AprY3MaJuY3
AuSeY4OctY4
NovY4DecY4
JanY4FebY4
MarY4AprY4
MaJuY4
Month of the Study
*Note: Each line represents 1 of the 18 schools in the study
Across the Eighteen Schools, Over TimeAverage Number of Coaching Sessions Accumulated Per Teacher
Across Seventeen Schools, Over Time
17
![Page 27: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/27.jpg)
Minimum coach-ing
25th percentile coaching
50th percentile coaching
75th percentile coaching
Maximum coaching
-0.4
-0.2
0
0.2
0.4
0.6
Year 1Year 2Year 3
Valu
e-ad
ded
0 0 1 5 12
Value-added by coaching, year 1
No coaching effect
![Page 28: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/28.jpg)
Minimum coach-ing
25th percentile coaching
50th percentile coaching
75th percentile coaching
Maximum coaching
-0.4
-0.2
0
0.2
0.4
0.6
Year 1Year 2Year 3
Valu
e-ad
ded
0 4 8 12 33
Value-added by coaching, year 2
No coaching effect
![Page 29: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/29.jpg)
Minimum coach-ing
25th percentile coaching
50th percentile coaching
75th percentile coaching
Maximum coaching
-0.4
-0.2
0
0.2
0.4
0.6
Year 1Year 2Year 3
Valu
e-ad
ded
0 8 14 24 43
Value-added by coaching, year 3
No coaching effect
![Page 30: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/30.jpg)
Summary of findings
• Evidence that the mechanism for improved value-added shifts from over time– Year 1: Coaching has no value-added– Year 2: Coaching begins to add to value-added for
student learning– Year 3: Coaching becomes the primary mechanism for
value-added to student learning• Cumulative coaching explains differences in
teacher value-added effects, but not school effects
![Page 31: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/31.jpg)
Implications
• Coaching explains differences in teachers’ value-added to student learning
• Shift in coaching effects from negative in Year 1 to positive in Years 2 and 3 raises interesting hypotheses but offer no answers– A selection effect (on the part of coach or teacher)– A dosage effect– A change in coaching expertise effect– Unexplored school/coach effects
• Direct positive effects of coaching on students appear to take time to emerge
![Page 32: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/32.jpg)
Limitations
• Limited sample, especially at school level, limits ability to explore contextual mechanisms
• Coaching was embedded in a school-wide reform model
• Professional development for coaches is more intense than in most other models
![Page 33: Coaching in Literacy Collaborative and Its Effects on Teachers and Students Gina Biancarosa, University of Oregon Anthony S. Bryk, Carnegie Foundation.](https://reader036.fdocuments.in/reader036/viewer/2022062308/56649e745503460f94b7477c/html5/thumbnails/33.jpg)
Future Steps
• Continued analyses of current data– Length of coaching session– Focus of coaching session– Observation vs. modeling
• Development and piloting of the Performance-based Assessment of Literacy Coaching (PALC)