Supporting Implementation Fidelity: Coaching Early Educators

Lise Fox, University of South FloridaMary Louise Hemmeter, Vanderbilt UniversityPat Snyder and Tara McLaughlin, University of

Florida

Two Intervention-Based ProjectsEmbedded Instruction for Early LearningTools for Teachers (TfT)

Evaluating the Efficacy of the Teaching Pyramid Model: Using the TPOT

Helping early childhood practitioners use

classroom prevention and intervention strategies

that promote young children's social-emotional development and prevent

challenging behavior

Helping preschool teachers in early learning

programs learn how to use effective supports

and instruction for young children with disabilities

According to NPDCI, the key components of professional development include:

 • (a) the characteristics and contexts of the learners (i.e., the "who" 

of professional development, including the characteristics and contexts of the learners and the children and families they serve);

• (b) content (i.e., the "what" of professional development; what professionals should know and be able to do; generally defined by professional competencies, standards, and credentials); and

• (c) the organization and facilitation of learning experiences (i.e., the "how" of professional development; the approaches, models, or methods used to support self-directed, experientially-oriented learning that is highly relevant to practice).

Rationale for Coaching Approaches

• High-quality materials and PD workshops not sufficient for use of evidence-informed practices in practice settings

• Difficult to define impacts of coaching in EC PD

• Variants and descriptions of coaching not well distinguished in the literature

• Some empirical support for efficacy of performance feedback in relation to implementation fidelity

• Few studies focused on self-monitoring as a form of “coaching” and in relation to implementation fidelity

Two forms of coaching

On-site Coaching Self-Coaching

Similar components; different deliveryOn-site Coaching

(Teacher and Coach)

Entry and relationship building (First 3 sessions)Learn about the teacher, the classroom and the teachers’ goals, share information about the coaching process, and provide supportive feedback.

Goal settingWork with teacher to identify personal development goals and weekly implementation goals.

ObservationObserve in the classroom, engage in coaching strategies as needed.

FeedbackDiscuss observation, provide supportive and corrective feedback , engage in coaching strategies as needed, support teacher reflection and future planning.

Action plan (problem solving)Identify weekly goals, develop a plan for successful implementation , and provide resources and supports as needed.

Self-Coaching(Teacher and Web-based Supports)

Entry and orientation to the website (1-hour orientation) Learn about the website and self-coaching process, explore website features and how to use the web-based supports to coach.

Goal settingUse the online forms to identify personal development goals and graphing tool plan weekly implementation goals.

Self-monitoringWork toward personal development goals and collect and record data on implementation goals.

Self-evaluationUse self-coaching checklist to guide evaluation and reflection of goals, and use graphing tool to evaluate implementation .

Action plan (problem solving) & Resource seekingSelect weekly goals, develop an action plan for weekly goals, and use available resources on web site to problem solve when situations arise.

On-site Coaching

Evaluating the Efficacy of the Teaching Pyramid Intervention

Who– Preschool teachers in classrooms that include children

with and without disabilities– Teachers in preschool classrooms

What– Teaching Pyramid practices

How– High quality workshops – Provision of classroom materials– Coaching with performance feedback in classroom

Few Childre

n

Some Children

All Childre

n

The Pyramid Model: Promoting Social and Emotional Competence and Addressing

Challenging Behavior

Coaching Teachers in Implementing the Pyramid

Hearing about it in training

Implementing it in daily practice

Teaching Pyramid Coaching Process

• Preparing for Coaching– Conduct baseline observations using the TPOT– Meet with teacher to review TPOT and develop action plan

• Observe teacher with a focus on action plan items– Use specific TPOT items– Use other observational measures– Model, feedback, and provide suggestions as planned

• Debrief with teacher– Provide positive performance feedback– Provide corrective performance feedback– Discuss issues related to individual children– Develop implementation steps– Plan for next observation– Use a variety of strategies (e.g., video, problem solving)

• Follow-up:– Provide email feedback following meetings– Provide materials as needed

• Support teacher around developing plans for children at the top of the Pyramid

Let’s look at it—written debrief

Hi ----, I enjoyed observing in your classroom today. I can¹t believe how well Mary is doing with

her writing. She picked several words from the word wall today and wrote them on the dry erase board. She did a great job! I was able to read all of the letters and she was able to tell me each word. I continue to be amazed at her progress.

You did a great job of embedding many learning trials into small group and centers. Almost all of the trials were complete with an antecedent, behavior, and consequence or error correction. I observed 4 trials for engagement, six for counting/matching numerals, and two social bids. I also observed Ms. Rogers providing two engagement trials. You have done an excellent job of instructing your paraprofessionals in delivering complete learning trials. You make such good use of their expertise in the classroom. Having them as support I am sure makes your job a little easier.

I noticed that today you delivered several trials for the counting/matching numeral goal. This was great and Mary seemed to respond well. I did notice, however, that by the time she got to the center where she was counting chicks she did not seem as interested in counting. I don¹t think there is anything wrong with the activity, but perhaps because it came right after the counting at the sensory table she seemed to lose interest quickly. One suggestion might be to separate those activities so that they are not back to back. This will give her time to complete other activities in between and increase her interest level when she works on counting again.

Just to follow-up on our conversation on Wednesday regarding embedding sequencing trials, you do have many natural opportunities to do this throughout your day. Anytime Mary completes a multi-step activity you could have her tell you how she completed the activity. I have observed you doing this with puzzles, art projects, and in dramatic play.

Other activities that require a sequence of steps occur multiple times throughout the day. One strategy I used to use is having children tell me the centers they visited each day in the correct sequence. This is just one idea. It is just a matter of taking advantage of naturally occurring opportunities and this is something you do very well. You might try planning some activities with this focus.

I will be in around 9:00 on Wednesday. I am planning to stay to meet with you at 1:15. I would like to discuss your progress on data collection and any other issues or concerns that you have.. You are doing a fabulous job. Please feel free to email or call if you have any questions.

I look forward to seeing you next week.

open the e-mail

provide supportive performance

feedback

provide corrective performance

feedback

provide targeted support

end the e-mail

support planned actions

Teacher Goal Planning Form

Teacher: ________________________ Coach: _______________________ Date: _________________ Use the action planning forms on page 5 of your Implementation Guides 1-4 to begin thinking about the Teaching Pyramid in your classroom. Please fill in this form and bring it with you to our next coaching meeting. The strengths I have on each level of the Pyramid are:

Nurturing and responsive relationships (Guide 1):

High quality environments (Guide 2):

Social Emotional Teaching Strategies (Guide 3):

Intensive Supports (Guide 4):

The 3 most difficult parts of the Pyramid for me to use in my classroom are: 1. 2. 3.

The first 3 things I would like to work on are: 1. 2. 3.

Goal Action Steps Materials or Resources Needed

Timeline My goal is met when… Date Action Step Completed

1. Create and post a visual schedule, teach the schedule to the children, and refer to it throughout the day.

1a. List the parts of a typical classroom day. 1b. Decide what format the pictures should be (Boardmaker, photo) and the size of the schedule pieces. 1c. Post the schedule on the magnetic shelf in the Circle Time area 1d. Teach the schedule in large group time by showing the class the schedule, modeling its use, and reminding children to check it throughout the day. 1e. Refer children to the schedule when an unusual event is about to occur (use the “Special Activity” card)

Coach will create schedule pieces for:

Morning Work Calendar Group Bathroom (3) Literacy Group Centers Music &

Movement (2) Math Science Lunch Social Studies Recess Nap Snack Story Dismissal Special Activity

Pieces will be approx 3.5”X3.5” Laminated cardstock; sized to fit magnetic frames Save to MS Word and burn to CD

Coach will drop materials off by Tuesday 11/11. Can begin teaching and using schedule by 11/12 or 11/17.

I have posted the schedule in my classroom. I review the schedule during most routines. I move a magnet to let the children know that a routine is finished. I encourage children to “check the schedule” when they seem confused or off-task. I use the special activity cards to prepare children for an unusual or special event (assembly, field trip, birthday, playing in the gym instead of the playground).

Notes:

The Teaching Pyramid Observation Tool (TPOT)

• The TPOT was developed to measure the extent to which the Teaching Pyramid practices are being implemented in a classroom

• Provides information that can be used to identify training needs of teachers

Using the TPOT

• Observations– Conducted for a minimum of 2 hours

• Must observe centers or free play and at least one teacher-directed activity

– Focus of observation is lead teacher’s behavior

• Interviews– For those practices that cannot be observed in a

2-hour observation

Format of the TPOT

• Three types of Items– Environmental items (Items 1-7) - yes/no based on

observation– Ratings of practices (items 8-22) - ratings based on

observation and/or teacher report– Red Flags (items 23-38) - yes/no based on

observation

Environmental Arrangement Items

- Clear boundaries- Move easily around room- Lack of large open spaces- Adequate number of centers- Materials support play- Preparation of centers- Classroom rules

TPOT Items Based Primarily on Observations

• Schedule and Routines• Transitions between activities• Teachers engage in supportive conversations with children• Promoting children’s engagement• Teaching children behavior expectations• Providing directions• Effective strategies to respond to problem behavior• Teaching social skills and emotional competencies• Teaching children to express emotions• Teaching problem solving• Supporting friendship skills

TPOT Items Scored Based on Teacher Report

• Supporting children with persistent problem behavior

• Communication with families to promote involvement

• Involving families to support social emotional development and addressing problem behavior

• Strategies to build collaborative teaching with other adults

Red Flag Items

• Represents issues related to teacher training, support, or program practices

• Indicates areas for immediate attention and training/support

Sample Item

11. Promoting children’s engagement

0 1 2 3 4 5

A classroom receives a score of “0” if all behaviors under a score of “1” are not observed

__Y __N (1.1) Teacher offers general guidance to children to select activities or use materials to promote engagement*

__Y __N (1.2) Teacher provides fun activities that will support the engagement of almost all of the class *

__Y __N (1.3) Teacher communicates with children on eye level almost all of the time *

__Y __N (3.1) Structured large-group (e.g., circle) activities are structured so that children are actively engaged (responding, interacting) almost all of the time

__Y __N (3.2) Teacher assists individual children in selecting center activities and becoming actively engaged*

__Y __N (3.3) Children are provided with multiple opportunities to make meaningful choices within activities (choosing center, choosing how to use materials, choosing where to sit, etc.)

__Y __N (3.4) Teacher frequently comments positively on children who are engaged in activities

__Y __N (5.1) Teacher assists individual __N/O children who are exhibiting problem behavior within an activity become actively engaged (score N/O if no opportunity to observe) *

__Y __N (5.2) Teacher modifies instruction or activity when children lose interest in large- and small-group activities

Notes:

Sample Item

11. Promoting children’s engagement

0 1 2 3 4 5

A classroom receives a score of “0” if all behaviors under a score of “1” are not observed

__Y __N (1.1) Teacher offers general guidance to children to select activities or use materials to promote engagement*

__Y __N (1.2) Teacher provides fun activities that will support the engagement of almost all of the class *

__Y __N (1.3) Teacher communicates with children on eye level almost all of the time *

__Y __N (3.1) Structured large-group (e.g., circle) activities are structured so that children are actively engaged (responding, interacting) almost all of the time

__Y __N (3.2) Teacher assists individual children in selecting center activities and becoming actively engaged*

__Y __N (3.3) Children are provided with multiple opportunities to make meaningful choices within activities (choosing center, choosing how to use materials, choosing where to sit, etc.)

__Y __N (3.4) Teacher frequently comments positively on children who are engaged in activities

__Y __N (5.1) Teacher assists individual __N/O children who are exhibiting problem behavior within an activity become actively engaged (score N/O if no opportunity to observe) *

__Y __N (5.2) Teacher modifies instruction or activity when children lose interest in large- and small-group activities

Notes:

ENV = 7

Anchor= 2.87

Red Flags = 0

Ellen's 3yr Class Pre TPOT Graph

0

1

2

3

4

5

6

7

8

Env(#1-7)

#8 #9 #10 #11 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21 #22 RedFlags

Series1Series2

Key1-7. Environment 15. Social Skills8. Schedules 16. Express Emotions9. Transitions 17. Problem Solving10. Conversations 18. Friendship Skills11. Promote Engagement 19. Persistent Problem Behavior12. Behavior Expectations 20. Communication w / Families13. Proving Direction 21. Involving Families14. Problem Behavior 22. Relationships w / Adults 23 – 38. Red Flags

0

1

2

3

4

5

6

7

Env(#1-7)

#8 #9 #10 #11 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21 #22 RedFlags

Mallory's Pre-K Pre/Post TPOT Graphs

Series1

Series2

Key1-7. Environment 15. Social Skills8. Schedules 16. Express Emotions9. Transitions 17. Problem Solving10. Conversations 18. Friendship Skills11. Promote Engagement 19. Persistent Problem Behavior12. Behavior Expectations 20. Communication w / Families13. Providing Direction 21. Involving Families14. Problem Behavior 22. Relationships w / Adults 23 – 38. Red Flags

PRE

POST

ENV = 7/ 5

Anchor= .733 / 1.0667

Red Flags = 4 / 6

What have we learned about coaching?

• It takes a significant amount of coaching to get teachers to criterion levels of using the Pyramid practices

• Teachers need help knowing where to start in terms of implementing the Pyramid practices

• Teachers need help accessing and developing materials

• Getting teachers focus on the bottom of the pyramid can be difficult

• We may need a process for rolling out coaching

Self-Coaching

“Coaching” in Tools for Teachers

Impact of Professional Development on Preschool Teachers’ Use of Embedded Instruction Practices

Institute of Education Sciences Project Number R324A070077

Embedded Instruction for Early LearningTools for Teachers (TfT)

Embedded Instruction for Early Learning: Tools for Teachers

Who– Preschool teachers in classrooms that include children with

disabilities– Certified teachers, at least 2 years experience

What– Embedded instruction– Plan, implement, and evaluate embedded instruction practices

with fidelity How

– High quality workshops and support materials– Coaching with performance feedback in classroom– Self-coaching with web-based supports

Two forms of coaching

On-site Coaching Self-Coaching

Research Design• Phase II: Feasibility study

– Multiple baseline across teachers– 3 variations

• Phase III: Potential efficacy trial – Preschool teachers randomly assigned to conditions– Three conditions

• Business-as-usual (no intervention)• Tools for Teachers workshops plus on-site coaching• Tools for Teachers workshops plus self-coaching

www.embeddedinstruction.net

Selecting Personal Goals

Creating an Action Plan

Self-Coaching Checklist

The Graphing Tool

Features of the Graphing Tool

Daily Schedule

Child Roster

Adding New Learning Targets

Trial Plan

Trial Implementation

Teacher Performance Report

Reports for Child’s Learning Target

“Using the self-coaching process you have the tools for improving your practice, however, it is up

to you to take the time to use the tools.”

What are we learning about on-site and self-coaching coaching?

• It may be important for teachers to have some experiences with on-site coaching before self-coaching.

• It takes time for teachers to learn embedded instruction practices and implement them with fidelity.

• The importance of a foundational curriculum in the classroom and the development of functional and generative learning targets.