Effective Implementation as a Way of Work Minnesota’s Journey March 22, 2012 State Capital,...

Effective Implementation as a Way of Work

Minnesota’s Journey

March 22, 2012

State Capital, Bismarck, North Dakota

Cammy Lehr, Ph.D.

Debra Price-Ellingstad, Ed.D.

Minnesota Department of Education

• Introductions• Overview of the Structure within the State

– MDE Organizational Structure– Developing the infrastructure (teams and functions)

• Description of the Journey– How it all started– The birth of CPEP– Where we are now

• Competency Drivers and Application– Triangle (getting on the same page)– State Department Work (across divisions)– MDE Initiatives (extended example - PBIS)– District Work

Agenda

education.state.mn.us 2

• What does the organizational structure of the Minnesota Department of Education look like?

• See handout

MDE Organizational Structure


Compliance and Crises, Urgent, Time Sensitive!!• Assist Districts and Schools not meeting Standards• Deal with urgent and high profile issues

Practice Improvement – “The Willing”• TA targeted at teacher or admin level• Issues that intersect with ‘local values’• Broad professional development initiatives• Pilots and demonstrations

System Stability • Regulatory roles• Basic Data Systems• Financing and Fiscal Accountability• Accreditation and Licensing Standards• Safety Standards• Work with Legislature & Stakeholders

EducationalBest Practices

Initiatives

System StabilityMandates,

System Supports,Foundational Polices & Regulations

SEA Responsibilities and Leverage PointsSEA Responsibilities and Leverage Points

EducationalBest Practices

Initiatives

System StabilityMandates,

System Supports,Foundational Polices & Regulations

SEA Responsibilities and Leverage PointsSEA Responsibilities and Leverage Points

Challenges• Mandates insufficient•Training alone not enough• Pockets of excellence but…• …Pilots come and go• Uneven access• Silos of TA• Lack of integration

• In classrooms• At District level• With broad SEA goals

© Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

Outcomes for Children, Families,

& Communities

What are we working for?Education that “works” for ALL students “from the classroom to the capitol”

Bureaucracy

Teachers & Staff

Policies

Districts and Schools

Building Implementation

Teams

State/DistrictLeadership

Teams

TeachersStudents

Sys

tem

C

han

ge

MD

E R

egio

na

l Im

ple

me

nta

tio

n

Te

am S

ys

tem

Su

pp

ort

Pra

ctic

e-

Po

licy

C

om

mu

nic

ati

on

Cy

cle

Po

licy

Su

pp

orts

Effe

ctive

Pra

ctice

Fostering Communication Between and Across Systems

SISEP 2012

• An important step to establish infrastructure that facilitates communication up, down, and across

• Can we repurpose existing teams?• Who do we need on the teams in terms of role and

function?• Takes time to establish trust• Terms of Reference

– Document outlining the purpose, vision, scope of work, roles, communication pathways

– Functions as internal memorandum of understanding– Links the team to broader systems work

Developing the Infrastructure(teams and functions)


© Fixsen & Blase, 2008

All Students & Families

All Students & Families

School Teachers and

Staff

School Teachers and

Staff

StateDepartment Leadership

StateDepartment Leadership

1 for each SchoolBuilding

Implementation Team

Building Implementation

Team

SISEP’s Vision of Teams and Communication Linkages

Staff with special implementation skills

“District” Implementation

Teams

“District” Implementation

Teams1 for every group of 15-25 Schools

Regional Implementation

Teams

Regional Implementation

Teams

1 for every group of 4 “Districts”

Adult interactions produce Student benefits

SISEP 2012

District Leadership & Implementation Teams(Assistant Superintendent, Title I Intervention Specialist

and others)

MDE 1st GenerationImplementation Team

School Principals, Staff Development, PBIS, RtI , Curriculum and Development

Middle High SchoolElementary Other/EC

State Executive Team

State Im

plem

entatio

n S

pecialists

CPEP State Leadership Team

MDE Teams and

Communication Linkages

• Provide leadership and facilitate policy level changes to support implementation of effective educational practice– Example: Regional Centers of Support

• Support MDE staff learning about and incorporating implementation concepts into MDE work– Attend/participate/provide words of welcome at

Implementation Forums

– Support Book Club

– Support Team Infrastructure

Executive Team

SISEP 2012


• Existing Structure (RtI Leadership Team)• N = about 25• Meets 1X monthly (last Tuesday of the month

9:00-11:00 A..)• E-mail communication and reminders• Key Functions

Lead, analyze progress toward statewide efforts, recommend actions for improvement, problem solve


Internal MDE• Assistant Commissioners • Directors• Supervisors• Specialists

External MDE (n = 6)• Parent, MN Association of School Administrators,

Association of Elementary Principals, Education Minnesota (teacher union), PACER (parent advocacy), MN Administrators of Special Education

A Few Thoughts…

• Communication, communication, communication• Trust building• Active involvement – solicit input• Revisit accomplishments and progress• Not overwhelming or too time consuming• Value contributions – recognition for participation –

show how their input is being used• Wide representation – create a welcoming and safe

environment• Follow up with individuals

• Know innovations very well (formal and craft knowledge)

• Know implementation very well (formal and craft knowledge)

• Know improvement cycles to make interventions and implementation methods more effective and efficient over time

• Functional Worker Bees– Meet twice monthly (formal meeting)

– N = 14

– Represent multiple divisions and content

Regional Implementation Team


SISEP Invests in the Minnesota TeamFormed in the Spring of 2009

What Are We Learning ?

• Common vocabulary is essential• Continuous learning is expected• Learning curve is steep• Membership turnover is inevitable• Both technical and adaptive skills are

important• Peer-to-peer coaching improves our collective

performance– Requires trust– Requires time to reflect

• How it all started…

• The birth of CPEP

• Where we are now!

Description of the Journey


National Center on State Implementation of Scaling-up of Evidence-based Practices

(SISEP)• June, 2008, Minnesota was one of four states

selected to work actively with a new national technical assistance center

• Funded by United States Department of Education• Based at the University of North Carolina• Dean Fixsen, Karen Blase (U of North Carolina),

George Sugai (U of Connecticut), Rob Horner (U of Oregon)

• Goal: Building state capacity for scaling up evidence-based practices

Capacity Development in Minnesota

• Staff from National Center on State Implementation and Scaling Up of Evidence-based Practices provided ongoing technical assistance – Monthly MN on-site visits from SISEP staff – MDE Implementation Team to NC Institute in

March of 2009– Website – Resource materials (briefs, binder, book)– Community of Practice (includes all 4 states)

• SISEP intensive technical assistance work ends September 2012

So…What Do We Scale Up? aka “the birth of CPEP”

• Which programs and practices?

• Decision is based on student need

• A process or framework is needed that can be used across programs and practices to guide scaling up

• Applicable at the

– Classroom

– School

– District

– State Level

Guiding Principles to Inform this Work

• Not a mandate, offered as a way of work• Work collaboratively with the willing• Must apply to ALL students• Must incorporate communication linkages and

feedback loops, up and down as well as across (state, regional, local)

• Multidisciplinary• NOT another program – but a tool to help organize

and integrate• Must incorporate common principles of effective

practice AND key components of implementation

Development of the FrameworkCommon Principles of Effective Practice

• Worked with several initiatives in place at MDE – Positive Behavioral Interventions and Supports,

Dropout Prevention, Response to Intervention, Reading First

– Identified key elements of each “from the balcony” necessary for effectiveness

– Synthesized to identify commonalities – Identified a supporting research base for each

principle of effective practice– Solicited feedback from multiple stakeholders

Minnesota’s Common Principles of Effective Practice (CPEP) and Implementation

Working Together to Build Capacity for Effective Implementation of Effective

Educational Practices

(see handout)

The “Big Eight”Principles of Effective Practice (“What”)

• Defined and Measurable Goals and Outcomes• Evidence Based Practices• Data Driven Decision Making• Multi-Tiered Systems of Support that Accelerate the

Learning of All Students• Implementation of Instruction/Intervention as Intended • Culturally Responsive Teaching and Learning • Alignment with MN P-12 Academic State and Local

Standards • Student, Parent and Community Engagement

Development of the Framework Key Components of Implementation

• National Center on State Implementation and Scaling Up of Best Practices (SISEP)

• National Implementation Research Network (NIRN) – Mission: to advance the science and practice of

implementation, organizational change, and system transformation … to effect positive outcomes for children and youth

Key Components of Implementation (“How”)

• Stages of Implementation (exploration, installation, initial implementation, full implementation, sustainability)

• Staff Competence Drivers (selection, training, coaching)

• Organization Drivers (systems intervention, facilitative administration, data support systems)

• Leadership at ALL Levels (classroom, school, district state; technical and adaptive)

• Performance Assessment (fidelity checks for effective implementation)

Effective Programs

X Effective Implementation

= ________________________________

Outcomes that Benefit Individuals

and Society

What We Want to See Consistently in Minnesota’s Schools to Effect Positive

Student Outcomes

1.0

X

0.0

0.0

How can the CPEP Framework be used?

• As a framework to review educational initiatives and innovations

• As a tool to plan professional development• A lens through which to integrate programs and

initiatives• A reflection tool for assessing best practice• A resource to gather basic information on

implementation and effective practice

What Are We Learning?

• Repetition is important and required• Continuous improvement occurs as more is

understood• Do not impose or mandate, but invite participation• Coaching is important • Do not expect perfection immediately as folks are

exploring concepts• Build capacity at all levels • Model and provide opportunities for ongoing

professional development – just in time learning

A word on Implementation Stages

Exploration

• Assess needs

• Examine innovations

• Examine Implementation

• Assess fit

2 - 4 Years – School Level

Installation

Acquire resources

Prepare organization

Prepare implementation

Prepare staff

Initial Implementation

Implementation drivers

Manage change

Data systems

Improvement cycles

Full Implementation


Implementation outcomes

Innovation outcomes

Standard practice

– Triangle (getting on the same page)

– State Department Work (across divisions)

– MDE Initiatives (Example, PBIS)

– District Work

Competency Drivers and Application


Implementation Drivers…

Build Competency and ConfidenceDevelop, improve, and sustain competent & confident use of innovations.

Change Organizations and SystemsCreate and sustain hospitable organizational and systems environments for effective instructional and educational services

© Fixsen & Blase, 2008

Performance Assessment

(Fidelity Checks for Effective Implementation)

Staff Coaching

Staff Training

Staff Selection

Systems Intervention(alignment of policies and practices)

Facilitative Administration

Data Support SystemC

ompe

tenc

y D

river

s

Com

pete

ncy

Driv

ers

Organization D

rivers

Organization D

rivers

LeadershipLeadership

Adaptive Technical

CPEP Implementation Components (the “how”) (a.k.a. Implementation Drivers)

State Department Work Across MDE DivisionsBuilding Impacting District and Schools

General Capacity Building– Book Club process– Implementation Forums– Regional Implementation Team Learning

(coaching, rehearsal)– Paired learning (across divisions)– Technical Assistance from SISEP staff– State Capacity Assessment– Website with 2 pagers

• Read Well by 3rd Grade – Implementation Appraisals for each core component– Implementation Rubric

• State Wide System of Support – new Regional Centers of Excellence to be implementation informed and trained

– Implementation Specialist – Utilization of teams

• NCLB Waiver - Principal and Teacher Evaluation Models– Stage-based approach– Incorporate Plan Do Study Act feedback– SEA role in providing ongoing training

• Q-Comp– Networking sessions incorporated principles of effective practice and

information on implementation components– Tools developed to reflect on best practice with implementation drivers



• Early Childhood Centers of Excellence – TACSEI – Regional Training Centers– Communication Feedback Loops

• Statewide roll-out of new manuals (LD manual)– Training implications

• RFP Processes– Alternative Delivery of Specialized Instructional Services

(ADSIS) requires all interventions to have common principles of effective practice

– ADSIS Leadership and Implementation Teams – facilitates communication and coordination with continuum of supports available to students




• Implementation is universal

– Like gravity, implementation influences are always at work (not about intention)

– Implementation principles are not dependent on disciplines, educational content area, developmental ages, region, school level, etc.

Example: Minnesota SW-PBIS Initiative Through an Implementation Lens

Implementation Drivers•Competency & Organization•Integrated & Compensatory

• Application for training moves schools through the “Exploration” stage. Requirements include:– a school leadership team (represents the roles,

functions, and diversity of the school)

– administrator participation

– 80% staff buy-in

– School-wide Information System (SWIS or other)

– Requires an internal coach

– Trained School Evaluation Tool (SET) evaluator

• Trainer Selection (and training)

• Coach Selection

SW-PBIS Selection


• Two-year team training cohort– Year 1 = 2 days 3 x per year– Year 2 = 1 day 3 x per year

• Trainers reflect the diversity of team membership

• Standardized scope and sequence (revised based on data)

• Moves schools from installation to initial/full implementation

• Summer Institute for sustainability

• Online training modules (under development based on data)

SW-PBIS Training


Cohort 5 School SET ResultsFall 09 – Spring 11

42

• Coaching at multiple levels– Regional Coaches

– District Coaches

– Building/Program Coaches

– All inform training scope and sequence (feedback loop)

• Coaches Workgroup– Wiki site

– Coaches calendar

– Coaches scope and sequence (under development)

– Networking opportunities

SW-PBIS Coaching


44

CoachingSW-PBIS Coaching

• MDE Evaluation Contractor – Wilder Research– Training evaluations

– State-wide and Regional Evaluation Reports

– SET Evaluation Exchange

• School and District-level Assessment Data– Office Discipline Referrals (School-Wide Information

System or other)

– Team Implementation Checklist (TIC)

– Self-Assessment Survey (SAS)

– School Evaluation Tool (SET)

– Benchmarks of Quality (under consideration)


SW-PBIS Performance Assessment

Data Collection Cycle


Linked Implementation

TeamsImprovement

Cycles

Regional Implementation Project Teams

StateTeam

District Teams

SchoolTeams

SW-PBIS Systems Intervention

Informed by Data

• Aspects covered in application– Administrator participation in team training

– Release time for team member training

– Permission to revise office discipline referral form

– Data support system (SWIS or other)

• Staff person designated as coach

• Staff person designated as SET evaluator

• Secure resources for implementation– Team meeting time

– Rewards

SW-PBIS Facilitative Administration


• Wilder Research – PBIS Evaluation Contractor

• PBIS Assessment– TIC

– SAS

– SWIS

– SET

• SET Exchange Program

• Data-based Recognition System

SW-PBIS Data Support Systems


Terms of Reference address both technical and adaptive challenges through shared understanding of:

– Vision/Goals and Objectives

– Scope and Boundaries (what the work is and is not, when are we done)

– Roles and Responsibilities of individuals (who participates in what ways)

– Linking Communication Protocols (with whom do we communicate, how, and how often and for what purpose)

– Resources available to the project

– Authority

– Deliverables

– Implementation Plans

SW-PBIS Leadership Teams


What Are We Learning?SW-PBIS Implementation and Scale-up

• State Implementation Team membership will expand with scale-up

• Five-year + implementation blue print is essential

• Practice data-based decision making

• Focus on infrastructure and capacity-building

• Anticipate implementation “bumps”

• Prepare for the technical and adaptive challenges of role release

District Selection

District Entry

•Forest Lake – approximately 15 schools; 7000 students (learning platform – Multi Tiered Systems of Support/Literacy) Suburban/Rural•Mahtomedi – approximately 4 schools; 3000 students (learning platform – RtI) Suburban•St. Paul - approximately 69 schools; 39,000 students (learning platform – PBIS) Urban

Work with Districts


How Do Regional Implementation Team Members Work With District Teams?

• Convene, collaborate, partner with districts to build on strengths

• Provide ongoing technical assistance (training and coaching) on content and application related to the Science of Implementation

• Help to analyze the infrastructure needed for sustained implementation or effective practices

• Help embed sustained use of principles of effective practice (e.g., use of data systems for decision making, fidelity of implementation, evidence based practice)

• Listen and communicate needs and feedback across various teams

• Avoid jargon, provide just in time assistance, need to screen for leadership that is open to being coachable

• Spending time on exploration will save time down the road

• Hold accountable to expectations and sequence for process, watch out for turnover in leadership before a foothold is established

• Creating a functional leadership and implementation team – the perfect storm to get the work done

Stories Related to Work with the Districts


Remember what we said aboutimplementation stages?

Exploration

• Assess needs

• Examine innovations

• Examine Implementation

• Assess fit

2 - 4 Years – School Level

Installation

Acquire resources

Prepare organization

Prepare implementation

Prepare staff

Initial Implementation


Manage change

Data systems

Improvement cycles

Full Implementation


Implementation outcomes

Innovation outcomes

Standard practice

So… How Are We Doing?

Implementation Book Club Feedback

• Implementing the Findings of Research: Bridging the Gap Between Knowledge and Practice ( Wallace, Blase, Fixsen, & Naoom, 2008)

• Six one-hour monthly sessions• Sixty-four total participants• Across 8 MDE Divisions• Core group of 21 (attended 4 or more sessions)• Average attended 3 sessions• 90% reported content was excellent/good• 87% would recommend the book club series to colleagues • 90% could apply the CPEP principles to their work at MDE

SMT Investment SMT Alignment RIT Functioning DLIT Functioning SISEP 2012


Implementation can be done on purpose•Letting it happen

– Recipients (WHO) are accountable•Helping it happen

– Recipients (WHO) are accountable•Making it happen

– Implementation Teams (WHO) are accountable (programs go where they are most needed)

Based on Hall & Hord (1987); Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou (2004); Nord & Tucker (1987)


Active intentional management of implementation must be done on purpose to increase likelihood of reaching intended results.

"Successful programs do not contain the seeds of their own replication“ (Schorr, 1993)


Sustainable benefits require organization and system change and this is hard work.

•Innovative practices do not fare well in existing organizational structures and systems (legacy systems)•Organizational and system changes are essential to successful use of innovations

(Institute of Medicine, 2000; 2001; 2009; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999; Ulrich 2002)


http://education.state.mn.us/MDE/EdExc/BestPrac/ImpleEffecPrac/index.html

http://sisep.fpg.unc.edu


Effective Implementation as a Way of Work Minnesota’s Journey March 22, 2012 State Capital,...

Documents

Transcript of Effective Implementation as a Way of Work Minnesota’s Journey March 22, 2012 State Capital,...