Michelle A. Duda, Ph.D., BCBA,Dean L. Fixsen, Ph.D.

& Karen A. Blase Ph.D.,

Melissa Van Dyke, LCSWFrank Porter Graham Child Development Institute

University of North Carolina – Chapel Hill

Working Smarter: The implementation journey to

improve student outcomes

Innovation FluencyPart 1

A Problem

• Too often, the choice of an intervention to implement has had little to do with the quality or weight of the evidence regarding practices

• Decisions are made for a variety of reasons not necessarily related to the data about effectiveness (e.g. philosophy, values, comfort, acceptance, availability, finances)

Follow Through Programs

Figure 1: This figure shows the average effects of nine Follow Through models on measures of basic skills (word knowledge, spelling, language, and math computation), cognitive-conceptual skills (reading comprehension, math concepts, and math problem solving) and self-concept. This figure is

adapted from Engelmann, S. and Carnine, D. (1982), Theory of Instruction: Principles and applications. New York: Irvington Press.

Innovation Fluency

• Definition: Innovation Fluency refers to the degree to which the team knows the innovation with respect to:

1.Evidence2.Program and Practice Features3.Intervention Readiness for Replication4.Implementation Capacity Requirements

• Because all of the above impact the success of implementation efforts

Intervention Science

Research has identified programs and practices that can help students, communities, and education systems

• What Works Clearinghouse (http://ies.ed.gov/ncee/wwc/)

• NREPP (http://www.nrepp.samhsa.gov/)

• Colorado Blueprints (http://www.colorado.edu/cspv/index.html)

• Education research journals/ American Education Research Association (http://www.aera.net/)/ J. of Evidence-Based Practices in Schools

1. Research View of Evidence

• Descriptions of practices • Observational & survey information • Correlational studies• Quasi-experimental studies• Randomized clinical trials (programs)

o in situations of convenienceo in clinical context

• Single-subject experiments (practices, components, programs, systems)

A Different View of Evidence

• From an implementation perspective, what do we need to know about evidence?

1. Evidence Related to the Intervention

o Theoretical Foundations – Intervention What principles make “it” work?

o Empirical Support – Intervention How much research has been done? How much of a difference could it make (e.g.

effect size)?o Fidelity Data (e.g. predictive and discriminative)o Practical to Collect and Analyzeo Cost-effectiveness Datao Populations (e.g. culture, race, language,

ethnicity)o Expected Student Outcomes – Intervention

What outcomes have been achieved in the “real world”?

2. Program and Practice Features

o Start where it matters: At the interface between Teachers and Students

o Build on those Program and Practice features that interface as the foundation, then….

o Identify what needs to happen at the classroom level so the teacher can interact with students as intended

o Identify what needs to happen at the school level to support what happens at the classroom level

o Identify what needs to happen at the Division level to support school and classroom implementation

o ….and the State?

A Different View of Program and Practice Features

• From an implementation perspective, what do we need to know about P & P Features?

2. Program and Practice Features

1.What you can observe happening at classroom, building, and within the division level ?

2. Fit with current priorities and issues

3. Resource Requirements

Program and Practice Features through an Implementation Lens

o Fit with: Current District and School Initiatives & Priorities AYP Priorities RtI Processes Current Organizational Structures at School and District Level Parent and Community Values – Social Validity Current skill set of staff expected to implement the innovation

o Resource Requirements Materials Staffing (recruitment, reallocation) Training costs Coaching and Supervision IT and Data Requirements Administrative time and supports

Do you Recognize The Implementation

Drivers?

Innovation Fluency

• The “it” must be operationalized whether it is:

An Evidence-Based Practice or Program A Best Practice Initiative A Systems Change Initiative

• Operationalize Part of Speech:  verb Definition:  to define a

concept or variable so that it can be measured or expressed quantitatively

Webster's New Millennium™ Dictionary of English, Preview Edition (v 0.9.7)Copyright © 2003-2008 Lexico Publishing Group, LLC

Practice Profile

• Increased Innovation Fluency Through the Development and Use of Practice Profiles

• Identifies critical components• For each critical component:

o Identifies gold standardo Identifies acceptable variations in practiceo Identifies ineffective practices and

undesirable practices

o Heartland Area Education Agency 11, Iowa

Practice Profiles

• Each critical component is a heading• Each level of implementation becomes a

dimension on the rubric associated with that critical component.

Critical Component Ideal Implementation

Acceptable Variation

Unacceptable Variation

Unacceptable Variation

Critical Component 1: Description

Description of implementer

behavior

DrasticMutation

Adapted from work by Heartland Area Education Agency 11, Iowa

ACTIVITY ONE

Critical Component

Ideal Implementation

Acceptable Variation

Unacceptable Variation

Unacceptable Variation

(Drastic Mutation)

ESD Team Membership

Team Has broad representation

Membership includes invested individuals with knowledge about behavior and ESD implementation.

Team Incl. Behavior Specialist, Principal, 3 lead teachers, parent, school psych

Membership includes invested individuals with knowledge about behavior and ESD implementation.Team Incl.Behavior Specialist, Principal, 2 lead teachers and a parent

Membership includes individuals with minimal knowledge about behavior and ESD implementation.

Team incl.Admin Assistant and 2 teachers

Membership includes individuals with common planning periods.

Team includes 3 teachers, but team members change each semester

Part ONE: Practice using the practice profile format.

EXAMPLE: School Level

ACTIVITY ONE

Critical Component

Ideal Implementation

Acceptable Variation

Unacceptable Variation

Unacceptable Variation

(Drastic Mutation)

Link school teams to supporting resources

Part ONE: Practice using the practice profile format.

EXAMPLE: Coaching/District Level

ACTIVITY ONE

Part Two: Reflection Questions

• How does a schools' implementation of a core feature support the division goals/outcomes?  

• How does the division support the school in implementing these core features?  

3. Intervention Readiness for Replication

o What are the Qualified “purveyor” and researcher connections?

o Are the Content ‘experts’ and qualified TA available?o Have Replication Success (e.g. % of attempted replications still

implementing) been demonstrated? o Are the Core Intervention Components (program features)

well operationalized so that Practice Profiles can be developed if necessary?

o Have the Implementation Drivers been: Well operationalized (Selection, Training, Coaching, Data

processes, etc)? Demonstrated to be effective (e.g. change & sustain new

teacher behavior)? Practical and affordable (e.g. Drivers have been utilized in

typical settings)?

4. Implementation Capacity Requirements

o Sufficient Staff who meet minimum requirements

o Region/Division prepared to define practice/program features and elements if necessary (e.g. Practice Profiles)

o Region/Division able to sustain Implementation Drivers Fiscally Structurally Functionally

o Buy-in Process Operationalized School District Parents

Innovation Fluency

• Implementation Teams need to understand the innovation so that they can:

o Help Districts/Divisions “choose” wisely Address a prevalent and significant need Worth it to scale-up (e.g. effect size of .50 or greater)

o Evaluate the innovation’s readiness for implementation

o Assess the implementation capacity of the District in relation to a number of factors related to the innovation

For More Information

Dean L. Fixsen, Ph.D.• 919-966-3892• fixsen@mail.fpg.unc.edu

Michelle A. Duda, Ph.D.• 919-636-0843• duda@mail.fpg.unc.edu

At the Frank Porter Graham Child Development InstituteUniversity of North Carolina

Chapel Hill, NC

www.scalingup.org http://nirn.fpg.unc.edu/

http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/ 

For More Information

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).Download all or part of the monograph at:http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/ To order the monograph go to:https://fmhi.pro-copy.com/