Implementation: The Missing Link between Research and Practice
-
Upload
ezekiel-whitley -
Category
Documents
-
view
17 -
download
0
description
Transcript of Implementation: The Missing Link between Research and Practice
Dean L. Fixsen, Karen A. Blase,
Sandra F. Naoom, Melissa Van Dyke,
Frances WallaceNational Implementation Research Network
Louis de la Parte Florida Mental Health Institute
Implementation: The Missing Link between Research and Practice
NECTAC Conf 2007
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Early Education
20 million preschool kids
3,143 counties
60 states & U.S. jurisdictions
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Outcomes for Children, Families,
& Communities
A Functional System
Bureaucracy
Teachers
Policies
Schools
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Evidence-Based Movement
The “evidence-based movement” is an international experiment to make better use of research findings in typical service settings.
The purpose is to produce greater benefits to students and society.
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Science to Service
SCIENCE SERVICEGAPIMPLEMENTATION
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Science to ServiceScience to Service Gap
What is known is not what is adopted to help students, families, and communities
Implementation Gap What is adopted is not used with fidelity
and good outcomes for consumers.
What is used with fidelity is not sustained for a useful period of time.
What is sustained is not used on a scale sufficient to impact social problems.
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Making use of Science
Letting it happen
Recipients are accountable
Helping it happen
Recipients are accountable
Making it happen
Implementation teams are accountable
Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
0
100
200
300
400
500
600
700
800
900
≤1972 1974 1976 1978 1980 1982
0
50
100
150
200
250
300
CumulativeCouples
CumulativeHomes
Fixsen, Blase, Timbers, & Wolf (2001)
Teaching–Family Model
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Follow Through Programs
Figure 1: This figure shows the average effects of nine Follow Through models on measures of basic skills (word knowledge, spelling, language, and math computation), cognitive-conceptual skills (reading comprehension, math concepts, and math problem solving) and self-concept. This figure is
adapted from Engelmann, S. and Carnine, D. (1982), Theory of Instruction: Principles and applications. New York: Irvington Press.
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
School Wide PBS
SWPBS Schools
0
1000
2000
3000
4000
5000
6000
7000
Aug 04 Aug 05 Aug 06 Aug 07
Year
Tota
l Num
ber
Sch
ools
4.2% of all schools
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Reviews
Human service prevention and treatment programs (e.g. education, substance abuse, adult / children’s MH, justice, health)
Advanced manufacturing technologies
AMA clinical guidelines
Engineering: bridge maintenance
Hotel service management
National franchise operations
Cancer prevention & treatment
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Ineffective MethodsExcellent experimental evidence for
what does not work
Diffusion/dissemination of information by itself does not lead to successful implementation (research literature, mailings, promulgation of practice guidelines)
Training alone, no matter how well done, does not lead to successful implementation
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Ineffective MethodsExcellent evidence for what does
not work
Implementation by edict by itself does not work
Implementation by “following the money” by itself does not work
Implementation without changing supporting roles and functions does not work Paul Nutt (2002). Why Decisions Fail
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
What Works
Effective intervention practices
+
Effective implementation practices
=
Good outcomes for consumers
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implement Innovations
Effective NOT Effective
Effective
NOT Effective
IMPLEMENTATION
INT
ER
VE
NT
ION Student Benefits
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation
An effective intervention is one thing
Implementation of an effective intervention is a very
different thing
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
EBPs & Implementation
From an implementation perspective, what do we need to know about innovations such as evidence-based programs?
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
EBPs & Implementation
The usability of a program has little to do with the quality or weight of the evidence regarding that program
Evidence on intervention effectiveness for specific populations helps us choose what to implement
Evidence on the effectiveness of the intervention does not help implement the program or practice successfully
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
EBPs & Implementation
Core intervention components
Clearly described (who/what)
Practical measure of fidelity
Fully operationalized (do/say)
Field tested (recursive revision)
Contextualized (org./systems fit)
Effective (worth the effort)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation
What do we need to know about successful implementation methods?
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Stages of Implementation
Implementation is not an event
A mission-oriented process involving multiple decisions, actions, and corrections
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Stages of Implementation
Exploration
Installation
Initial Implementation
Full Implementation
Innovation
Sustainability
Implementation occurs in stages:
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
2 – 4 Years
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Stages of Implementation
Exploration
Installation
Initial Implementation
Full Implementation
Innovation
Sustainability
Implementation occurs in stages:
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
2 – 4 Years
Intervention Outcomes
0% 100%
Implementation Outcomes
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
INTEGRATED & COMPENSATORY
CONSULTATION & COACHING
CONSULTATION & COACHING
STAFF PERFORMANCE EVALUATION
STAFF PERFORMANCE EVALUATION
FACILITATIVE ADMINISTRATIVE
SUPPORTS
FACILITATIVE ADMINISTRATIVE
SUPPORTS
RECRUITMENT AND SELECTION
RECRUITMENT AND SELECTION
PRESERVICE TRAINING
PRESERVICE TRAINING
SYSTEMSINTERVENTIONS
SYSTEMSINTERVENTIONS
Implementation Drivers
DECISION SUPPORT DATA SYSTEMS
DECISION SUPPORT DATA SYSTEMS
OUTCOMES(% of Participants who Demonstrate Knowledge, Demonstrate
new Skills in a Training Setting, and Use new Skills in the Classroom)
TRAININGCOMPONENTS
KnowledgeSkill
DemonstrationUse in the Classroom
Theory and Discussion
10%
5% 0%
..+Demonstration in Training
30%20%
0%
…+ Practice & Feedback in Training
60% 60% 5%
…+ Coaching in Classroom
95% 95% 95%
Joyce and Showers, 2002
Integrated & Compensatory
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implement Innovations
Effective NOT Effective
Effective
NOT Effective
IMPLEMENTATION
INT
ER
VE
NT
ION Perf. Assess.
Fidelity
Implementation without fidelity does not work
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Functional Fidelity Assessment
0%
10%
20%
30%
40%
50%
Homes
1.0
1.2
1.5
1.7
1.9
2.0
1.1
1.4
1.6
1.8
1.3
Delinquency
Teaching
rs = – .94
Pe
rce
nt
Pa
ren
tal-
Tea
ch
ing
Wit
h Y
ou
ths
Me
an
Se
lf Rep
orte
dD
elin
qu
en
cy
Ratin
gs
1 2 3 4 5 6
Bedlington, et al., 1988
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Functional Fidelity Assessment
0%
10%
20%
30%
40%
50%
Homes
5.0
5.4
6.0
6.4
6.8
7.0
5.2
5.8
6.2
6.6
5.6
Youth Ratings
Teaching
rs = .94
Pe
rce
nt
Pa
ren
tal-
Tea
ch
ing
Wit
h Y
ou
ths
Me
an
Yo
uth
Sa
tisfac
tion
Ra
ting
s
1 2 3 4 5 6 7 8 9 10
Bedlington, et al., 1988
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation
Who does the work?
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Team
A group that knows the innovation very well (formal and practice knowledge)
A group that knows how to implement that innovation with fidelity and good effect
A group that accumulates data & experiential knowledge -- more effective and efficient over time (information economics, K. Arrow)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Team
Policy members (change policy, barrier busters, facilitators)
Practice members (do the innovation, test policies, feedback)
Families and stakeholders
Management members (roles and functions)
Daily / Weekly / Monthly Meetings (urgent, unfiltered, goal focused)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Team
School
Management (leadership, policy)
Administration (HR, structure)
Supervision (nature, content)
Teacher
State and Community Context
District
Imp
lem
enta
tio
n T
eam
Simultaneous, Multi-Level Interventions
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation TeamChange the behavior of adult education professionals
“Systems don’t change, people do” (J.W.)
Change organizational structures, cultures, and climates
Change the thinking of system directors and policy makers
Successful and sustainable implementation of evidence-based programs always requires organization and systems change.
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Costs and Savings
Implementation Costs & Savings(Inflation Adjusted)
60
70
80
90
100
110
120
1 Yr Pre During PostYear 1
PostYear 2
PostYear 3
Ch
ang
e in
Bu
dg
et (
Per
cen
t)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Budget Changes (4 years)
Pre Post
Administration 39% 19%
Treatment Units 59% 75%
Training & Eval. 2% 6%
Reduction in administrative staff, reduction in maintenance staff and repairs, reduction in food/staff costs, nearly 100% staff redeployment/rehiring into newly defined positions
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
“Making it happen”
Implementation: Active involvement of implementation teams that work at the intersection of practices, programs, systems, communities, & scientists
Implementation teams are accountable for assuring use of innovations with fidelity and good outcomes
Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Team
Implementation Team
Prepare Communities
Prepare schools faculty, staff
Work with Researchers
Assure Implementation
Prepare Districts Assure Student Benefits
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Systems Change
“Systems trump programs.”Patrick McCarthy, Annie E. Casey Foundation
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Systems Change
Teachers / staff impact students
It is the job of administrators, managers, and funders to align policies and structures to facilitate effective teacher / staff practices
There is no such thing as an “administrative decision” – they are all education decisions
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Systems Change
New practices do not fare well in old organizational structures and systems
Develop new position descriptions and job functions in state departments and districts focused on implementation (effective use) of policies and innovations
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
A Sobering Observation
"All organizations [and systems] are designed, intentionally or unwittingly, to achieve precisely the results they get."
R. Spencer DarlingBusiness Expert
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Systems Change
State Department
Districts
Schools
Teachers/ Staff
Effective Practices
AL
IGN
ME
NT
Federal Departments
Imp
lem
enta
tio
n T
eam
s
FORM FOLLOWS FUNCTION
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Creating Capacity for Competent Change
New OSEP Center
State Implementation and Scaling up of Evidence-based Practices (SISEP)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Creating Capacity for Competent Change
State Transformation Team that learns complex skills related to creating and sustaining RITs
Regional Implementation Teams that learn complex skills related to system, school, and teacher change
Capacity = knowledge, skills, and experience; self-correcting, self-sustaining knowledge utilization
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
State Transformation
Team
RegionalImplementation
Team
RegionalImplementation
Team
RegionalImplementation
Team
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Outcomes for Children, Families,
& Communities
Functional Education
Bureaucracy
Teachers
Policies
Schools
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Thank YouWe thank the following for their support
Annie E. Casey Foundation (EBPs and cultural competence)
William T. Grant Foundation (implementation literature review)
Substance Abuse and Mental Health Services Administration (implementation strategies grants; NREPP reviews; SOC analyses of implementation; national implementation awards)
Centers for Disease Control & Prevention (implementation research contract)
National Institute of Mental Health (research and training grants)
Juvenile Justice and Delinquency Prevention (program development and evaluation grants
Office of Special Education Programs (Capacity Development Center contract)
Agency for Children and Families (Child Welfare Leadership Development contract)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
For More Information
Dean L. Fixsen
813-974-4446
Karen A. Blase
813-974-4463
National Implementation Research Network
At the Louis de la Parte Florida Mental Health Institute
University of South Florida
http://nirn.fmhi.usf.edu
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
For More Information
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Download all or part of the monograph at:http://nirn.fmhi.usf.edu/resources/publications/Monograph/index.cfm
Implementation Research: A Synthesis of the Literature