Results Driven Accountability and Intensive Intervention: Using MTSS to Improve Outcomes for Students with Disabilities

Sarah Arden, Laura Berry Kuchle, Christopher Lemons, T. Chris Riley-Tillman

This document was produced under U.S. Department of Education, Office of Special Education Programs, Award No. H326Q110005. Celia Rosenquist serves as the project officer. The views expressed herein do not necessarily represent the positions or policies of the U.S. Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service or enterprise mentioned in this document is intended or should be inferred.

A note about questions…Please type

questions related to technical issues in

the Chat box.

Please type questions related to webinar content in

the Q&A box.

PanelistsSarah V. Arden, Ph.D. – Researcher and Technical Assistance Liaison at American Institutes for Research

Laura Berry Kuchle, Ph.D. – Researcher and Technical Assistance Liaison at American Institutes for Research

Chris Lemons, Ph.D. – Senior Advisor to the National Center on Intensive Intervention and Assistant Professor in the Department of Special Education at Peabody College of Vanderbilt University

T. Chris Riley-Tillman, Ph.D. – Trainer for the National Center on Intensive Intervention and Professor at the University of Missouri College of Education

Session Overview What is the vision of RDA How is intensive intervention part of RDA Overview of National Center on Systemic Improvement Introduction to DBI Why Do We Need to Assess DBI Implementation? DBI Implementation Rubric Lessons Learned: Strategies for Improving Technical

Assistance and DBI Implementation NCII Resources to Support Implementation

5

Results-Driven Accountability: VisionAll components of an accountability system will be

aligned in a manner that best support states in improving results for infants, toddlers, children, and

youth with disabilities and their families.

Shift from Compliance to Results + ComplianceSlide adapted from: OSEP Slides to Explain Results Driven Accountability (RDA) Retrieved from http://www2.ed.gov/about/offices/list/osers/osep/rda/index.html

6

SSIP

• Conduct root cause analysis (including infrastructure) to identify contributing factors

• For each contributing factor, identify both barriers and leverage points for improvement

• Search/evaluate evidence-based solutions (Exploration Phase)

• Develop action steps (address barriers/use leverage points)

• Develop Theory of Action• Develop Plan for Improvement

(Implementation Framework)

• Initiate Data Analysis• Conduct broad

Infrastructure Analysis• Identify problem area

• Evaluation of progress annually• Adjust plan as needed

How well is the solution

working?What is the problem?

Why is it happening?

What shall we do

about it?

SSIP

SSIP Phase I

SSIP Phase I and II

SSIP Phase III

SSIP Phase I

Slide from: OSEP Slides to Explain Results Driven Accountability (RDA) Retrieved from http://www2.ed.gov/about/offices/list/osers/osep/rda/index.html

7

State Systemic Improvement PlanYear 1—FFY 2013Delivered by April 2015

Year 2—FFY 2014Delivered by Feb. 2016

Years 3–6—FFY 2015–18Feb. 2017–Feb. 2020

Phase I Analysis Phase II Plan Phase III Evaluation Data analysis Infrastructure

analysis State-identified

measureable result Coherent

improvement strategies

Theory of action

Multiyear plan addressing:

• Infrastructure development

• Support early intervening services program and local education agencies in implementing evidence-based practices

• Evaluation plan

Reporting on progress including:

• Results of ongoing evaluation

• Extent of progress Revisions to the State

Performance Plan

Slide from: OSEP Slides to Explain Results Driven Accountability (RDA) Retrieved from http://www2.ed.gov/about/offices/list/osers/osep/rda/index.html

8

State-identified Measurable Result(s) State-identified Measurable Result(s) (SiMR)

• A child-level (or family-level, for Part C) outcome

• Not a process or system result

• May be a single result or a cluster of related results

Identified based on analysis of data

9

On What Are States Focusing?• Graduation: AK, DC, FL, GA, MN, MT, NC, ND, NJ, PA, RMI, VA, WV

• Reading/ELA: AR, AS, AZ, CNMI, CO, CT, DE, FSM, GU, HI, IA, ID, IL, IN, KS, LA, MI, MS, NE, NV, NM, NY, OH, OK, OR, Palau, SC, SD, TN, TX, VI, WA, WI, WY

• Math: KY, MD, ME, PR, RI, UT, VT

• Reading and Math: CA, MO

• Early Childhood Outcomes: MA, NH

• Post-school Outcomes: AL, BIE

Variations: Disability category; race/ethnicity; gender; grades; English learner; poverty status; subset of districts

10

Collaboration between Centers Led by WestEd Partners:

• AIR

• NASDSE

• SRI

• CCSSO

• Parent Center Network

• Meadows Center for Preventing Educational Risk (evaluator)

Learn more at http://ncsi.wested.org/

Ask NCSI: http://ncsi.wested.org/contact/

NCSI Resources: http://ncsi.wested.org/resources/

11

Targeted Outcomes of NCSI1. Increase the capacity of state education agencies (SEAs) and lead

agencies (LAs) to develop, implement, and evaluate their State Systemic Improvement Plans (SSIPs)

2. Increase SEAs’ and LAs’ knowledge, selection, and utilization of evidence-based practices (EBPs)

3. Improve SEA and LA infrastructure and coordination within SEAs and LAs for delivering effective technical assistance (TA) on implementing and scaling-up effective strategies, stakeholder engagement, resource mapping and allocation, and instructional collaboration

12

Targeted Outcomes of NCSI Cont.4. Increase the use of effective dissemination strategies by SEAs

and LAs to ensure that local education agencies (LEAs) and early intervention service (EIS) programs and providers have access to EBPs and can select and implement those EBPs in a sustainable manner

5. Increase the effectiveness of SEAs and LAs to meaningfully engage State and local stakeholders in the development and implementation of the SSIP

13

NCSI Approach to TA

14

Why Is This Important? Meeting SiMR goals will require a focus on improving

instruction. States will be in need of support on how to provide

intensive intervention for the kids who need it the most, including: • Evidence-based intervention strategies

• Overcoming implementation barriers

• Making connections to other state, district, and school initiatives

15

What Does This Mean for the Field? Data Use

• Knowing what data to look at…accessing those data…root cause analysis to make sense of the data

Knowledge Utilization

• Selecting coherent/aligned evidence-based practices…implementing them with fidelity…scaling up and sustaining them

Systems Change

• Infrastructure considerations…resource mapping and alignment…policies that support the goals

Communication & Collaboration

• Problem solving and planning with diverse stakeholders, in the right settings, at the right time…and working together for implementation to achieve results

16

What is Intensive Intervention and Why Do We Need to Assess Fidelity of Implementation?

17

What is Intensive Intervention?Intensive intervention addresses severe and persistent learning or behavior difficulties. Intensive intervention should be Driven by data Characterized by increased intensity (e.g., smaller group,

expanded time) and individualization of academic instruction and/or behavioral supports

18

What Intensive Intervention… Is… Individualized based on

student needs More intense, often with

substantively different content AND pedagogy

Comprised of more frequent and precise progress monitoring

Is Not… A single approach A manual A preset program More of the same Tier 1

instruction More of the same Tier 2

instruction

19

Why Do We Need Intensive Intervention?

Low academic achievement

Dropout rates

Arrest rates

20

What is NCII’s Approach toIntensive Intervention?Data-Based Individualization (DBI): A systematic method for using data to determine when and how to provide more intensive intervention: Origins in data-based program modification/experimental teaching

were first developed at the University of Minnesota (Deno & Mirkin, 1977).

It is a process, not a single intervention program or strategy. It is not a one-time fix, but an ongoing process comprising intervention

and assessment adjusted over time.

21

DBI: Integrating data-based decision-

making across academics and social behavior

22

Is DBI the same as MTSS?Is DBI special education? Many components of DBI are consistent with elements of

special education and tiered service delivery systems

Tiered Interventions (RTI, MTSS, PBIS)

• Universal, secondary, and tertiary interventions

• Progress monitoring• Team-based decisions based

on data

Special Education

• Individualized program• Progress monitoring• Team-based decisions

based on data

23

Who Needs DBI? Students with disabilities who are not making adequate

progress in their current instructional program

Students who present with very low academic achievement and/or high-intensity or high-frequency behavior problems (typically those with disabilities)

Students in a tiered intervention system who have not responded to secondary intervention programs delivered with fidelity

24

What Is Fidelity? Extent to which an intervention’s core components have

been implemented as planned (Nelson et al., 2012) For DBI, this includes the instructional platform, adapted

iterations of intervention, ongoing progress monitoring, and decision-making procedures.

See IRIS module for additional information: http://iris.peabody.vanderbilt.edu/module/fid/

25

Levels of Fidelity Student level

• For a given student, are assessment and intervention being carried out as planned?

• Student-level fidelity may be assessed through teacher logs, observations, or record review

Systems level (school or team)• Are essential components of DBI being implemented consistently?

• Are there systems-level problems that hinder DBI implementation?

• A systems-level rubric or interview might examine team meeting checklists, logs, or other records

26

Why Does Fidelity Matter? Fidelity assessment provides evidence that DBI is being

implemented as intended• Assessment, intervention, and decision-making processes

• Student and systems levels

Provides guidance on how to improve DBI implementation• Does the interventionist need additional training or support?

• Is an intervention adaptation needed?

• Is there a systems-level problem? (e.g., scheduling prevents sufficient intervention time, staff do not have access to evidence-based instructional platforms)

27

Developing a Rubric to Measure Implementation Fidelity

28

Developing a Systems-Level Rubric Based on the structure of the Center on Response to

Intervention’s RTI Essential Components Integrity Rubric http://www.rti4success.org/resource/essential-components-rti-integrity-rubric-and-worksheet

Identify structures, resources, and practices needed for quality school-level implementation of DBI

Reflect lessons learned from NCII knowledge development sites

29

Measuring School-Level Implementation NCII’s DBI Implementation Rubric and Interview: http://

www.intensiveintervention.org/resource/dbi-implementation-rubric-and-interview

Aligned with the essential components of DBI and the infrastructure that is necessary for successful implementation in Grades K–6

30

Key Findings From Knowledge Development Sites Purpose: to learn about strategies for implementing

intensive intervention from sites that have demonstrated positive outcomes for students with disabilities (SWDs)

Implementing Intensive Intervention: Lessons Learned From the Field (NCII, 2013) http://www.intensiveintervention.org/resource/implementing-intensive-intervention-lessons-learned-field

31

Lessons Learned1. Intensive intervention is most likely to be facilitated when

implemented as a component of a multi-tiered system of support (MTSS)

2. Family engagement supports implementation

3. Intensive behavioral intervention is often more challenging because of limited progress monitoring

4. Inconsistent decision rules for intensifying supports can hinder intervention planning and resource allocation

5. Hidden inefficiencies may overtax staff unnecessarily

32

Components/Infrastructure Required for DBI Implementation1. System Features

2. Data and Decision Making

3. Intervention

4. DBI Process

5. DBI Evaluation

33

1. System Featuresa) Leadership

b) School Schedules

c) Resources

d) Cultural and Linguistic Responsiveness

e) Communication With and Involvement of Parents

f) Communication With and Involvement of All Staff

34

2. Data and Decision Makinga) Process to Identify Students in Need of Intensive

Intervention

b) Academic Progress-Monitoring Tools

c) Behavior Progress-Monitoring Tools

d) Progress-Monitoring Procedures

e) Diagnostic Assessment

f) Fidelity of Assessment Implementation

35

3. Interventiona) Intensive Intervention

b) Adaptation of Intervention

c) Fidelity of Implementation of Intervention

d) Interventionist Characteristics

e) Relationship to Core Instruction

36

4. DBI Processa) Team or Appropriate Support Mechanism

b) Regular Meetings

c) Data Review

d) Documentation

e) Decision Rules

f) Overall Implementation of DBI Process

37

5. DBI Evaluationa) Evaluation

38

Rating Scale1–5 points with anchors for 1 point = little or no implementation 3 points = partial or inconsistent implementation 5 points = complete and consistent implementation

39

DBI Implementation Interview Script and note-taking template for gathering information to

evaluate rubric Sample questions for each rubric item

40

Lessons Learned: Initial use in NCII Intensive Sites Strategies for Improving DBI Implementation

41

Highest Rated Items Trained, experienced interventionists Teams to support decision making for DBI, with a regular

meeting schedule Aligning intervention to student need and core

expectations, addressing prerequisite skills as appropriate District and school leadership support for DBI Technically rigorous academic progress-monitoring tools

that are sensitive to student improvement

42

Lowest Rated Items Evaluation Schools may informally review implementation without

setting specific goals Evaluation may be difficult or a low priority if other pieces

are not clearly articulated and in place

Behavior progress monitoring Fewer validated tools available compared to academics New to many schools

43

Lowest Rated Items Continued…. Fidelity Few schools systematically monitor

Decision rules May not be clearly articulated in one or more areas Application may not be consistent

Overall implementation May be inconsistent or poorly documented

44

Essential Elements of DBI Implementation Staff commitment Student plans Student meetings Valid, reliable data Inclusion of students with disabilities

Staff CommitmentKey Element Flexibility Within

Implementation

Commitment of: Principal Intervention staff Special educators

Specific intervention staff involved including staff who work with students with intensive needs in the area(s) of concern. (e.g., reading specialists, social workers) 

46

Student PlansKey Element Flexibility Within

ImplementationStudent plans are developed and reflect: Accurate and timely student

data Goal(s) for the intervention

based on valid, reliable assessment tools

Timeline for executing and revisiting the intervention plan

Content area(s) Number of student plans Grade level(s)

47

Student MeetingsKey Element Flexibility Within

Implementation

Student meetings are data driven

There is a regularly scheduled time to meet

Meetings are structured to maximize efficiency and focused problem solving

Frequency Schedule Team members  

48

Progress Monitoring

Key Element Flexibility Within Implementation

Valid, reliable progress monitoring tools are used.

Data are graphed. Data are collected at

regular intervals.

Choice of tool Use of progress-

monitoring data at other tiers

49

Students With Disabilities

Key Element Flexibility Within Implementation

Students with disabilities must have access to intensive intervention.

Who delivers intervention for students with disabilities

Inclusion of students with and without IEPs

50

NCII & NCSI Resources to Support Implementation

51

Addressing Common Barriers to Implementation Systems level considerations—infrastructure and staff

commitment• Module to assess and support readiness to implement DBI

Identifying intervention and assessment tools for mathematics and behavior• Tools charts

• Sample lessons and activities

Consistent procedures and documentation• Student meeting tools

52

Getting Ready to Implement Intensive Intervention: Infrastructure for DBI Module introducing schools to DBI and infrastructure

needed for implementation: http://www.intensiveintervention.org/resource/getting-ready-implement-intensive-intervention-infrastructure-data-based-individualization

Handouts and activities to assess readiness and begin action planning

53

Fidelity Resources

http://www.intensiveintervention.org/fidelity-resources

54

Fidelity Resources Continued

http://www.rti4success.org/resource/essential-components-rti-integrity-rubric-and-worksheet

Tools Charts

Academic Progress Monitoring http://www.intensiveintervention.org/chart/progress-monitoring

Academic Intervention http://www.intensiveintervention.org/chart/instructional-intervention-tools

Behavioral Progress Monitoring http://www.intensiveintervention.org/chart/behavioral-progress-monitoring-tools

Behavioral Intervention http://www.intensiveintervention.org/chart/behavioral-intervention-chart

56

Mathematics: Sample Lessons & Activities

http://www.intensiveintervention.org/resources/sample-lessons-activities/mathematics

57

Implementing Behavioral Strategies: Considerations and Sample Resources

http://www.intensiveintervention.org/implementing-behavioral-strategies-considerations-and-sample-resources

58

Individual Student Meeting Tools

http://www.intensiveintervention.org/tools-support-intensive-intervention-data-meetings

59

Learn More About DBI NCII website Examples of Standards-Aligned Instruction Across Tiers DBI Training Series Webinars Connect to NCII

60

Find Out What NCII Has to Offer

www.intensiveintervention.org

61

Examples of Standards-Aligned Instruction Across Tiers

http://www.intensiveintervention.org/illustration-standards-relevant-instruction-across-levels-tiered-system

62

DBI Training Series Eight training modules focusing on components of DBI for

academics and behavior One module focused on

readiness and action

planning Include

Slides and speaker notes Activities Coaching guides

http://www.intensiveintervention.org/content/dbi-training-series

63

Webinars View archived webinars and look for announcements about the next live webinar:

www.intensiveintervention.org

64

Connect to NCII Sign up on our website

to receive our newsletter and announcements

Follow us on YouTube and Twitter• YouTube Channel:

National Center on Intensive Intervention

• Twitter handle: @TheNCII

65

Connect to NCSI This website provides immediate, easy

access to foundational NCSI information, previously developed technical assistance resources, a snapshot of the NCSI team, and contact information.

www.ncsi.wested.org/contact/

66

Ask the NCSI Ask the NCSI is intended to support

states to (1) obtain information about evidence-based practices;

(2) develop, implement and evaluate State Systemic Improvement Plans (SSIPs);

(3) learn about practices being implemented in other states; and

(4) find out what current research says about “what works” to improve results for children with disabilities.

www.ncsi.wested.org/contact/

67

Questions

68

ReferencesNational Center on Intensive Intervention. (2013). Implementing intensive

intervention: Lessons learned from the field. Washington, DC: U.S. Department of Education, Office of Special Education Programs. http://www.intensiveintervention.org/sites/default/files/Lessons_Learned_From_Field_0.pdf

Nelson, M. C., Cordray, D. S., Hulleman, C. S., & Sommer, E. C. (2012). A procedure for assessing intervention fidelity in experiments testing educational and behavioral interventions. The Journal of Behavioral Health Services & Research, 39(4), 374–396.

69

NCII & NCSI Disclaimers This presentation was produced under the U.S. Department of

Education, Office of Special Education Programs, Award No. H326Q110005 for which Celia Rosenquist serves as the project officer and No. H326R140006 for which Perry Williams and Shedeh Hajghassemali serve as project officers. The views expressed herein do not necessarily represent the positions or polices of the U.S. Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service, or enterprise mentioned in this website is intended or should be inferred.

Although permission to redistribute this webinar is not necessary, the citation should be: National Center on Intensive Intervention and National Center on Systemic Improvement (2015). Results Driven Accountability and Intensive Intervention: Using MTSS to Improve Outcomes for Students with Disabilities. Washington, DC: U.S. Department of Education, Office of Special Education Programs, National Center on Intensive Intervention.

Contact UsNational Center on Intensive Intervention1000 Thomas Jefferson Street NWWashington, DC 20007-3835

www.intensiveintervention.org

ncii@air.org@TheNCII

National Center on Systemic Improvement730 Harrison StSan Francisco, CA 94107

http://ncsi.wested.org/

NCSI@wested.org