Developing and Implementing a Quality RtI Process Edward Daly, Ph.D. University of Nebraska-Lincoln.
-
Upload
paulina-powers -
Category
Documents
-
view
218 -
download
3
Transcript of Developing and Implementing a Quality RtI Process Edward Daly, Ph.D. University of Nebraska-Lincoln.
Developing and Developing and Implementing a Implementing a Quality RtI ProcessQuality RtI Process
Edward Daly, Ph.D.Edward Daly, Ph.D.
University of Nebraska-LincolnUniversity of Nebraska-Lincoln
Taking RtI “to Taking RtI “to Scale”…Scale”…
And then I looked out my And then I looked out my window…window…
= 1
Is This Really Something Is This Really Something New?New? No!No! Previous versions of IDEA have required…Previous versions of IDEA have required…
““Prereferral” intervention Prereferral” intervention The need to rule out lack of instruction in reading or math or The need to rule out lack of instruction in reading or math or
limited English proficiency limited English proficiency Gathering relevant functional and developmental information Gathering relevant functional and developmental information
and information related to enabling the child to be involved in and information related to enabling the child to be involved in and progress in the general curriculum…and progress in the general curriculum…
The conceptual basis for intervention design has existed for The conceptual basis for intervention design has existed for more than 30 years (Bijou, 1970; Bijou, Peterson, & Ault, 1968).more than 30 years (Bijou, 1970; Bijou, Peterson, & Ault, 1968).
National Academy of Sciences 1982 report on National Academy of Sciences 1982 report on overrepresentation of minorities (Heller, Holtzman, & Messick, overrepresentation of minorities (Heller, Holtzman, & Messick, 1982)1982)
So, What is New?So, What is New?
Intervention data can now be used to Intervention data can now be used to make eligibility decisionsmake eligibility decisions Includes child study assessment informationIncludes child study assessment information Functional assessmentsFunctional assessments Direct assessment and behavioral Direct assessment and behavioral
observationsobservations
Why is RtI Important to Why is RtI Important to BothBoth General and Special Education?General and Special Education?
The Paradox of Special EducationThe Paradox of Special Education Skrtic (1991)Skrtic (1991)
““Structurally, special education is a non-rational Structurally, special education is a non-rational system, an organizational artifact that functions as system, an organizational artifact that functions as a legitimizing device. Culturally, it distorts the a legitimizing device. Culturally, it distorts the anomaly of school failure and thus preserves the anomaly of school failure and thus preserves the prevailing paradigm of school organization, which prevailing paradigm of school organization, which ultimately reinforces the presuppositions of ultimately reinforces the presuppositions of organizational rationality and human pathology in organizational rationality and human pathology in the profession and institution of education in the profession and institution of education in society.” (p. 182)society.” (p. 182)
Why is RtI Important to Why is RtI Important to BothBoth General and Special Education?General and Special Education?
RtI presents a unique opportunity for RtI presents a unique opportunity for educators to get it right…educators to get it right… Transdisciplinary integration of services for Transdisciplinary integration of services for
students across a continuum that meets all students across a continuum that meets all students’ needsstudents’ needs
Implementing RtI: Implementing RtI: Technical AdequacyTechnical Adequacy
Focus on Reliability and Validity of DecisionsFocus on Reliability and Validity of Decisions Standards for Educational and Psychological Testing: Standards for Educational and Psychological Testing: “It is the “It is the
user who must take responsibility for determining whether or user who must take responsibility for determining whether or not scores are sufficiently trustworthy to justify anticipated not scores are sufficiently trustworthy to justify anticipated uses and interpretations.” (p.31)uses and interpretations.” (p.31)
Barnett, Lentz, & Macmann (2000): “Although psychometric Barnett, Lentz, & Macmann (2000): “Although psychometric variables such as reliability and validity have been traditionally variables such as reliability and validity have been traditionally seen as quality of assessment instruments… practitioners seen as quality of assessment instruments… practitioners must consider the psychometric qualities of their must consider the psychometric qualities of their decisionsdecisions across the sequence of problem solving.” (p. 356)across the sequence of problem solving.” (p. 356)
Problem Solving as a Continuous Evaluation CycleProblem Solving as a Continuous Evaluation Cycle
Problem Solving as a Problem Solving as a Continuous Evaluation CycleContinuous Evaluation Cycle
“Is there a discrepancy
between current and expected
performance?”
Why is there a problem?
“By how much should the student grow?”
“What will be done to resolve the problem?”
“Did it work? What do we do
next?”
Implementing RtI: Implementing RtI: Technical AdequacyTechnical Adequacy
Reliability and Validity of DecisionsReliability and Validity of Decisions Problem Solving as a Continuous Problem Solving as a Continuous
Evaluation CycleEvaluation Cycle Local Validation!Local Validation!
Implementing RtI: Implementing RtI: Technical AdequacyTechnical Adequacy
Validating Practice…Easier said than Validating Practice…Easier said than done…done… Reading research has not been readily Reading research has not been readily
translated into practice.translated into practice. Generalizability to actual classroom conditionsGeneralizability to actual classroom conditions Weak generalization of treatment effectsWeak generalization of treatment effects
(Lyon & Moats, 1997)(Lyon & Moats, 1997)
Practitioners need methods that guide them in Practitioners need methods that guide them in investigatinginvestigating interventions directly for their interventions directly for their students.students.
Scientifically supported interventions are not Scientifically supported interventions are not enough!enough!
NASDSE: “Selection and implementation of NASDSE: “Selection and implementation of scientifically based instruction/intervention scientifically based instruction/intervention markedly increases the probability of, but does markedly increases the probability of, but does not guarantee, positive individual response. not guarantee, positive individual response. Therefore, individual response is assessed in RtI Therefore, individual response is assessed in RtI and the modifications to instruction/intervention and the modifications to instruction/intervention or goals are made depending on results with or goals are made depending on results with individual students.”individual students.”
Batsche, Elliot, Graden, Grimes, Kovaleski, Prasse, Reschly, Batsche, Elliot, Graden, Grimes, Kovaleski, Prasse, Reschly, Schrag, & Tilly (2005)Schrag, & Tilly (2005)
Empirical Validation Versus Empirical Validation Versus LocalLocal Validation Validation
May Have Great Scientific May Have Great Scientific Support, But Not A Valid Support, But Not A Valid Intervention for This Student!Intervention for This Student!
010203040506070
1 3 5 7 9 11 13 15 17 19
Correct Read Words and Errors Per Minute Ongoing
Progress Monitoring
Time
Baseline Planned Intervention
This One is Valid for This This One is Valid for This Student!Student!
010203040506070
1 3 5 7 9 11 13 15 17 19
Correct Read Words and Errors Per
Minute
Time
Baseline Planned Intervention 1 Intervention 2
Technical Adequacy of RtI:Technical Adequacy of RtI:Local ValidationLocal Validation
We have to We have to proveprove it works for students. it works for students. If it does not work, we have not finished If it does not work, we have not finished
our job.our job. We are actually testing the treatments We are actually testing the treatments
and not the student!and not the student!
Developing a Valid RtI Developing a Valid RtI Process for StudentsProcess for Students
Quality IndicatorsQuality Indicators Developmental Progression of Developmental Progression of
ImplementationImplementation Strong Operational Model for Continuum Strong Operational Model for Continuum
of RtI Service Deliveryof RtI Service Delivery Evaluation Plan: Validating the ModelEvaluation Plan: Validating the Model
Procedural IntegrityProcedural Integrity Child OutcomesChild Outcomes Refinements and Ongoing EvaluationRefinements and Ongoing Evaluation
Quality Indicators in Quality Indicators in Nebraska: Implementing RtI Nebraska: Implementing RtI in Schoolsin Schools
Parent InvolvementParent Involvement Universal Screening and AssessmentUniversal Screening and Assessment Individual Progress MonitoringIndividual Progress Monitoring Planned Service Delivery RulesPlanned Service Delivery Rules Scientifically Supported InstructionScientifically Supported Instruction Intervention DeliveryIntervention Delivery SLD VerificationSLD Verification
Quality Indicators in Quality Indicators in Nebraska: Organizational Nebraska: Organizational CapacityCapacity
School-wide buy-in and implementation School-wide buy-in and implementation planplan
Team leadershipTeam leadership Integration of ServicesIntegration of Services Implementation InfrastructureImplementation Infrastructure
Developmental Progression Toward Full Developmental Progression Toward Full Scale Implementation of RTIScale Implementation of RTI
Problem Solving Team Engages inData Based Problem Solving
Universal Screening to Identify Students At-Risk for
Academic Difficulties
Coordinated Intervention Delivery
Application of Problem Solving
Data to LD Eligibility
Integrating RtI With Integrating RtI With Existing PracticesExisting Practices
School Improvement PlanSchool Improvement Plan Pre-referral Intervention teamsPre-referral Intervention teams Title I ServicesTitle I Services Professional Learning CommunitiesProfessional Learning Communities
Implementing RtI: Implementing RtI: Procedural IntegrityProcedural Integrity
PROCEDURAL CHECKLIST as an PROCEDURAL CHECKLIST as an operational definition of RtIoperational definition of RtI AccountabilityAccountability Formative evaluation and program Formative evaluation and program
improvementimprovement
Implementing RtI: Implementing RtI: Procedural IntegrityProcedural Integrity
PROCEDURAL CHECKLIST as an PROCEDURAL CHECKLIST as an operational definition of RtIoperational definition of RtI AccountabilityAccountability Formative evaluation and program Formative evaluation and program
improvementimprovement
Procedural Checklist Procedural Checklist Outcome ExampleOutcome Example
Procedural ChecklistProcedural Checklist
04-0504-05 05-0605-06 ChangeChange
NN 6464 229229 +165+165
MeanMean(% steps (% steps
completed)completed)
80%80% 93%93% +13%+13%
SDSD(% steps (% steps
completed)completed)
19%19% 9%9%
Procedural Checklist Procedural Checklist Outcome ExampleOutcome Example
Differences across 04-05 and 05-06 Differences across 04-05 and 05-06 school years:school years:
Approximately 250% more students served in Approximately 250% more students served in the RtI processthe RtI process
A 13% increase in the percentage of A 13% increase in the percentage of procedural steps completed procedural steps completed
Less variability in percentage of steps Less variability in percentage of steps completed among casescompleted among cases
Procedural Checklist Procedural Checklist Outcome ExampleOutcome Example
Despite vast increase in number of students Despite vast increase in number of students served in RtI process in 2served in RtI process in 2ndnd year, higher year, higher procedural fidelity was obtained!!procedural fidelity was obtained!!
Possible influences:Possible influences: Increased organization, comfort and familiarity Increased organization, comfort and familiarity
with processwith process More training for pilot school psychologistsMore training for pilot school psychologists Increased streamlining of services within Increased streamlining of services within
buildingsbuildings
Descriptive Evaluation of Descriptive Evaluation of Individual Child Study Cases Individual Child Study Cases
Produces a quantitative synthesis across Produces a quantitative synthesis across outcome metricsoutcome metrics
Procedural Integrity (Procedural Checklist)Procedural Integrity (Procedural Checklist) Effect Sizes & Percentage of Nonoverlapping Effect Sizes & Percentage of Nonoverlapping
DataData Graphic Analysis WorksheetGraphic Analysis Worksheet Goal Attainment ScalingGoal Attainment Scaling Significance According to “Structured Criteria Significance According to “Structured Criteria
for Visual Inspection of Data”for Visual Inspection of Data” Treatment integrityTreatment integrity Social validitySocial validity
““The significant problems The significant problems we face cannot be solved by we face cannot be solved by the same level of thinking the same level of thinking that created them.”that created them.”
Albert EinsteinAlbert Einstein
Real Change in SchoolsReal Change in Schools
Fullan (1991)Fullan (1991) ““innovations can be adopted for symbolic political or innovations can be adopted for symbolic political or
personal reasons: to appease community pressure, personal reasons: to appease community pressure, to appear innovative, to gain more resources. All of to appear innovative, to gain more resources. All of these forms represent these forms represent symbolicsymbolic rather than rather than realreal change.” (p. 28)change.” (p. 28)
““Even good ideas may represent poor investments Even good ideas may represent poor investments on a large scale if the ideas have not been well on a large scale if the ideas have not been well developed or if the resources to support developed or if the resources to support implementation are unavailable.” (p. 28)implementation are unavailable.” (p. 28)
ConclusionsConclusions
Building a Reliable and Valid Model for Building a Reliable and Valid Model for RtIRtI Quality IndicatorsQuality Indicators Procedural IntegrityProcedural Integrity Local Validation Through Databased Local Validation Through Databased
Problem SolvingProblem Solving Strategic and Sequential Development and Strategic and Sequential Development and
RefinementRefinement
ReferencesReferences American Educational Research Association, American Psychological American Educational Research Association, American Psychological
Association, & the National Council on Measurement in Education Association, & the National Council on Measurement in Education (1999). (1999). Standards for educational and psychological testing.Standards for educational and psychological testing. Washington, DC: American Psychological Association. Washington, DC: American Psychological Association.
Barnett, D. W., Lentz, F. E., Jr., & Macmann, G. (2000). Psychometric Barnett, D. W., Lentz, F. E., Jr., & Macmann, G. (2000). Psychometric qualities of professional practice. In E. S. Shapiro & T. R. Kratochwill qualities of professional practice. In E. S. Shapiro & T. R. Kratochwill (Eds.), (Eds.), Behavioral assessment in schools: Theory, research, and clinical Behavioral assessment in schools: Theory, research, and clinical foundations.foundations. (2nd ed., pp. 355-386). New York, NY: Guilford (2nd ed., pp. 355-386). New York, NY: Guilford Publications. Publications.
Bijou, S. W. (1970). What psychology has to offer education--now. Bijou, S. W. (1970). What psychology has to offer education--now. Journal of Applied Behavior Analysis, 3,Journal of Applied Behavior Analysis, 3, 65-71. 65-71.
Bijou, S. W., Peterson, R. F., & Ault, M. H. (1968). A method to integrate Bijou, S. W., Peterson, R. F., & Ault, M. H. (1968). A method to integrate descriptive and experimental field studies at the level of data and descriptive and experimental field studies at the level of data and empirical concepts. empirical concepts. Journal of Applied Behavior Analysis, 1,Journal of Applied Behavior Analysis, 1, 175-191. 175-191.
Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing the identification of learning disabilities. for reconceptualizing the identification of learning disabilities. Learning Learning Disabilities Research and Practice, 13,Disabilities Research and Practice, 13, 204-219. 204-219.
References ContinuedReferences Continued Fisher, W. W., Kelley, M. E., & Lomas, J. E. (2003). Visual aids and Fisher, W. W., Kelley, M. E., & Lomas, J. E. (2003). Visual aids and
structured criteria for improving visual inspection and interpretation of structured criteria for improving visual inspection and interpretation of single-case designs. single-case designs. Journal of Applied Behavior Analysis, 36, Journal of Applied Behavior Analysis, 36, 387-406. 387-406.
Fullan, M. G. (1991). Fullan, M. G. (1991). The new meaning of educational changeThe new meaning of educational change (2nd ed.). (2nd ed.). New York, NY: Teachers College Press. New York, NY: Teachers College Press.
Heller, K. A., Holtzman, W. H., & Messick, S. (Eds.). (1982). Heller, K. A., Holtzman, W. H., & Messick, S. (Eds.). (1982). Placing Placing children in special education: A strategy for equity.children in special education: A strategy for equity. Washington, DC: Washington, DC: National Academy Press. National Academy Press.
Lyon, G. R., & Moats, L. C. (1997). Critical conceptual and methodological Lyon, G. R., & Moats, L. C. (1997). Critical conceptual and methodological considerations in reading intervention research. considerations in reading intervention research. Journal of Learning Journal of Learning Disabilities, 30,Disabilities, 30, 578-588. 578-588.
President’s Commission on Excellence in Special Education. (2002). President’s Commission on Excellence in Special Education. (2002). A A new era: Revitalizing special education for children and their familiesnew era: Revitalizing special education for children and their families. . Available on-line: Available on-line: http://http://www.ed.gov/inits/commissionsboards/whspecialeducationwww.ed.gov/inits/commissionsboards/whspecialeducation
Skrtic, T. M. (1991). Skrtic, T. M. (1991). Behind special education: A critical analysis of Behind special education: A critical analysis of professional culture and school organization.professional culture and school organization. Denver, CO: Love Publishing Denver, CO: Love Publishing Co.Co.