Combining Response to Intervention (RTI) with Comprehensive
Transcript of Combining Response to Intervention (RTI) with Comprehensive
Combining Response to Intervention (RTI) with Comprehensive Evaluation for Serving
All Children in Schools James B. Hale, PhD, MEd, ABPdN, ABSNP
Professor of Education and Medicine
University of Calgary
Kelly Ryan Hicks, M.Sc.Registered Psychologist
School Psychologist, Calgary Board of Education
http://www.educ.ucalgary.ca/braingain
Manitoba Association of School Psychologists14-15 November 2013
The Education Pioneers and the Search for Differentiated Instruction
• Orton (1937) Incomplete left cerebral dominance caused strephosymbolia – twisted symbols
• Werner (1940): TBI attention/perceptual problems required individualized assessment and intervention
• Fernald (1943): Visual-Auditory-Kinesthetic-Tactile (VAKT) approach
• Clements (1966): Minimal Brain Dysfunction
• Kirk (1968): “Learning Disabilities” and educational focus
• Ysseldyke (1973): Failed aptitude-treatment interactions (ATI), focus on outcome, not process
• Hammill (1974): Aptitude-treatment interactions meaningless, focus on academics
The Education Pioneers and the Search for Differentiated Instruction
• Cronbach (1975): Correlational versus euxperimental psychology and importance of outcomes
• PL 94-142 (1975): Specific Learning Disabilities (SLD) definition and “ability”–“achievement”discrepancy
• Bandura (1977): Social Learning Theory and impact of cognition on behaviour and environment
• Deshler and Schumaker (1982): Learning Strategies curriculum at the University of Kansas
• Gardner (1983): Multiple intelligences and humanistic approach to education
• Sternberg (1985): Triarchic Theory of Intelligence (Experiential, Componential, Contextual) important for separating knowledge and skill
Why Differentiated Instruction Hasn’t Worked(or Has Ever Really Been Attempted)
• Limited teacher training in applying science to practice
• Teachers taught about “multiple intelligences”, but few understand how they affect student performance or teaching
• Teachers have limited training in causes of learning problems
• Teachers have to use trial and error to understand how different disorders respond to intervention
• Training and support of teachers in serving diverse student needs is essential, but they don’t get enough help
Teacher Issues
• School psychologists have limited training in individual differences, classroom instruction, and neuroscience of learning
• Educational disability categories nonspecific, so team decisions are inconsistent and not related to instruction
• School psychologist evaluations often focused on a number approach, not about child strengths and needs
• School psychologist evaluations used for entitlement purposes (e.g., “test and place”), not used for guiding intervention
Why Differentiated Instruction Hasn’t Worked(or Has Ever Really Been Attempted)
School Psychologist Issues
Alberta Ministry of Education Literacy Benchmarks: Brain-Curricula Connections for ALL Children
?
• Advances in the neuroscience of learning and behaviour
• Brain changes more quickly than thought before: Implications for typical learners and instruction
• Brain matters for early intervention and identification of special need
• Brain matters for differential diagnosis of disability
• Brain matters for targeted interventions for struggling learners
School Neuropsychology: Three Realities
Reality #1 The brain is the organ responsible for learning
Reality #2 Children with disabilities process information differently than typical children
Reality #3 Teaching results in changes in brain functioning
Neuropsychology in the Schools: Will Teachers Lead and School Psychologists Follow?
• Limited to no training in neuroscience of learning, behaviour, and disability limited
• Publications/journals include little to no information on neuropsychology
• Anti-testing advocates suggest no “aptitude-treatment interactions” based on research that is over 30 years old
Will School Psychologists Be Left Behind?
Neuroscience and
Special Education Eve Müller, Ph.D.
July, 2011
inForum – An Analysis of a
Critical Issue in Special Education
Brief Policy Analysis
The purpose of this document is to
provide a brief overview of how links are
being developed between the rapidly
expanding field of neuroscience and the
practice of special education. The
document introduces definitions and
terminology and provides an overview of
how findings from neuroscience are being
applied to the field of special education.
Current State of School Psychology
• Teachers see cognitive diversity in classroom
• Teachers have desire to learn brain-based educational practices, but have no classes on topic
• Teachers seek left-brain/right-brain and multiple intelligences orientation
• Brain-based training being replaced with the neuroscience of education
Current State of Teacher Education
Is DSM-5 The Answer?
• Former Chair of DSM-IV says “saddest moment” in 45 years of being leader of psychiatry
• Opposed by 50 mental health organizations, ignore 10 worse changes
• Changes in autism, anxiety, eating, mood, ADHD, and behaviourdisorders all problematic
• Not evidence-based – limited to no science behind practice, more based on special interests and financial incentives
• DSM-5 violates most sacred tenet in medicine- First Do No Harm!
Embrace the Neuroscience Revolution!
Blessing or Bane? Allen Frances, M.D.
• Thomas Insel, MD, Director of NIMH: DSM diagnoses are a “consensus about clinical symptoms” practice largely abandoned in modern medicine
• Patients with mental disorders “deserve better”
• Embrace genetics, neuroimaging, and cognitive neuroscience to “lay the foundation” for a new classification system
National Insitutes of Mental Health: Thomas Insel, M.D.
If The Model is Broken Let’s Fix it:Why Comprehensive Evaluations Are
Necessary for Effective Service Delivery
Problems with TraditionalSLD Identification Practices
• Pre-referral strategies not effective, documented, OR funded
• Early intervention critical, but young children seldom qualify – must “fail” before services are provided
Wait and Fail
• Standardized test results unrelated to outcomes (Due to IQ only?)
• Crystallized abilities, sociocultural, language, and performance issues
• Test Bias versus Fairness and interpretation of results
Norm Referenced Assessment
Problems with TraditionalSLD Identification Practices (Continued)
• Poorly operationalized SLD definitions and methods
• No documentation of “deficit in basic psychological processes”
• Disconnect between SLD definition and SLD eligibility
LD Definition Ignored
• Ability-Achievement Discrepant and Low Achieving groups similar (funding? differential diagnosis?)
• Learning delays not deficits – All children learn similarly, just different intensity needed (notsupported; See Francis et al. 1996 and Castellanos et al., 2002)
Low Achievement vs. Discrepancy
Defining Learning Disabilities in CanadaLearning Disabilities Association of Canada, 2002
Learning Disabilities affect the acquisition, organization, retention, understanding or use of information
SLD results from impairments in one or more processes related to perceiving, thinking, remembering or learning, such as language; phonological; visual-spatial; processing speed; attention, memory, and executive functions.
SLD interferes with acquisition and use of oral language (e.g. listening, speaking, understanding); reading (e.g. decoding, phonetic knowledge, word recognition, comprehension); written language (e.g. spelling and written expression); and/ormathematics (e.g. computation, problem solving).
1. Average Intelligence with Processing Impairments AffectingAcademic and Psychosocial Functioning
SLD occurs in individuals with otherwise average thinking and/or reasoning, so are distinct from global intellectual deficiency
SLD can lead to organizational skill, social perception, social interaction and perspective taking difficulties.
Defining Learning Disabilities in CanadaLearning Disabilities Association of Canada, 2002
2. Etiology, Prognosis, and Exclusion: Not Due to Poor Instruction
SLD is lifelong, but expression varies over lifetime depending on the interaction between environment and the individual’s strengths and needs.
SLD is suggested by unexpected academic under-achievement or achievement which is maintained only by unusually high levels of effort and support.
SLD is due to genetic and/or neurobiological factors or injury that alters brain functioning in a manner which affects one or more processes related to learning.
SLD is not due primarily to hearing and/or vision problems, socio-economic factors, cultural or linguistic differences, lack of motivation or ineffective teaching, although these factors may further complicate outcomes.
SLD may be comorbid with various conditions including attentional, behavioural and emotional disorders, sensory impairments or other medical conditions.
Defining Learning Disabilities in CanadaLearning Disabilities Association of Canada, 2002
3. Intervention: More Intense is NOT Sufficient
SLD requires early identification and timely specialized assessments and interventions involving home, school, community and workplace settings.
The interventions need to be appropriate for each individual's learning disability subtype and, at a minimum, include the provision of:
specific skill instruction;
accommodations;
compensatory strategies; and
self-advocacy skills.
(LDA BOLD PROVIDED ON WEBSITE)
Identifying Children with Learning Disabilities:IDEA (2004) Definitions and Practices
• Learning Disabilities is a disorder of one or more of the basic psychological processes that adversely affects achievement in one or more academic domains
Defining Learning Disabilities
• Discrepancy between “ability” and “achievement”
• Failure to respond to scientific research-based intervention
• May permit the use of other alternative research-based procedures for determining whether a child has a specific learning disability (§300.8(c)(10) OSERS Final Regulations, August, 2006)
Determining Learning Disabilities
• Uniform discrepancy application is insensitive to developmental differences in cognition and achievement
• Difficulty distinguishing between children with SLD and low achievers
• Inconsistent application of discrepancy across schools, districts, and states
• Over-identification of students from diverse backgrounds
• Measurement problems that result in poor decision-making
• Unclear which IQ score should be used to establish “ability” for discrepancy calculation
• Early identification critical but unlikely (“wait-to-fail”)
• Encourages “test and place” practices which are not an effective use of resources, or related to intervention
Ability-Achievement Discrepancy Problems:
Problems with IDEA Re-AuthorizationFor Determining Specific Learning Disabilities
Problems with IDEA Re-AuthorizationFor Determining Specific Learning Disabilities
Response-to-Intervention Problems:• No consensus on type of RTI to use (i.e., standard protocol or
problem-solving)
• No agreed upon curricula, instructional methods, or measurement tools with adequate technical quality
• RTI research has focused on word reading at the early elementary grades, with methods across grades and content areas not empirically established
• No consensus on the definition of empirically-based approaches
• Single subject design cannot be used because manipulation of more than one independent variable in problem-solving RTI precludes determining causation
• No empirically-supported literature supporting determination of response or failure to respond
Problems with IDEA Re-AuthorizationFor Determining Specific Learning Disabilities
Response-to-Intervention Problems (Continued):• No agreed upon teacher training standards or supervision
methods to ensure interventions are carried out with integrity
• RTI has no mechanism for differential diagnosis of SLD and other disorders
• RTI is nothing more than a model of “diagnosis by treatment failure”, which has long been proven to be a poor model in medicine
• There is no true positive in an RTI model – all children who fail to respond are considered SLD by default
• Method does not pass “Dead Man’s Test”, as all dead people are now SLD!
Learning Disabilities Association of America Summit on Specific Learning Disability Evaluation, Identification, and Service Delivery
Keynote Address
• Hale, J. B. & McHale-Small, M. (2010, February). Achieving professional consensus on specific learning disabilities evaluation, identification, and service delivery. Keynote Address at LDA SLD Summit, Baltimore, MD.
Symposium (Chair: James B. Hale; Discussant: Larry Silver)
• Flanagan, D. P. (2010, February). Cognitive assessment is important for SLD identification and treatment. Symposium presentation at LDA SLD Summit, Baltimore, MD.
• Pugh, K. R. (2010, February). Neuroimaging in reading disability research: Benefits and challenges. Symposium presentation at LDA SLD Summit, Baltimore, MD.
• Mazzocco, M. (2010, February). Mathematical learning disabilities: Evaluation, identification, and service delivery. Symposium presentation at LDA SLD Summit, Baltimore, MD.
• Ortiz, S. O. (2010, February). Fairness and equity in the evaluation of specific learning disabilities in English language learners. Symposium presentation at LDA SLD Summit, Baltimore, MD.
• Simon, J. (2010, February). Legal aspects of specific learning disability evaluation, identification, and service delivery. Symposium presentation at LDA SLD Summit, Baltimore, MD.
Learning Disabilities Association of America White Paperand Learning Disability Quarterly Publication
http://www.iqscorner.com/2010/08/hale-et-al-2010-expert-consensus.html
Learning Disability Quarterly, 33, 223-236
The Learning Disabilities Association of America’s
White Paper on Evaluation, Identification, and Eligibility Criteria
for Students with Specific Learning Disabilities
The background and reason for the White Paper became apparent when the
Individuals with Disabilities Education Act (IDEA) Statute was published in 2004.
Members of the LDA Board of Directors were pleased that the definition of Specific
Learning Disabilities (SLD) remained intact. But when the Regulations were
published in 2006, it was surprising to find that the SLD evaluation criteria and
identification criteria were no longer aligned with the SLD definition in IDEA. Both of
these criteria changed from taking the cognitive nature of SLD into consideration, to
instead aligning IDEA with the regulations in the Elementary and Secondary
Education Act (ESEA/NCLB) and putting the emphasis on identifying students who
are not achieving adequate for the child’s age or the attainment of State‐approved
grade‐level standards, not abilities. In effect, the new criteria virtually eliminated a
great many students with SLD, including some who have high academic
achievement in some areas but markedly low achievement in other areas.
In 2008 LDA partnered with a group of professionals who were also concerned that
the cognitive nature of SLD was not given much, or in some cases, no consideration
but rather was looked upon as a condition that is educational in nature. The idea for
the White Paper grew out of this partnership of professionals and members of LDA
and was presented at a Symposium held at the LDA International Conference held in
Baltimore, February 2010.
LDA White Paper, February 2010
http://www.ldanatl.org/pdf/LDA%20White%20Paper%20on%20IDEA%20E
valuation%20Criteria%20for%20SLD.pdf
Identifying Reading Disabilities by Responsiveness-to-Instruction:
Specifying Measures and CriteriaDouglas Fuchs, Lynn S. Fuchs, Donald L. Compton
Abstract
This article first describes two types of assessment (problem solving and standard
treatment protocol) within a "responsiveness-to-instruction" framework to identify
learning disabilities. It then specifies two necessary components (measures and
classification criteria) to assess responsiveness-to-instruction, and present
pertinent findings from two related studies. These studies involve databases at
grades 1 and 2, which were analyzed to compare the soundness of alternative
methods of assessing instructional responsiveness to identify reading disabilities.
Finally, conclusions are drawn and future research is outlined to prospectively and
longitudinally explore classification issues that emerged from the analyses.
Learning Disability Quarterly, 27, 216-227 (2004)
Is the Expert Panel Opinion Justified?Examining Research on Determining RTI
Hitting the Moving Target Known as Reading Development
Some Thoughts on Screening Children for Secondary InterventionsDeborah L. Speece
Abstract
Attention to growth may net more accurate screening results than have been
apparent in the long history of early identification research. Instead of relying
solely on measures administered at one time point, including a measure of
children's growth or responsiveness may add an important dimension to the
screening equation and provide a clearer view of who may require secondary
intervention. Several possibilities were examined that led to suggestions for
further research. It was recommended that screening for secondary
interventions be viewed as a three-legged stool that incorporates not only
individual differences at the screen and outcome but also development
conceptualized as rate of learning.
Journal of Learning Disabilties, 38, 487-493 (2005)
Is the Expert Panel Opinion Justified?Examining Research on Determining RTI
Agreement Among Response to Intervention Criteria for Identifying
Responder Status Amy E. Barth, Karla K. Stuebing, Jason L. Anthony, Carolyn A. Denton,
Patricia G. Mathes, Jack M. Fletcher and David J. Francis
Abstract
In order to better understand the extent to which operationalizations of response
to intervention (RTI) overlap and agree in identifying adequate and inadequate
responders, an existing database of 399 first grade students was evaluated in
relation to cut-points, measures, and methods frequently cited for the
identification of inadequate responders to instruction. A series of 543 2× 2
measures of association (808 total comparisons) were computed to address the
agreement of different operationalizations of RTI. The results indicate that
agreement is generally poor and that different methods tend to identify
different students as inadequate responders, although agreement for
identifying adequate responders is higher. Approaches to the assessment of
responder status must use multiple criteria and avoid formulaic decision making.
Learning and Individual Differences, 18, 296-307 (2008)
Is the Expert Panel Opinion Justified?Examining Research on Determining RTI
Examining Agreement and Longitudinal Stability Among Traditional and RTI-
Based Definitions of Reading Disability Using the Affected-Status Agreement
Statistic Brown-Waesche, J. S. B., Schnatschneidedr, C., Maner, J., Ahmed, Y., & Wagner, R.
Abstract
Rates of agreement among alternative definitions of reading disability and their 1- and 2-year
stabilities were examined using a new measure of agreement, the affected-status agreement
statistic. Participants were 288,114 first through third grade students. Reading measures were
Dynamic Indicators of Basic Early Literacy Skills Oral Reading Fluency and Nonsense Word
Fluency, and six levels of severity of poor reading were examined (25th, 20th, 15th, 10th, 5th,
and 3rd percentile ranks). Four definitions were compared, including traditional unexpected
low achievement and three response-to-intervention-based definitions: low achievement, low
growth, and dual discrepancy. Rates of agreement were variable but only poor to
moderate overall, with poorest agreement between unexpected low achievement and the
other definitions. Longitudinal stability was poor, with poorest stability for the low
growth definition. Implications for research and practice are discussed.
Journal of Learning Disabilities © 2011
Is the Expert Panel Opinion Justified?Examining Research on Determining RTI
Essentials of Specific Learning Disability Identification
• “There is a plethora of reasons why children do not respond to our best attempts at intervention, only one of which is SLD. Identifying a child as having SLD only because he or she did not respond to intervention is essentially diagnosis by default – something most researchers and practitioners agree is not scientifically or empirically sound practice.”
• “[D]efinition and measurement problems explain, in part, why studies that have tried to use RTI methods to determine SLD have been unsuccessful: that different methods for determining response result in different subsets of children classified as responders or nonresponders…In other words, using RTI for SLD identification is unreliable and, therefore, invalid because there is no true positive in an RTI model.”
• “Despite its promise in serving the needs of many children, we must conclude RTI can never be a valid method for SLD identification, because it is scientifically flawed as a method of disability determination.”
Hale, Wycoff, Fiorello, 2010
Journal of Psychoeducational Assessment
© 2010 SAGE Publications
DOI: 10.1177/0734282910388598
http://jpa.sagepub.com
Forest Grove School District v. T.A. Supreme Court Case:
Implications for School Psychology PracticeShauna G. Dixon1, Eleazar C. Eusebio2, William J. Turton2, Peter W. D. Wright3 and James B. Hale4
AbstractThe 2009 Forest Grove School District v. T.A. United States Supreme Court case could have significant
implications for school psychology practice. The Court ruled that the parents of a student with a disability were
entitled to private school tuition reimbursement even though T.A. had not been identified with a disability or
previously provided with special education services. The district multidisciplinary team had not identified T.A.’sdisability in part because of a limited psychoeducational evaluation that did not assess him in “all areas of
suspected disability” which is a requirement of the Individuals with Disabilities Education Act (IDEA; 20 U.S.C.
§ 1414(b)(3)(B)). Dissatisfied with the team’s decision, the parents sought an independent comprehensive
evaluation of T.A.’s psychological processes which revealed the full nature of his disability and needs. Although
T.A. had been unresponsive to interventions in his home school, this comprehensive psychoeducational
evaluation led to more targeted interventions in his private school placement and likely contributed to T.A.’ssubsequent academic and behavioral success. This Supreme Court case highlights the need for school-
based teams to conduct comprehensive evaluations in all areas of suspected disability, including
psychological processes such as attention, memory, and executive function. The relevant IDEA 2004 and
governing 2006 federal regulations are presented, followed by recommendations for school psychologist
assessment practices to ensure compliance with IDEA statutory and regulatory requirements.
The United States and Canadian Supreme Court Positions
What about Canada? See McGovern v. BC Supreme Court of Canada, 2004-2007; Moore v. British Columbia-Education - 2012 SCC 61 - 2012-11-09 Appeals
IDEA (2004) Definitions and PracticesOSERS Commentary
• “RTI is only one component of the process to identify children in need of special education and related services.”
• “Determining why a child has not responded to research-based interventions requires a comprehensive evaluation”
• “An RTI process does not replace the need for a comprehensive evaluation”
• “Eligibility group can determine that a child has an SLD if the child meets the criteria in §300.309(a)(1) and exhibits a pattern of strengths and weaknesses in performance, achievement, or both, relative to age and State-approved grade-level standards, or intellectual [cognitive] development.”
The Special Case of Specific Learning Disabilities (SLD): Critical Issues
• IQ interpretation is not empirically-supported, does not lead to intervention, and discrepancy is invalid for SLD identification, so should we avoid ability-achievement discrepancy?
• Response-to-Intervention (RTI) can help many children be successful, but nonresponse can be due to multiple possible causes and is not empirically-supported, therefore, RTI is invalid for SLD identification, so should we avoid using RTI?
• How do you document the “deficit in the basic psychological processes” given that neither RTI nor discrepancy address this critical SLD definition
So what’s a responsible school psychologist supposed to do? (AND maintain your sanity!)
Hale (2006) Model of Service Delivery
Source: Hale, J. B. (2006). Implementing IDEA with a three-tier model that includes response to intervention and cognitive assessment methods. School Psychology Forum: Research and Practice, 1, 16-27.
Tier 1: 85%
Served
Problem-Solving RTI Approach Individualized Special
Education
Standard Protocol Instruction
RTI Approach
Tier 2: 10%
Served
Tier 3:5%
Served
Curriculum-BasedMeasurement
Ongoing Progress Monitoring
Individualized MeasurementSingle Subject Designs
Individualized MeasurementSingle Subject Designs
ComprehensiveCognitive
Hypothesis Testing Evaluation
Tier 1Standardized Protocol RTI
• Standardized curriculum and measurement tools in all academic areas (at least reading, math, and writing)
• Standardized evidence-based instructional format and techniques
• Standardized teacher training programs
• Standardized assessment of teacher competence
• Standardized benchmarks for academic achievement across curricular areas and grade levels
• Standardized assessment of student progress toward benchmarks (e.g., DIBELS, AIMSweb).
• Standardized criteria for acceptable performance and referral for Tier 2 RtI services
Ensures EXTERNAL Validity in SLD Determination
See Gerber, 2005; Fuchs, Mock, Morgan, & Young, 2003; O’Connor, Harty, & Fulmer, 2005;Vaughn, Linan-Thompson, & Hickman, 2003; Vellutino et al., 1996
Tier 1Standardized Protocol RTI
Maximizes EXTERNAL Validity
• Standardized Instruction, Curriculum, Methods, Measures, Benchmarks + Teacher Competency!
• Ongoing Progress Monitoring (e.g., DIBELS, AIMSweb)
• Decision Points (Duration? Local vs. National Norms? Response vs. Nonresponse?)
Responder?
YES, Continue GeneralEducation and Progress Monitoring
NO, Begin Tier 2Problem-SolvingModel Intervention
Tier 2Problem Solving Model RTI
• Problem Identification - Target behaviors, observable and measurable, hierarchical ordering/prioritize
• Problem Analysis - Functional determinants, ABC-functional analysis, Who/What/Where/When/Why, skill vs. performance deficit, discrepancy between expected and actual
• Plan Development/Implementation – Brainstorming, feasibility, prioritizing, resources, decision, commitment
• Plan Evaluation/Recycling - Outcome focused, self-correcting, data-based decision making, single subject designs, continue/terminate/recycle as necessary
Ensures INTERNAL Validity in SLD Determination
See Deno, 2002; Fuchs et al., 2004; Kovaleski, 2002; Marston, 2005; Reschly, 2005; Tilly, Reschly, & Grimes, 1999
Tier 2Problem Solving Model RTI
Maximizes INTERNAL Validity
• Problem Identification
• Problem Analysis
• Plan Development/Implementation
• Plan Evaluation/Recycling
Responder?
YES, Return to Tier 1General Education and Progress Monitoring
NO, Consider ComprehensiveEvaluation and Cognitive Hypothesis Testing Model
Tier 1Standardized RTI Approach
Responder?
YES, Continue GeneralEducation and Progress Monitoring
NO, Begin Tier 2Problem-SolvingModel Intervention
Responder?
YES, Return to Tier 1General Education and Progress Monitoring
NO, Consider ComprehensiveEvaluation and Cognitive Hypothesis Testing Model
Tier 2Problem Solving Model RTI Approach
The Cognitive Hypothesis Testing Model
Source: Hale, J. B., & Fiorello, C. A. (2004). School Neuropsychology: A Practitioner’s Handbook. New York, NY: Guilford Press.
Theory
Hypothesis
Data Collection
Interpretation
1. Presenting Problem
2.Intellectual/Cognitive Problem
3. Administer/Score Intelligence Test
4. Interpret IQ or Demands Analysis
5. Cognitive Strengths/Weaknesses
6. Choose Related Construct Test
7. Administer/Score Related Construct Test
8. Interpret Constructs/Compare
9. Intervention Consultation
10. Choose Plausible Intervention
11. Collect Objective Intervention Data
12. Determine Intervention Efficacy
13. Continue/Terminate/Modify
Tier 3Comprehensive Evaluation for Special Education
Determination and Service Delivery
Cognitive Hypothesis Testing forIntervention Development and Efficacy:
SNAP-FIT in Local Area Schools
Implementing Response to Intervention in Schools: Basic Tenets
• RTI can foster good instructional practices and early identification of learning difficulties
• RTI can serve many more children, especially before significant instructional time is lost
• RTI can be essential practice for serving children with learning delays, but probably not most children with learning deficits
• RTI can reduce need for costly comprehensive evaluations and special education services
Fuchs, Fuchs, & Compton, D.L. (2004). Identifying reading disabilities by responsiveness-to-instruction: specifying measures and criteria. Learning Disabilities Quarterly, 27, 216-227.
Hale, Naglieri, Kaufman, & Kavale (2006). Implementation of IDEA: Integrating response to intervention and cognitive assessment methods. Psychology in the Schools, 43, 753-770.
Mastropieri,& Scruggs, (2005). Feasibility and consequences of response to intervention: examination of issues
and scientific evidence as a model for the identification of individuals with learning disabilities..Journal of Learning Disabilities, 38, 525-531.
Implementing Response toIntervention in Schools
• Provides systematic screening of all students
• Provides frequent progress monitoring to evaluate instruction and responsiveness of students
• Provides quantitative and graphical display of student progress and reduces bias in reporting
• Ensures quality of general education program and empirically-supported instruction
• Provides struggling students with extra assistance to experience success
• Reduces number of referrals for comprehensive evaluations and special education service delivery
SNAP-FIT: Combining RTI with Comprehensive Evaluation in a Balanced Practice Model
• Training teachers and school psychologists in practices and principles of RTI
• Training teachers and school psychologists in practices and principles of neuroscience
• Increasing intensity first provided in multiple tiers (standardized and problem solving)
• For nonresponders, comprehensive SNAP-FIT evaluations followed by individualized instruction and single subject designs
• Elementary and middle school pilot, but trainings and principles incorporated at both primary and secondary schools
Data Collection, Analysis, and Decision RulesStep 1: Tier 1 Universal Screening – benchmark scores used for
universal screening of all students in September, January, and May. Consultation with teachers to ensure evidence-based teaching
Step 2: Examine oral reading scores, students <25th %ile for oral reading require more careful analysis
Step 3: Students <25th %ile in oral reading considered for Tier 2 individualized instruction according to their areas of deficit (phonemic awareness, phonics, fluency, comprehension)
Step 4: Students 25-50th %ile, word list used to determine if sight word recognition weakness
Step 5: After phonemic awareness, phonics, orthographic, sight word identification, and fluency rule outs, child considered for comprehension intervention group if reading problem reported
Step 6: Students performing above 75th percentile in benchmark assessments considered for enrichment groups
Step 7: After 4-6 weeks, review Tier 2 progress monitoring data from six intervention groups designed to meet skill deficits
Following decisions for each student:
• Student made sufficient progress, and should return to Tier 1
• Student has made some progress; however, he/she should remain in Tier 2 intervention for further skill development
• Student has not made sufficient progress, and should be considered for a comprehensive evaluation and targeted intervention using the Cognitive Hypothesis Testing approach
Data Collection, Analysis, and Decision Rules
Decision Questions for Referral for and Completion of Comprehensive Evaluation
• Should the intervention be more intense or has instruction been sufficiently modified ?
• Was treatment integrity maintained throughout the tiers?
• Is it ethical to keep a student in early RTI tiers who is not responding to more intense over a series of intervention periods?
• Is RTI data sufficient to provide a historical context for the comprehensive evaluation and can it be used to guide cognitive hypothesis generation?
• Has RTI data been combined with comprehensive evaluation data to plan targeted interventions?
• What resources are necessary to carry out targeted remedial or compensatory interventions?
• How will individualized progress monitoring of targeted interventions be evaluated and decisions made regarding tier status?
Math Benchmark Testing
0%
10%
20%
30%
40%
50%
60%
Fall Winter Spring
Below Basic
Basic
Proficient
Advanced
RTI School vs. Non-RTI School “Hit Rate” for Nonresponders at Tiers 1 and 2
0%
20%
40%
60%
80%
100%
120%
06-07 07-08 08-09 09-10
Pe
rce
nt
Fo
un
d E
xce
pti
on
al
School Year
RTI School
Non-RTI School
This could suggest that nonresponse is an accurate Indicator of SLD statusBUT, cognitive hypothesis testing results revealed ADHD, Tourette’s, Generalized Anxiety, Bipolar, ODD, and several subtypes of reading, math, and writing disability
IQ and “g” – Fact, Legend, or Myth?Why IQ and Intelligence Testing Nearly Destroyed Our Profession
• Reschly & Gresham (1989): A Leap of Faith
• Herrnstein & Murray (1994): The Bell Curve
• Macmann & Barnett (1997): Myth of the Master Detective
• McDermott & Glutting (1997): Illusions of Meaning “JUST SAY NO”
• Glutting et al. (2006): Distinctions without a Difference
• Lezak (1988): IQ: Rest in Peace
• Kaplan (1988): The Boston Process Approach
• McGrew & Flanagan (1998): CHC & Cross Battery Approach
• Flanagan & Kaufman (2004): Intelligence Testing
• Hale & Fiorello (2004): Cognitive Hypothesis Testing
• Hale et al. (2007): The Demise of Global IQ
IQ: The Only Score IQ: The Misleading Score
Major Methods of Studying Intelligence• Factor Analysis: One or Many Factors?
One – Spearman’s (1904) General Intelligence (g)
Many – Thurstone (1938) Primary Mental Abilities
• Regression Methods: Hierarchical Regression/ANCOVA?
Enter FSIQ/Enter Factors – Glutting et al. (1997, 2006)
Enter Factors/Enter FSIQ – Hale et al. (2001; 2007)
• Commonality Analysis: Unique or Shared Variance?
Limited shared factor variance in predicting achievement (Fiorelloet al., 2001, 2007; Hale et al. 2001; 2007; 2008) andneuropsychological functions (Hale et al., 2002)
Limited shared factor variance in Full Scale IQ for typical children and children with disabilities (Fiorello et al., 2001, 2007; Hale et al., 2001;2007, 2008)
Single vs. Multiple IntelligencesWISC-IV/WIAT-II Linking Sample (N = 668)
Vocabulary
Comprehension
Similarities
Digit Span
Letter/Number Seq.
Block Design
Picture Concepts
Matrices
Coding
Symbol Search
.71
.74
.50
.60
.57
.42
.41
.51
.36
.42
.77
.84
.82
.84
.75
.72
.66
.65
.85
.79
g
VC
WM
PR
PS
Note. VC = Verbal Comprehension, WM = Working Memory; PR = Perceptual Reasoning; PS = Processing Speed
Step 1. Enter FSIQ
Predicting Basic Reading with WISC-III/ WIAT Standardization Data
Step 2. Enter VC, PR, WM, PS
Unexplained
FSIQ
VC,PR,WM,PS
Step 1. Enter FSIQ
Step 2. Enter VC, PR, WM, PS
Predicting Reading Composite with WISC-IV/WIAT-II Standardization Data
Note. FSIQ = Full Scale IQ; VC = Verbal Comprehension, PR = Perceptual Reasoning; WM = Working Memory; PS = Processing Speed.
Incremental Validity Studies: The Regression Magic Show The “IQ Only” World of Joseph Glutting et al. (1997; 2006)
FSIQ =53.2%
FSIQ =53.2%
Factors =10.3%
Factors =1.8%
FSIQ =38.7%
FSIQ =38.7%
Step 1. Enter VC, PR, WM, PS
Predicting Basic Reading with WISC-III/ WIAT Standardization Data
Step 2. Enter FSIQ
Unexplained
FSIQ
VC,PR,WM,PS
Step 1. Enter VC, PR, WM, PS
Step 2. Enter FSIQ
Predicting Reading Composite with WISC-IV/WIAT-II Standardization Data
Note. FSIQ = Full Scale IQ; VC = Verbal Comprehension, PR = Perceptual Reasoning; WM = Working Memory; PS = Processing Speed.
Incremental Validity Studies: The Regression Magic Show WHAT HAPPENED TO IQ???
Now You See It, Now You Don’t!
Factors =55%
FSIQ =0%
Factors =49%
Factors =55%
Factors =49%
FSIQ =0%
Step 1. Enter 10 Subtests
Predicting Reading Composite with WISC-IV/ WIAT-II Standardization Data
Step 2. Enter FSIQ
Unexplained
FSIQ
10 Subtests
Note. FSIQ = Full Scale IQ; 10 Subtests = Similarities, Vocabulary, Comprehension, Digit Span, Letter-Number Sequencing, Block Design, Picture Concepts, Matrix Reasoning, Coding, and Symbol Search
Incremental Validity Studies: The Regression Magic Show Just Say YES to Subtest Interpretation (???)
Subtests =56.6%
Subtests =56.6%
FSIQ =0%
NO, This does not support subtestinterpretation, HierarchicalRegression cannot be used to determine the incrementalvalidity of subtests, factors, or IQbecause they all come from thesame source – the subtests!
Full Scale IQ Commonality Analysis: Unique andShared Variance Among Factor Predictors
IQ/g Perspective Multifactorial Perspective
g
VC
PR PS
WM
g
VC
PR PS
WM
RESEARCH QUESTION: Is FSIQ composed of more shared Factor variance OR unique Factor variance?
More Shared Factor Variance More Unique Factor Variance
Note. VC = Verbal Comprehension, WM = Working Memory; PR = Perceptual Reasoning; PS = Processing Speed
Children with Learning Disabilities (N = 128)
WISC-IV/WIAT-II FSIQ Commonality Analyses
Hale, J. B., Fiorello, C. A., Kavanagh, J. A., Holdnack, J. A., & Aloe, A. M. (2007). Is the demise of IQ interpretationjustified? A response to special issue authors. Applied Neuropsychology, 14, 37-51.Fiorello, C. A., Hale, J. B., Holdnack, J. A., Kavanagh, J. A., Terrell, J., & Long, L. (2007). Interpreting intelligence testresults for children with disabilities: Is global intelligence relevant? Applied Neuropsychology, 14, 2-12.
WISC-IV/WIAT-II FSIQ Commonality AnalysesChildren with Attention-Deficit/Hyperactivity Disorder (N = 71)
Hale, J. B., Fiorello, C. A., Kavanagh, J. A., Holdnack, J. A., & Aloe, A. M. (2007). Is the demise of IQ interpretationjustified? A response to special issue authors. Applied Neuropsychology, 14, 37-51.Fiorello, C. A., Hale, J. B., Holdnack, J. A., Kavanagh, J. A., Terrell, J., & Long, L. (2007). Interpreting intelligence testresults for children with disabilities: Is global intelligence relevant? Applied Neuropsychology, 14, 2-12.
WISC-IV/WIAT-II FSIQ Commonality AnalysesChildren with Traumatic Brain Injury (N = 29)
Hale, J. B., Fiorello, C. A., Kavanagh, J. A., Holdnack, J. A., & Aloe, A. M. (2007). Is the demise of IQ interpretationjustified? A response to special issue authors. Applied Neuropsychology, 14, 37-51.Fiorello, C. A., Hale, J. B., Holdnack, J. A., Kavanagh, J. A., Terrell, J., & Long, L. (2007). Interpreting intelligence testresults for children with disabilities: Is global intelligence relevant? Applied Neuropsychology, 14, 2-12.
Are Subtests, Factors, and IQ Equivalent?They Should Be!!!
Word Pseudoword ReadingReading Decoding Comprehension
Predictor(s) R2 %Change R2 %Change R2 %Change
Standardization Sample
16 Total Subtests .574 -- .424 -- .589 --10 Core Subtests .499 -13% .377 -11% .537 -9%4 Factor Indexes .481 -16% .361 -15% .521 -12%Full Scale IQ .464 -19% .350 -17% .491 -17%
Reading Disability Sample
16 Total Subtests .326 -- .295 -- .481 --10 Core Subtests .294 -10% .254 -14% .449 -7%4 Factor Indexes .230 -29% .173 -41% .378 -21%Full Scale IQ .183 -44% .147 -50% .326 -32%
WIAT-II Reading Variance Accounted for by WISC-IV Predictor Combinations
Fiorello, C. A., Hale, J. B., & Snyder, L. E. (2006). Cognitive hypothesis testing and response to intervention for children with reading disabilities. Psychology in the Schools, 43, 835-854.
Increasing WISC-IV Predictive ValidityWhy Interpretation Must Occur Below Global IQ
Fiorello, C. A., Hale, J. B., & Snyder, L. E. (2006). Cognitive hypothesis testing and response to intervention for children with reading disabilities. Psychology in the Schools, 43, 835-854.
Numerical Operations Math Reasoning Predictor(s) R2 %Change R2 %Change
Standardization Sample
16 Total Subtests .588 -- .679 --10 Core Subtests .475 -19% .587 -14%4 Factor Indexes .448 -24% .569 -16%Full Scale IQ .446 -24% .564 -17%
Math Disability Sample
16 Total Subtests .514 -- .295 --10 Core Subtests .401 -12% .254 -27%4 Factor Indexes .357 -31% .173 -39%Full Scale IQ .280 -46% .147 -41%
WIAT-II Math Variance Accounted for by WISC-IV Predictor Combinations
Are Subtests, Factors, and IQ Equivalent?They Should Be!!!
Hale, J. B., Fiorello, C. A., Miller, J. A., Wenrich, K., Teodori, A. M., & Henzel, J. (2008). WISC-IV assessment and intervention strategiesfor children with specific learning disabilities. In A. Prifitera, D. H. Saklofske, & L. G. Weiss (Eds.), WISC-IV clinical assessment andintervention (2nd ed.) (pp. 109-171). New York, NY: Elsevier.
Increasing WISC-IV Predictive ValidityWhy Interpretation Must Occur Below Global IQ
Math Disability Subtypes
Si Vo Co WR In DS DF DB LNS Ar BD PCs MR PCn Cd SS
11
10
9
8
7
6
5
WISC-IV Subtests
Me
an
Sca
led
Sco
re
Fluid/Quantitative ReasoningMild Executive/Working MemoryRight Hemisphere SLD (NVLD)Numeric-Quantitative Knowledge Dyscalculia-Gerstmann Syndrome
Hale, J. B., Fiorello, C. A., Miller, J. A., Wenrich, K., Teodori, A. M., & Henzel, J. (2008). WISC-IV assessment and intervention strategies for children withspecific learning disabilities. In A. Prifitera, D. H. Saklofske, & L. G. Weiss (Eds.), WISC-IV clinical assessment and intervention (2nd ed.) (pp. 109-171).New York, NY: Elsevier.
The Construct Validity of IQ
IQ/g Perspective:
Flat Profiles
Multifactorial Perspective:
Variable Profiles and Disabilities
g
VC
PR PS
WM
g
VC
PR PS
WM
More Shared Factor Variance More Unique Factor Variance
Note. VC = Verbal Comprehension, WM = Working Memory; PR = Perceptual Reasoning; PS = Processing Speed
Concordance-Discordance Model: Preferred “Third Method Approach” for
Learning Disability Identification
Identifying Children with Learning Disabilities:IDEA (2004) Definitions and Practices
• Learning Disabilities is a disorder of one or more of the basic psychological processes that adversely affects achievement
Defining Learning Disabilities
• Discrepancy between “ability” and “achievement”
• Failure to respond to scientific research-based intervention
• May permit the use of other alternative research-based procedures for determining whether a child has a specific learning disability (§300.8(c)(10) OSERS Final Regulations, August, 2006)
Determining Learning Disabilities
The approach that determines the cause of nonresponseand type of disability!
Pattern of Strengths and WeaknessesModels for SLD Identification
(see Hale, Flanagan, & Naglieri, 2008)
• Significant difference between cognitive strength and academic deficit
• Significant difference between cognitive strength and cognitive deficit
• No significant difference between cognitive deficit and academic deficit, and these should be associated in literature
• Use standard error of the difference formula
Avoids Mark Penalty
Takes ability-achievement and processing into account (meeting IDEA statutory and regulatory requirements)
Empirically-valid approach according to IDEA “Third Method”regulatory requirement
The Concordance-DiscordanceModel of SLD Identification(WIAT-III Uses This Model!)
Processing Strength Composite
Processing Weakness Composite
Achievement DeficitSignificantDifference
SignificantDifference
No SignificantDifference
Source: Hale, J. B., & Fiorello, C. A. (2004). School Neuropsychology: A Practitioner’s Handbook. New York, NY: Guilford Press.
Specific Learning Disability in Math Reasoning?Marci’s WISC-IV/WIAT-III Results
Measure/Subtest SS/ss Measure/Subtest SS/ss________________________________________________________________________________
Global Scores (SS)Verbal Comprehension 99 Perceptual Reasoning 82Working Memory 91 Processing Speed 88
Subtest Scores(ss)Similarities 10 Block Design 7Vocabulary 11 Picture Concepts 8Comprehension 9 Matrix Reasoning 6(Information 11) (Picture Completion 8)
Digit Span 9 Coding 7Letter-Number Sequencing 8 Symbol Search 9
WIAT-III Math Reasoning Subtest SS = 81________________________________________________________________________________Note. Global Scores are reported in SS, Subtest Scores are reported in ss; SS =Standard Score, M = 100, SD = 15; ss = Scaled Score, M = 10, SD = 3.
The Concordance-Discordance Model of LD Identification: Marci’s Results
Processing Strength CompositeVerbal Comprehension SS = 99, rxx = .94
Processing Weakness CompositePerceptual Reasoning SS = 82, rxx = .93
Achievement DeficitMath Reasoning SS = 81, rxx = .92
SignificantDifference
SignificantDifference
No SignificantDifference
The Concordance-Discordance Model of LD Identification: Marci’s Results
Processing Strength CompositeVerbal Comprehension SS = 99, rxx = .94
Processing Weakness CompositePerceptual Reasoning SS = 82, rxx = .93
Achievement DeficitMath Reasoning SS = 81, rxx = .92
SED = SD 2 – rxx – ryy
SED = 15 2 – .94 – .92
SED = (5.61) X 2.58 = 14.48 15
VC SS = 99; MR SS = 81, so 99–81 = 18Obtained Diff (18) > Critical Diff (15)?
YES! Significant Difference
The Concordance-Discordance Model of LD Identification: Marci’s Results
Processing Strength CompositeVerbal Comprehension SS = 99, rxx = .94
Processing Weakness CompositePerceptual Reasoning SS = 82, rxx = .93
Achievement DeficitMath Reasoning SS = 81, rxx = .92
SignificantDifference?YES
SED = SD 2 – rxx – ryy
SED = 15 2 – .94 – .93
SED = (5.41) X 2.58 = 13.95 14
VC SS = 99; PR SS = 82, so 99 – 82 = 17Obtained Diff (17) > Critical Diff (14)? YES! Significant Difference
The Concordance-Discordance Model of LD Identification: Does Marci Meet Criteria for SLD?
Processing Strength CompositeVerbal Comprehension SS = 99, rxx = .94
Processing Weakness CompositePerceptual Reasoning SS = 82, rxx = .93
Achievement DeficitMath Reasoning SS = 81, rxx = .92
SignificantDifference
SignificantDifference
No SignificantDifference
• Discordance between cognitive strength and cognitive weakness? YES, deficit in the basic psychological processes established
• Concordance between cognitive deficit and academic deficit? YES, processing deficit likely related to achievement deficit
• Discordance between cognitive strength and academic deficit? YES, unlikely the cause of achievement deficit is the cognitive strength, so discordance suggests achievement deficit is NOT due to low global ability
Step Clinical Objective Clinical Question/Decision Rules
1 Score standardized cognitive test and determine whether global composite score (e.g., IQ), factor scores, or subtest scores should be interpreted.
1a. Are all subtest scores consistent enough to interpret global composite score (e.g., IQ)?YES, C-DM unlikely, probably not SLD; discontinue or consider other possible measure of processing deficits. NO, C-DM possible; go to Step 1b.
1b. If not consistent across the entire test, are the subtest scores consistent within factors to interpret factor scores?YES, C-DM possible; go to Step 2.NO, Consider subtest combinations to form new factor score within cognitive measure, go to Step 1c.
1c. If no subtest combinations appear to represent a new factor, can other standardized measures be added to cognitive measure to create new factor score?YES, new subtest combination appropriate for use in C-DM model, go to Step 2. NO, consider combining subsets from additional measure of at least two subtests to create new factor score for use in C-DM analyses, go to Step 2
Steps in Concordance-Discordance Model of LD Identification
Step Clinical Objective Clinical Question/Decision Rules2 Score standardized achievement test and examine to see
if composites or subsets indicate achievement deficit
2a. Do standardized achievement scores indicate an
academic deficit that is consistent with prior evaluation
(e.g., nonresponse to intervention), classroom
permanent products, and teacher reported
achievement deficits?
YES, C-DM possible, go to Step 3.
NO, Explore other possible causes for poor test
performance, or explanations for poor performance in
the classroom; consider achievement retesting to
verify/refute achievement deficit; return to Step 2 or
discontinue.
3 Review cognitive (e.g., CHC) and/or neuropsychological
literature to ensure obtained cognitive deficit(s) is
associated with achievement deficit(s)
3a. Could obtained cognitive deficits interfere with
deficient academic achievement area?
YES, cognitive and/or neuropsychological deficits
have been found to be related to deficit achievement
area in the literature, go to Step 4
NO, C-DM unlikely unless research not conducted,
check for ecological validity of cognitive and
achievement deficits; return to Step 2 or discontinue.
4 Obtain reliability coefficients for cognitive strengths,
cognitive deficit(s) and achievement deficit(s)
4a. Are factor/subtest reliability coefficients (e.g.,
coefficient alpha) available in the cognitive and
achievement technical manuals?
YES, factor strengths and deficits, and achievement
score reliabilities are in the manuals; go to Step 5.
NO, new factor scores and reliability coefficients
must be computed; average factor scores and reliability
coefficients for new factors (use Fisher’s z-
transformation for reliabilities, see Hale et al., 2008);
go to Step 5.
Step Clinical Objective Clinical Question/Decision Rules5 Calculate standard error of the difference (SED) formula to
establish discordance between cognitive strength and cognitive
deficit
5a. Enter reliability coefficients for cognitive strength and
deficit into SED formula, and solve for SED:
5b. Multiple obtained SED value by 1.96 for p < .05, or 2.58
for p < .01.
5c. Is obtained difference between cognitive strength and
deficit greater than SED critical value?
YES, there is a significant difference between cognitive
strength and deficit, child likely has a deficit in the basic
psychological processes that is interfering with academic
achievement; go to Step 6.
NO, consider other possible cognitive deficit responsible
for achievement deficit, go to Step 1; or the child may have
another disability interfering with achievement, consider
further evaluation; or the child does not have a SLD, try to
serve in intensive response-to-intervention model.
6 Calculate SED formula to establish discordance between
cognitive strength and achievement deficit
6a. Enter reliability coefficients for cognitive strength and
academic deficit into SED formula, and solve for SED:
5b. Multiple obtained SED value by 1.96 for p < .05, or 2.58
for p < .01.
5c. Is obtained difference between cognitive strength and
academic deficit greater than SED critical value?
YES, there is a significant difference between cognitive
strength and deficit, child likely has unexpected
underachievement consistent with a specific learning disability;
go to Step 7.
NO, consider other possible cognitive deficit and/or
achievement deficit, go to Step 1; or the child may have
another disability interfering with achievement, consider
further evaluation; or discontinue, the child does not have a
SLD, try to serve in intensive response-to-intervention model.
Step Clinical Objective Clinical Question/Decision Rules
7 Calculate SED formula to establish concordance between
cognitive deficit and achievement deficit
6a. Enter reliability coefficients for cognitive deficit and
academic deficit into SED formula, and solve for SED:
5b. Multiple obtained SED value by 1.96 for p < .05, or 2.58
for p < .01.
5c. Is obtained difference between cognitive strength and
academic deficit less than SED critical value?
YES, there is no significant difference between cognitive
deficit and the achievement deficit, cognitive deficit plausible
cause for achievement deficit, consider team determination of
specific learning disability classification; begin individualized
instruction in inclusive or more restrictive environment as
necessary; go to Step 8.
NO, is the achievement deficit significantly below the
cognitive deficit? If so, this could mean other factors are
causing additional impairment, consider for specific learning
disability classification and individualized service delivery, and
additional evaluation to determine why achievement deficit is
substantial; go to Step 8.
NO, is the achievement deficit significantly above the
cognitive deficit? If so, this could mean the child is using a
compensatory strategy to score better on the academic
measure, determine if results still warrant specific learning
disability classification and/or individualized service delivery;
go to Step 8.
8 Determine if C-DM findings have ecological validity and achieve
team consensus for SLD or other disorder determination.
Re-examine empirical literature, RTI data, teacher reports,
classroom permanent products, classroom observations, and
other evaluation data (including C-DM results) to determine if
child meets IDEA statutory and regulatory requirements of
SLD or other disorder warranting special education services;
consider within the context of other team evaluation data;
consider SLD classification and service delivery in least
restrictive environment.
Understanding Psychological Processes for Effective Application of the
Concordance-Discordance Model
Cognitive Product or Process for Interpretation of Assessment Data: Synthesizing Paradigms
• Compares individual to norm
• Focuses on score: Assessment Product
• Interpret levels of performance
• Compares individual to self
• Focuses on performance: Assessment Process
• Interpret patterns of performance
Nomothetic Assessment
Idiographic Assessment
BOTH essential for effective assessment and intervention
Developing and Testing HypothesesAbout Cognitive Processes and Achievement
• Examine profile for significant subtest or factor scatter
• If variable profile, consider Cattell-Horn-Carroll (CHC) factor interpretation
• If global scores are invalid, interpret subtest performance within and between subtests using process approach
• Use Demands Analysis to examine input, processing, and output demands
(Caution: Processing demands more relevant)
• Demands Analysis is prone to clinician error or misinterpretation, must validate findings with additional measures
(Caution: Avoid cookbook interpretation)
• Data must show convergent, divergent, AND ecological validity
Input Demands:
• Models and abstract visual pictures
• Lengthy oral directions-receptive language
• Demonstration and modeling
• Perception of timed task
• Low cultural knowledge and emotional content
Demands Analysis: WISC-IV Block DesignDetermining InputProcessingOutput Demands
Processing Demands:
• Novel task for most children (note beginning vs. later task performance-also when lines removed)
• Visual processing: Spatial-global
• Visual processing: Directional orientation-local
• Perceptual analysis (divergent thought) and synthesis (convergent thought)
• Planning, strategizing, and monitoring performance: Match to sample
• Sustain attention, inhibit impulsive/error responding
Demands Analysis: WISC-IV Block DesignDetermining InputProcessingOutput Demands
Demands Analysis: WISC-IV Block DesignDetermining InputProcessingOutput Demands
Output Demands:
• Fine motor response, arrange manipulatives
• Bimanual sensory (visual-somatosensory) –motor coordination
• Processing speed
Demands Analysis: WJ-III Visual-Auditory LearningDetermining InputProcessingOutput Demands
EXAMINER: “Each of these drawings is a word. As soon as I tell you what a drawing says, you say it back to me”
“Woman” “Cat” “And”
EXAMINER: “What does this say?”
CORRECT CHILD RESPONSE: “Woman and cat.”
Demands Analysis: WJ-III Visual-Auditory LearningDetermining InputProcessingOutput Demands
Input Demands:
• Brief oral directions
• Modeling and demonstration
• Semiabstract figures/symbols
• Low cultural and language knowledge
Demands Analysis: WJ-III Visual-Auditory LearningDetermining InputProcessingOutput Demands
Processing Demands:
• Visual perception of figures/symbols
• Sound-word/symbol-rebus association
• Working memory: Encoding and retrieval from memory
• Ability to benefit from feedback
• Inhibition of error responses
• Sequencing: Syntax knowledge
• Attention and executive demands: Moderate
• Memory: Associative/lexical-semantic
• Fluid Reasoning: Secondary
Demands Analysis: WJ-III Visual-Auditory LearningDetermining InputProcessingOutput Demands
Output Demands:
• Oral language formulation
• Brief oral response
• Articulation
Demands Analysis: WISC-IV InformationDetermining InputProcessingOutput Demands
Input Demands:
Output Demands:
Processing Demands:
DEMANDS ANALYSIS
James B. Hale and Catherine A. Fiorello
Student’s Name: _____________________________ Age: ______ Grade:
______
Test/Subtest _________________________
INPUT (Circle All That Apply) PROCESS(Circle All That Apply) OUTPUT (Circle All That Apply)
Instructions
Demonstration/Modeling
Gesture/Pantomime
Brief Oral Directions
Lengthy Oral Directions
Timing
Overall Time Limit
Speed Bonus
Teaching
Sample Item
Teaching Item(s)
Dynamic Assessment
Feedback When Correct
Querying
Stimulus
Visual
Pictures/Photos
Abstract Figures
Models
Symbols (letters, numbers)
Written Language
Large/Small
Color Important
Auditory
Brief Verbal
Lengthy Verbal
Spoken
Tape/CD (Headphones used? Y N)
Background Noise
Content
L M H Cultural Knowledge
L M H English Language Knowledge
L M H Emotional Content
Left Hemisphere
Concordant/Convergent (“Explicit”)
Right Hemisphere
Discordant/Divergent (“Implicit”)
Executive Functions (Frontal-Subcortical Circuits)
Sustained Attention/Concentration
Inhibit/Impulsive
Working Memory (Specify _______________)
Flexibility/Modify/Shift Set
Performance Monitoring/Benefit from Feedback
Planning/Organization/Strategy Use
Memory Encoding/Retrieval
Novel Problem Solving/Reasoning
Temporal Relationships/Sequential Processing
Expressive Language (L R)
Neuropsychology Functional Domains
Sensory Attention (T O P) (L R)
Primary Zones (T O P) (L R)
Secondary/Tertiary Zones (T O P) (L R)
Prior Learning/Long-Term Memory
Sensory-Motor Coordination
Multimodal Integration
Dorsal Stream (Occipital-Parietal)
Ventral Stream (Occipital-Temporal)
Receptive Language (L R)
CHC Abilities and Narrow Abilities
Higher-Level Processing
Gf–Fluid Reasoning _____
Glr–LT Storage & Retrieval _____
Gv–Visual Processing _____
Ga–Auditory Processing _____
Lower-Level Processing
Gs–Processing Speed _____
Gsm–ST Memory _____
Acquired Knowledge & Achievement
Gc–Crystallized Intelligence _____
Oral
Brief Oral
Lengthy Oral
Report of Strategy Use
Motor
Fine Motor—Point
Fine Motor—Graphomotor
Fine Motor—Manipulatives (e.g., blocks, pictures)
Visual-Sensory-Motor Integration
Gross Motor
Written Language
Brief Written Response
Lengthy Written Response
Response Format
Open/Free Response
Constrained/Multiple Choice
Other
Input:________________________________________
_____________________________________________
___________________________
Processing:____________________________________
_____________________________________________
___________________________
Output:_______________________________________
_____________________________________________
___________________________
Comments:____________________________________
_____________________________________________
_____________________________________________
_____________________
The Cognitive Hypothesis Testing Model
Source: Hale, J. B., & Fiorello, C. A. (2004). School Neuropsychology: A Practitioner’s Handbook. New York, NY: Guilford Press.
Theory
Hypothesis
Data Collection
Interpretation
1. Presenting Problem
2.Intellectual/Cognitive Problem
3. Administer/Score Intelligence Test
4. Interpret IQ or Demands Analysis
5. Cognitive Strengths/Weaknesses
6. Choose Related Construct Test
7. Administer/Score Related Construct Test
8. Interpret Constructs/Compare
9. Intervention Consultation
10. Choose Plausible Intervention
11. Collect Objective Intervention Data
12. Determine Intervention Efficacy
13. Continue/Terminate/Modify
Comprehensive Evaluation for Special Education Determination and Service Delivery
Traditional NomotheticSchool Psychology Assessment
Innovative Idiographic School Neuropsychological Evaluation
Best Practices in Psychological Evaluation
Relevance of Concordance-Discordance Model For Specific Learning Disability
Identification and Intervention
Cognitive and Achievement Profiles for No SLD Group
SI VO CO IN BD PCS MR PC DS LNS AR CD SS CA
10
9
8
7
CognItive Subtests
Me
an
Sca
led
Sco
re
XX
X XX
X
X
X
X
XX
X X
X
WR RF RC WA SP WF WE MC MF MWP
100
90
80
70
Achievement Subtests
Me
an
Sta
nd
ard
Sco
re
XX
XX
X X
X
X
X
X
Cognitive and Achievement Profiles for Left Hemisphere Subtype
SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA
10
9
8
7
CognItive SubtestsM
ea
n S
cale
d S
core
WR RF RC WA SP WF WE MC MF MWP
100
90
80
70
Achievement Subtests
Me
an
Sta
nd
ard
Sco
re
11
Cognitive and Achievement Profiles for Right Hemisphere SLD Subtype
SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA
10
9
8
7
CognItive SubtestsM
ea
n S
cale
d S
core
WR RF RC WA SP WF WE MC MF MWP
100
90
80
70
Achievement Subtests
Me
an
Sta
nd
ard
Sco
re
11
Cognitive and Achievement Profiles for Working Memory SLD Subtype
SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA
10
9
8
7
CognItive SubtestsM
ea
n S
cale
d S
core
WR RF RC WA SP WF WE MC MF MWP
100
90
80
70
Achievement Subtests
Me
an
Sta
nd
ard
Sco
re
11
Cognitive and Achievement Profiles for Processing Speed SLD Subtype
SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA
10
9
8
7
CognItive SubtestsM
ea
n S
cale
d S
core
WR RF RC WA SP WF WE MC MF MWP
100
90
80
70
Achievement Subtests
Me
an
Sta
nd
ard
Sco
re
11
Cognitive and Achievement Profiles for Executive SLD Subtype
SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA
10
9
8
7
CognItive SubtestsM
ea
n S
cale
d S
core
WR RF RC WA SP WF WE MC MF MWP
100
90
80
70
Achievement Subtests
Me
an
Sta
nd
ard
Sco
re
11
Cognitive and Achievement Profiles for SLD Subtypes and No SLD Group
SI VO CO IN BD PCS MR PC DS DF DB LNS AR CD SS CA
10
9
8
7
CognItive SubtestsM
ea
n S
cale
d S
core
WR RF RC WA SP WF WE MC MF MWP
100
90
80
70
Achievement Subtests
Me
an
Sta
nd
ard
Sco
re
11
XX
XX
X X
X
X
X
X
XX
X XX
X
X
X
X
XX
X X
X
Left Hemisphere SLDRight Hemisphere SLDWorking Memory SLDProcessing Speed SLDExecutive SLD
X No C-DM SLD
Predictor
Ability-Achievement
Discrepancy
Concordance-
Discordance Model WMI
Weakness
Subtype
PSI
Weakness
Subtype
Executive
Weakness
SubtypeNo Yes No Yes
R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss
Subtests
Factors
FSIQ
.41
.25
.21
-
-40%
-50%
.20
.10
.08
-
-50%
-60%
.35
.16
.13
-
-54%
-63%
.18
.09
.02
-
-50%
-89%
.65
.27
.05
-
-58%
-92%
.41
.17
.04
-
-59%
-90%
.49
.29
.07
-
-41%
-86%
WISC-IV Predictors of WJ-III Letter-Word Identification for SLD Identification Methods and Processing Subtypes
Note. WMI = Working Memory Index; PSI = Processing Speed Weakness; FSIQ = Full Scale Intelligence Quotient.
WISC-IV Predictors of WJ-III Calculation for SLDSLD Identification Methods and Processing Subtypes
Note. WMI = Working Memory Index; PSI = Processing Speed Weakness; FSIQ = Full Scale Intelligence Quotient.
Predictor
Ability-Achievement
Discrepancy
Concordance-
Discordance Model WMI
Weakness
Subtype
PSI
Weakness
Subtype
Executive
Weakness
SubtypeNo Yes No Yes
R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss R2 Loss
Subtests
Factors
FSIQ
.23
.17
.13
-
-26%
-43%
.28
.20
.15
-
-29%
-46%
.47
.24
.18
-
-49%
-62%
.14
.12
.11
-
-14%
-21%
.54
.22
.04
-
-59%
-93%
.26
.20
.12
-
-23%
-54%
.64
.20
.14
-
-69%
-78%
The Cognitive Hypothesis Testing Model
Source: Hale, J. B., & Fiorello, C. A. (2004). School Neuropsychology: A Practitioner’s Handbook. New York, NY: Guilford Press.
Theory
Hypothesis
Data Collection
Interpretation
1. Presenting Problem
2.Intellectual/Cognitive Problem
3. Administer/Score Intelligence Test
4. Interpret IQ or Demands Analysis
5. Cognitive Strengths/Weaknesses
6. Choose Related Construct Test
7. Administer/Score Related Construct Test
8. Interpret Constructs/Compare
9. Intervention Consultation
10. Choose Plausible Intervention
11. Collect Objective Intervention Data
12. Determine Intervention Efficacy
13. Continue/Terminate/Modify
Tier 3Comprehensive Evaluation for Special Education
Determination and Service Delivery
Linking Assessment to Intervention:The Problem-Solving Model Revisited
• Problem Identification - Referral and/or other problems
• Problem Analysis - Includes cognition, behavior, and environment
• Plan Development and Implementation -The best intervention is one that works!
• Plan Evaluation and Recycling - Progress monitoring required, and recycling is common!
Choosing Tier 3 Single Subject Designs,Progress Monitoring Strategies, and Goals
• Brainstorm multiple single subject designs for the target behaviour
• Brainstorm measurement tools and collect baseline data if not previously collected
• Decide on the criterion for acceptable performance based on benchmarks, performance discrepancy, and timeline
• Determine resources necessary to carry out intervention with integrity
• Encourage consultee (e.g., teacher, parent) ownership of intervention
• Monitor regularly with casual and formal contacts to ensure intervention implemented with integrity
Remediation or Compensation of Academic Weaknesses?
Source: Hale, J. B., & Fiorello, C. A. (2004). School Neuropsychology: A Practitioner’s Handbook. New York, NY: Guilford.
Younger Older
Severe
Mild
Ty
pe
of
Imp
airm
ent
Age
Compensate
Remediate
0
5
10
15
20
25
30
35
40
45
1 2 3 4 5 6 7 8 9 10 11 12 13
Day
Fran's CBM Chart
Baseline
Aimline
Trendline
Repeated Reading Instruction
Choosing Tier 3 Single Subject DesignsW
ord
s R
ead
Per
Min
ute
79
87
76
93
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
70
75
80
85
90
95
100
Pretest Posttest
Lett
er
Sounds
Know
n
Phonem
ic A
ware
ness
SS
Sam's Auditory Processing and Letter-Sound Knowledge
Pre-Post Design
CTOPP Phonemic Awareness
Letter Sounds Known
Choosing Tier 3 Single Subject Designs
Sarah’s Math Computation: ABAC Design
100
95
90
85
80
75
70
Per
cen
t C
orr
ect
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Baseline (A) Math Fact Flashcards Baseline (A) Flashcards+Self-Monitoring
Day
Choosing Tier 3 Single Subject Designs
Oscar’s ADHD Medication Response:ABC Design
4.0
3.0
2.0
0.0
1.0
Baseline Placebo 5mg MPH 10mg MPH
Note. Lower Ranks = Better Performance and Behavior; Order of Conditions = Baseline, 10mg MPH, Placebo, 5mg MPH
Cognitive Response(Fr = 32.62, p = .001; BP > LH)
Behavioral Response(Fr = 32.28, p = .0001; B>P,L,H; P>LH)
Conclusions• Educators and school psychologists need a better understanding
of how the brain affects learning and behavior in the classroom
• IQ is largely useless for SLD identification, and minimizes unique characteristics of diverse population
• Nomothetic interpretation of tests for identification purposes does not work well for differential diagnosis, and it does not inform individualized interventions
• “Test and place” focus reduces likelihood of needed early identification and service delivery, leads to too many false positives and false negatives, and decreases likelihood assessments can be related to intervention
• Although a multi-tier RTI approach is needed (and should be mandated!) and can inform intervention intensity, it is not sufficient for SLD identification, and does not inform individualized interventions
• Combining standardized RTI, problem-solving RTI, and CHT methods can guide instruction and ensure efficacious outcomes for all children, regardless of disability status
Conclusions Continued• For Tier 1 and Tier 2 nonresponders , concordance-discordance
model ensures children meet IDEA statutory and regulatory SLD requirements
• Effective school neuropsychological interpretation focuses on psychological processes (i.e., idiographic interpretation), but tests hypotheses with additional measures to ensure accuracy
• Differential diagnosis is only one component of comprehensive CHT evaluations; the real utility of the approach is guiding classroom interventions within the context of a problem-solving model to ensure ecological and treatment validity of findings
• Understanding brain-behavior relationships during psychological testing and classroom instruction is the future of classroom education!