Diagnosing the Learning Enabled: New Ways of Thinking About

Assessment, Identification and Intervention – And Why We’re

Thinkin’ That Way

Rhode Island Technical Assistance Project Summer Institute ‘03

W. David Tilly III, Ph.D.Heartland Area Education Agency 11

July 24 and 25, 2003

Correspondence about this presentation should be directed to David Tilly, Heartland AEA 11, 6500 Corporate Dr., Johnston, IA 50131. Email is dtilly@aea11.k12.ia.us, (515) 270-9030.

Where is Iowa?

IOWA

Who am I?

Like ME

I was born in Rhode Island I am a parent I’ve got tomatoes in my garden this year I have gone to a National Conference in the

past 5 years I have lived outside of the United States I am a special educator

Like ME

I have worked in education for: 20 years or more 15 years or more 10 years or more 5 years or more Less than 5 years

Take a Couple Minutes

Introduce yourself at your table Who are you? Where are you

from? What do you do in

your school?

Parking Lots

Big Picture Objectives

Updates on what’s going on Nationally and in RI in education

Some background on some of “Why” it’s going on Look at the big themes, processes, implementations

and outcomes Take some time to process this and to contemplate

possibilities. That is, I’m going to ask you to talk with eachother about the stuff I’m presenting

Whatchagonnadowhenyougohome?

Morning Advance Organizer

Discuss where educational assessment is going and for all students with a focus on helping students with learning challenges

To review some of the rationale behind reforming our assessment/identification systems in schools

To propose a more rational and helpful alternative system consistent with NCLB

To provide illustrations and data on the effects of successful implementation of that model

One More Objective for The Morning

To encourage you think

“Outside of the Box”

Current Realities: We Can View Them As Obstacles or Opportunities

Major Reform Federal Law

Changes Few Precedents

Leading to the Future

High Stakes Ready or not, Here

We Come

Big Picture Context (NCLB) We can’t ignore it We know enough to get

started We have enough

guidance too know some things about our near term future We know we have till

’13-’14 to get ALL students proficient in Reading and Math

Big Picture Context (NCLB) Activity 1

Question: What percentage of students in the buildings you work in are proficient in mathematics and reading?

Big Picture Context (NCLB) The accountability provisions in NCLB, with

subgroup analyses, can provide over a hundred opportunities a year to be designated a School in Need of Improvement (SINI)

Schools need to miss only one for SINI designation

Big Picture Context for Kids With Learning Problems

We know with the collateral changes in IDEA all means every, and no students can be exempted from the accountability system

The stakes are really high Both the House and Senate

Bills have some REALLY INTERESTING STUFF IN them!!!!

It Can No Longer Be Business As Usual

So Where Do We Start?

Richard Feynman has said:

“The best way to predict the future is to invent it”

It is Also True That...

If we don’t learn from the past, we’ll repeat it

One Perspective on History

Our education system has grown up through a process of “Disjointed Incrementalism” (Reynolds, 1988)

The currentEducationSystem’sProgrammaticEvolution

K-12 Education

Gifted

Title 1

SPED

Migrant

ELLAt Risk

Unintended Effects Conflicting programs Conflicting funding streams Redundacy Lack of coordination across

programs Nonsensical rules about

program availability for students

Extreme complexity in administration and implementation of the programs

Intensive Interventions 5%

Strategic Interventions 15%

Core Curriculum 80%

Adapted from: Sugai and Horner

We Have Got To Get More Systematic and Simplify – Especially in High Stakes Areas (RMS)

Students

School Curricula – Pick an area

We’

ll Com

e Bac

k to

Thi

s

This Sounds Good, But….

Our hands are tied Federal law prescribes

lots of how we’re organized

Especially in Special Education, there are lots of things we have to do

How can we get them done along with the new requirements?

SPED Assessment Survey ‘03

Activity #2: Complete Survey

Forces Underlying Our Assessment Systems For Kids

Ethical Reasons Legal Reasons Professional

Reasons Socio-Political

Realities

Ethical Reasons

Why Did You Become An Educator?

Legal Reasons: The Purpose of IDEA ‘97

To ensure that all children with disabilities have available to them FAPE...

To ensure that the rights of children with disabilities…

To Assist States $$… To ensure parents and

teachers have tools to improve results...

Legal Reasons: The Purpose of IDEA ‘97

To assess and ensure the effectiveness of efforts to educate children with disabilities

Professional Reasons: Professional Judgment

Determines how we carry out and meet our legal and inspirational purposes.AssumptionsPractices

In the Beginning

Assessment System for Students With Learning Problems Was Based On A Series of Assumptions

Activity #3: Name the Assumptions

Think about your experience.

What are one or more assumption about kids, teaching and/or learning problems inherent in how our special education system was defined.

If We Assume

Assumption 1: Available educational assessment procedures are sufficient for reliable and valid differential diagnosis.

If We Assume

Assumption 2: Thorough understanding of the intrapersonal (within person) cause of educational disabilities is the most critical factor in determining appropriate treatment.

Studentswith LD

LD Reading Methods

MR Reading Methods

If We Assume

Assumption 3: Persons within disability categories have similar educational needs that are different in educationally important ways from persons in other disability categories.

Studentswith MR

Studentswith EBD

EBD Reading Methods

If We Assume

Assumption 4: Matching treatments to disability type (or underlying characteristics) will result in maximally effective interventions.

Learning Disability X LD Methods = Effective Tx.

Auditory Learner X Auditory Instruction = Effective Tx.

Sequential Processor X Sequential Instr. = Effective Tx.

These Were The Assumptions

The year was 1975 Based on the best

information we had at the time

The best research of the time was considered

Logical and Rational System Structure If these assumptions are true,

then, from the standpoint of meeting our professional and legal purposes

The historical system is structured appropriately to meet our purposes

Nationally-normed, standardized tests are all we need to meet our purposes.

Effectiveness of service delivery could be determined by examining how many children we are helping.

The System Worked

Child find Children were placed in

special programs Services were delivered An ever increasing

number of professionals were involved

We got really efficient at the process!

Until… Activity #4

Turn To Activity Page Individually write down

some of the challenges (practical, professional, ethical, and/or political) to special education that you have experienced in Rhode Island throughout the past 10 years….

Until... Increases in SPED incidence (particularly in SLD and

recently in OHI) Increases in English Language Learners National Academy of Science Reports (1984, 1996,

2002) Inclusion Undocumented Effectiveness Fordham Foundation Report “Rethinking Special

Education for the New Millennium” National Movement Toward Better Educational Results

(e.g., Nation at Risk leading to Standards-Based Reform)

Until...

Increasing 504 Awareness Increased Poverty The Americans with Disabilities Act Judicial Findings Generally Supporting

Parents’ Wishes in Placement (e.g., more inclusive environments)

All Converging on IDEA ’97

IDEA ’97’s Contributions

Functional and Developmental Assessment in all assessment domains

Increased parental involvement

Inclusion of all kids in district and state assessments

FBA and Behavioral interventions

General Education Curriculum

I could go on….

No Child Left Behind Sweeping changes to

ESEA Increased complexity

(1100 pages of it) Increased accountability Increased rewards and

sanctions Increased prescriptiveness

(esp. in Reading) Increased linkages with

Special Education

President’s Commission Report: A New Era Launch the “antiquated wait

to fail” model SPED kids are GenEd. Kids

first! Empower parents Emphasize results over

compliance Use better approaches to

identifying kids with disabilities

Prepare teachers better SPED research needs

enhanced rigor

And Pending IDEA Reauthorization

Some highlights from the House Bill Adds language saying a

child will not be determined to have a disability if the determinate factor is lack of scientifically based instructional practices and programs that contain the essential components of reading as defined in ESEA

Pending IDEA Reauthorization – From the House Bill‘‘(6) SPECIFIC LEARNING DISABILITIES.—‘‘(A) IN GENERAL.—Notwithstanding sec- 13tion 607 of this Act, or any other provision of 14law, when determining whether a child has a 15specific learning disability as defined under this 16Act, the local educational agency shall not be 17required to take into consideration whether the 18child has a severe discrepancy between achieve- 19ment and intellectual ability in oral expression, 20listening comprehension, written expression, 21basic reading skill, reading comprehension, 22mathematical calculation, or mathematical rea- 23soning. 24

Pending IDEA Reauthorization Adds a provision allowing

LEAs to use a process that determines if a child responds to scientific, research-based intervention when determining whether a child has a specific learning disability.

Gets rid of benchmarks or short term objectives

Adds “academic” to functional and developmental information that must be collected

Pending IDEA Reauthorization Adds “prereferal services”

which permits an LEA to use up to 15 percent of its Part B funds for students who have not been identified as needing special education and related services but who need additional academic and behavioral support to succeed in general education.

Massive Part D changes

Should we change the way we do business? - Is no longer a question.

We're All Looking for New Ways to do Things,

But what are the possibilities?

How can we make a good system better?

Should We Change the Way We Do Business?

We're all looking for new ways to do things, but how do we do this within the context of NCLB?Legal Standards (shifting)Professional Knowledge (evolving)

Federal Law and Regs are Prescriptive About Few Things in Assessment and Intervention §300.532.

Administered in child's native language

Validated for specific purpose for which they are used

Administered by trained personnel in conformance with the instructions provided by their producer

Federal Law and Regs are Prescriptive About Few Things in Assessment and Intervention §300.532.

No single procedure may be used for determining an appropriate education

The evaluation must be conducted by a multidisciplinary team

The child must be assessed in all areas related to the suspected disability

Federal Law and Regs are Prescriptive About Few Things in Assessment and Intervention §300.532.

A variety of assessment tools and strategies are used to gather relevant functional and developmental information about the child...

That may assist in determining the content of the child’s IEP, including information related to enabling the child to be involved in and progress in the general curriculum

IMPORTANT POINT

There is tremendous flexibility within IDEA

One of Iowa’s greatest learnings as a state was that “we did it to ourselves”

That is, most of the restrictions we perceived as barriers to changing what we were doing – they were self imposed by our state’s interpretation of the Federal Law and Regulations

Professionally, we now have many

years experience implementing our

assessment system

Our Professional Obligation

Review practice and assumptions related to accomplishing our assessment purposes

We need to celebrate many positive outcomes Nearly 1 million previously excluded - included Services are provided Rights of children and their families are protected

Professionally, after 27 years we know

Assumption 1: Available educational assessment procedures are sufficient for reliable and valid differential diagnosis.

Many assessment devices used for differential diagnosis are not reliable and valid enough for use with individuals (e.g., Salvia and Ysseldyke, 1991; Witt, 1986).

Professionally, after 27 years we know

Assumption 2: Thorough understanding of the intrapersonal (within person) cause of educational disabilities is the most critical factor in determining appropriate treatment

Educational disability results from a complex interaction between curriculum, instruction, the environment and learner characteristics (e.g., Howell, 1993)

The Environment

Learner Instruction

Curriculum

Professionally, after 27 years we know

Assumption 3: Persons within disability categories have similar educational needs that are different in educationally important ways from persons in other disability categories.

Educational needs vary widely within and across disability category (e.g., Jenkins, Pious, & Peterson, 1988; Marston, 1987).

Studentswith LD

LD Reading MethodsMR Reading

Methods

Studentswith MRStudents

with EBD

EBD Reading Methods

Professionally, after 27 years we know

Assumption 4: Matching treatments to disability type (or underlying child characteristics) will result in maximally effective interventions.

Aptitude-by-treatment interactions (ATIs) have not been proven (e.g., Arter & Jenkins, 1979; Cronbach, 1975; Good, et al., 1993; Teeter, 1987, 1989; Ysseldyke & Mirkin, 1982).

The Reality

The effectiveness of any educational strategy for an individual can only be determined through its implementation.

In Short: We Need A Different Assessment System

We need a system: For identifying problems

more specifically and earlier That allows for a broader

range of explanations of why problems are occurring

Emphasizes assessment for Problem ID, Problem Analysis, Treatment Planning

and; Evaluating whether the interventions are effective

In Short: We NeedA School-

Wide Problem-Solving System

Quote

In the main, the bureaucratic structure of the workplace is more influential in determining what professionals do than are personal abilities, professional training, or previous experience. Therefore, change efforts should focus on the structure of the workplace, not on the teachers

Frymier, J. (1987, September). Bureaucracy and the neutering of teachers. Phi Delta Kappan, p. 10.

So How Do We Get There?

We need to create a new box, outside of our historical paradigm?

Let’s Examine the Parameters

Reexamine the assessment survey.

1. The primary purpose of special education assessment is eligibility determination for services. False

IDEA ‘97 discusses assessments for the many purposes including: to determine whether a child is a child with a disability to determine educational needs of such a child to measure a child’s progress toward annual goals to communicate with a child’s parents about

their children’s progress toward the annual goalsthe extent to which that progress is sufficient to enable

the child to achieve the annual goals by the end of the year

2. Individually administered IQ tests are not required by federal law for any special education disability category. True

IDEA ‘97 does not refer to IQ tests anywhere (nor any type of test)

There are places where the federal statute and regulations use terms such as "Cognitive Factors"614(b)(2)(C) and "Intellectual Functioning" 34 CFR 300.7(b)(6)

There are ways to make inferences about these constructs without using IQ tests.

3. It is a requirement of Federal law that disability labels be used to individually identify students

with disabilities. False

3 administrations made this position clear

At least 2 states are "noncategorical"

Iowa offers the option to AEAs - the Feds have OK'd this

3. It is a requirement of Federal law that disability labels be used to individually identify students

with disabilities. False

612(a)(3)(B) Nothing in this Act requires that children be classified by their disability so long as each child who has a disability listed in section 602 and who by reason of that disability, needs special education and related services is regarded as a child with a disability under this part.

4. Federal regulations do not require that standardized tests be used to identify students with disabilities. True

Federal language reads "Tests and other evaluation materials" 34 CFR 300§532a

According to Judy Heumann, "Evaluations under Part B can be accomplished by testing or by means other than testing..."

Federal regulations provide flexibility in the data sources that can be used to inform professional decisionmaking

5. It is a due process violation for support staff to collect any student-specific assessment data prior to receiving parental permission. True and False

Two issues: How professionals are funded (Incidental

Benefit rules) - No longer True!! Whether it is the policy and procedures of the

agency to make these types of services available to all general education children

Purpose of the evaluation is the key to determining the acceptability of an individual evaluation

6. All children with any of the 13 disabilities identified in IDEA ‘97 are entitled to special education. False

IDEA Entitlement = Disability + Need

This was one component of the landmark Rowley (1982) case decided by the US Supreme Court

Children with disabilities who do not need services to receive a FAPE are not entitled to receive unneeded services.

7. Data collected for 504 evaluations, Title I evaluations or “at risk” evaluations can and should be used if available, when conducting full and individual special education evaluations.

True

All information available on student performance should be considered.

The consent for evaluation is giving consent to consider entitlement for special education

Rationale to IDEA ‘97 Regs “... the amendments permit initial evaluations to be based on existing evaluation data and reports...”

8. IDEA requires that a teacher with disability-specific certification teaches children with the

corresponding disability. False

Federal regulations require that special education specified in a child's IEP is provided by qualified personnel. Teacher certification is a state licensure issue.

9. Federal regulations require the same assessment procedures to be used with all children suspected of having the same disability. False

The language term used in the statute is “full and individual evaluation”

It is a presumption that there may be overlap between the data sources collected for different children. However, no two children’s needs will be the same.

10. Placing children in classrooms based on their specific disability is permissible by federal law. False, False, False

“The unavoidable consequence of such a labeling practice is to identify and plan to meet each child's educational needs on the basis of what that child has in common with other children similarly identified rather than on the basis of that child's individualized needs. Thus it is the view of this office that any labeling practice that categorizes children according to their disability in order to facilitate the individual determination of any child's appropriate educational needs or services will be presumed to violate the protections accorded under Federal and State Law.”

Thomas Bellamy, former OSEP Director

So Where Does That Leave Us?

The Solutions(Research and Common Sense Into Action)

We can't work any harder!

So... We gotta work

smarter And... It will require the

whole system working together

The Solutions(Research and Common Sense Into Action)

Instructional design advances Especially in Reading!!

Behavior change technological advances FBA and it’s associated

systems-interventions Assessment systems linked

to intervention One integrated problem-

solving structure that eliminates bureaucratic silos

OSEP: Specific Learning Disabilities: Finding Common Ground – July ‘02

A multi-tiered, collaborative problem-solving approach, which incorporates early intervention, trial teaching, progress monitoring, and a inter-disciplinary evaluation, was named as a promising alternative the current IQ-Achievement discrepancy model for identifying students with Learning Disabilities.

To Move Toward Problem Solving

Best to attend to a series of “Big Ideas” or Principles

Principle #1: The Key to All is Every

Assessing and ensuring the effectiveness of, efforts to educate ALL Children

P.L. 105-17, Sec 601 (C)(10)(d)(4)

Principle #2: Shift in Emphasis For Assessment

Old Focus

New Focus

Diagnosing the “Disability” or “Diagnosing for Accountability”

Diagnosing the “Learning Enabled”

Shift From Problem Identification/Admiration to Problem Analysis

Our old system labeled and placed, or labeled and rewarded/sanctioned

Our new system needs to analyze problems: For the purpose of

understanding changeable factors related to learning problems

For the purpose of identifying possibly effective teaching strategies

Monitoring when teaching strategies are effective and helping us change them when they’re not

Illustration for Students with Disabilities (e.g., Melissa)

WISC-III Woodcock Johnson – R Stanford Diagnostic

Reading Test Motor Screen Bender Teacher Interview Speech Screening Health History Social History Educational History

Intervention Summary Review

Vision-Hearing Screening Parent and Teacher

Interviews CBM Normative

Comparisons Curriculum-Based

Evaluation Survey-Level Curriculum-Based

Evaluation Specific-Level Procedures

Our Old System Our New System

Principle #3

Support seamless school wide resource-allocation systems that:

Match resources deployed with nature and intensity of student need

Emphasize prevention and early intervention efforts

Level IVIEP

Consideration

• ImplementPlan

• Evaluate

• Define the Problem

• Develop a Plan

Am

ou

nt

of

Reso

urc

es

Need

ed t

o S

olv

e P

rob

lem

INTENSITY OF PROBLEM

Level IIConsultation withOther Resources

Level IIIConsultation WithExtended Problem

Solving Team

ConsultationLevel I

BetweenTeachers-Parents

Heartland AEA’s Problem Solving Approach

Look

Fam

iliar?

Iowa’s Experience: How it all started

Began in 1986-1987 Discussions with

stakeholders Parents Teachers Administrators Area Education Agency

Personnel Policy Makers

Over 4000 persons contributed

A Series of Questions Were Asked

What is working with the current system?

What components of the system are in need of reconsideration?

What barriers get in the way of trying these changes?

Important - There was no presumption that what we were doing was not being done well.

Iowa’s Experience Systematically piloted in late

’80s and early ’90s Changed state rules in ’95 Requires General Education

Intervention Defines systematic problem

solving Promotes assessments

tailored to individuals’ needs Assessment for identifying

problems, analyzing them, planning interventions, monitoring progress and evaluating effectiveness

What Happened: SPED Count

Special Education Enrollment in Iowa Public Schools as a Percentage of Total Certified Enrollment

0

10

20

30

40

50

60

70

80

90

100

1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002

Year Ending

Pe

rce

nt

Source: Iowa Department of Education Annual Condition of Education Report 2001

What Happened: Due Process Hearings in Iowa

SPED Hearings Held

0

1

2

3

4

5

6

7

8

1989-1990

1990-1991

1991-1992

1992-1993

1993-1994

1994-1995

1995-1996

1996-1997

1997-1998

1998-1999

1999-2000

2000-2001

2001-2002

Year

Nu

mb

er

of

He

ari

ng

s

Hearings Held

Source: Iowa Department of Education, Bureau of Children, Family and Community Services

Evaluation type, frequency, and percentage of total assessments in Heartland AEA 111997-2002

•Assessment Type•Initial Evaluations •Re-Evaluations

•Functional Academic Assessment •8,513 (51%) •5,201 (47%)

•Communication •2,985 (18%) •851 (8%)

•Behavioral Assessment •2,037 (12%) •1,817 (16%)

•Health •808 (5%) •481 (4%)

•Social Functioning •688 (4%) •589 (5%)

•Hearing •542 (3%) •234 (2%)

•School Readiness •404 (2%) •215 (2%)

•Motor Functioning •333 (2%) •454 (4%)

•Vision •175 (<1%) •140 (1%)

•Educational History •111 (<1%) •149 (1%)

•Vocational •99 (<1%) •908 (8%)

•Intellect •15 (<1%) •7 (0%)

•Grand Total •16,710 •11,046

•Number of Students Assessed •8,189 •5,027

Outcomes of BAT for Students- approx 10% of students ’99-’01 School years (n=532). Taken from Ikeda, M. J. and Gustafson, J. K. (2002). Heartland AEA 11's Problem Solving Process: Impact on Issues Related to Special Education. Heartland Area Education Agency, Johnston, IA

What Happened: Assessments

What Happened: Consumer Satisfaction

Question 1: The problem solving process supports teachers in improving the performance of students whose academic skills and behaviors are of concern. This includes the Building Assistance Team or other intervention supports.

•Gen Ed •Teachers

•n=390•Principal

•n=31

•Sp Ed •Teachers

•n=89

•Agree •87.3% •96.8% •92.13%

Question 2: Problem solving process leading to educational interventions is equally applicable for helping students in general and special education.

•Gen Ed •Teachers

•n=390•Principal

•n=31

•Sp Ed •Teachers

•n=89

•Agree •81.0% •96.7% •92.14%

Source: Heartland AEA 11 Consumer Satisfaction Survey 2000-2001

What Happened: Results for Kids

Pending Level III

Intervention (n=43)

SpecialEducation

Continuing Intervention in

General Education(n=219)

General Education

Problem Solved (n=107)

Staffed into Special

Education (n=116)

Outcomes of BAT for Students- approx 10% of students ’99-’01 School years (n=532). Taken from Ikeda, M. J. and Gustafson, J. K. (2002). Heartland AEA 11's Problem Solving Process: Impact on Issues Related to Special Education. Heartland Area Education Agency, Johnston, IA

Summary: Priorities for Assessment Revisions

Results not process Assess skills, not aptitudes Refocus on early identification and prevention

efforts. Provide a continuum of resources to meet students needs

Punch Line We have an exciting

opportunity before us We have many of the tools

we need to move ahead A broad base of

experimentation is occurring across the country

We can set direction for where we go next

The critical difference between places where change takes hold and flourishes and where it founders, is LEADERSHIP

Joel Barker, 1992

A leader is a person you will follow to a place that you wouldn't go by yourself.

The greatest danger for most of us is not that our aim is too high and we miss it, but that it is too low and we reach it.

Michelangelo

Activity Last

Individually, identify one idea that “made you think – HMMM”

Individually identify one thing you want to know more about.

Individually identify one question that you have.

Share these out at your table