Warlick, Quenemoen, Rigney 2004 Options for including students with learning disabilities in...

Warlick, Quenemoen, Rigney 2004

Options for including students with learning disabilities in assessments for

school accountabilityKen Warlick, University of Kentucky

Rachel Quenemoen, University of Minnesota

Sue Rigney, U.S. Department of Education


Objective

• Increase awareness of implications of federal assessment and accountability requirements on students with learning disabilities

• Increase awareness regarding accommodations

• Increase awareness regarding alternate assessments

• Increase awareness on emerging trends in assessment


Purpose of No Child Left Behind

“…to ensure that all children have a fair, equal, and significant opportunity to obtain a high-quality education and reach, at a minimum, proficiency on challenging State academic achievement standards and state academic assessments”


Purpose of Assessment Requirements of IDEA

• Improve results for student with disabilities through improved teaching and learning

• Raise expectations for students with disabilities

• Increase access to the general curriculum

• Provide parents information about their child’s achievement in relationship to the performance of other children in their school


Issues in both NCLB and IDEA

• Students with disabilities previously exempted from assessment and accountability system

• Students with disabilities previously received instruction in separate curriculum

• Change from low to high expectations for students with disabilities

• State leadership in fostering school and district accountability


Assessment Options

• General assessment

• General assessment with accommodations (or modifications)

• Alternate assessment on grade level achievement standards

• Alternate assessment on alternate achievement standards

• Computer-based assessments


Accommodations: The National Picture

• Accommodations use is on the rise.• About 50% of students with learning disabilities

receive an accommodation during testing.• Evidence from experimental studies indicates that

some accommodations boost performance.• Most common accommodations are

– Small group administration– Read-aloud– Extended time


Definitional IssuesSome accommodations are considered to change the construct assessed, and others are viewed as “ok”

Okay Not Okay

Accommodation

Adaptation

Standard administration

Modification

Modification

Non-approved accommodation

Non-allowed accommodation

Non-standard administration

Know what terms mean in your state and district!


What Makes An Assessment Accommodation OK?

It is aligned with instructional accommodations, but is not an excuse not to teach

Student needs it to demonstrate knowledge and skills – or to participate in assessment

The consequences of using the accommodation have been carefully considered

OK is not defined by what the test publisher says (unless has research evidence)


Closing the Achievement Gap Looks Like This:


Can states close achievement gap without paying attention to LD?

• Keep high incidence populations in school and in special education (LD 3 X as likely to drop out of school as non-disabled)

• LD comprises 50% of special education population• LD is a life-long disability. Student’s needs

change over time. They need learning and coping strategies but special education does not cure them


Research on Accommodations for LD Students – What Does It Tell Us?

• Accommodations most frequently studied are oral presentation (13) and extended time (10)

• Various research designs are used

• Many limitations in conducted research are identified


Issues:

• Tendency to allow too many accommodations, possibly reducing expectations for student learning.

• Poor decision making about accommodations, reflecting lack of knowledge about instructional accommodations.

• Use of accommodations as an excuse to exclude students scores on reports or accountability.


Promising Practices:

• States keeping track of how many students use accommodations, which ones, and their impact on achievement.

• Out-of-the-box thinking about the use of accommodated test scores.

• Clear decision-making criteria (e.g., alignment to instructional accommodations) and training on how to make decisions.

• Teaching high school students (at the latest) about the tests they take and about the accommodations that they need.


Implications for IEP teams

• Must be familiar with the state or district content/learning standards

• Must be familiar with the curriculum

• Must be familiar with typical instructional activities in general education


Adequate Yearly Progress


Alternate Assessments

Aligned with the State’s content standards.

Yield results separately in reading/language arts and math.

Designed and implemented to support use of the results to determine AYP.


Alternate Assessments should have…

Clearly defined structure Guidelines for which students may

participate Clearly defined scoring criteria and

procedures Report format that clearly

communicates student performance in terms of the academic achievement standards defined by the State


Alternate AssessmentsMust meet the same requirements for high technical quality that apply to regular assessments under NCLB:

Validity

Reliability

Accessibility

Objectivity

Consistent with nationally-recognized professional and technical standards.


States may use more than one alternate assessment

Alternate assessment scored against grade-level standards

Alternate assessment scored against alternate achievement standards


Out-of-Level Assessments

Could only be considered an alternate assessment based on alternate achievement standards if

Alternate achievement standards defined through a documented and validated standards-setting process

Proficient results included in the 1% cap


Researching Out-of-Level Testing

10 Recommendations: See Myth-Busters Handout


Universally Designed Assesmsents: A Quick Definition

Universally designed assessments are built from the beginning and continually refined to be accessible and valid for the greatest number of students!


But, what does that really mean?

• Do we want to change the standard of performance? NO

• Can we forget about accommodations if we do this? NO

• Is this all figured out – for now and forever? NO

• Is this something that will benefit only students with disabilities? NO


Universally designed assessments reflect good measurement qualities:

Minimize skills required beyond those being measured

Reflects the intended content standard (reviewers have information about what is being measured)

Measures What it Intends to Measure


An Example of Universal Design:

Mathematics Tests

The reading requirements of a math test often prevent students with marginal reading ability from demonstrating competency in math.


Ordering Pizza (Original Item)The cafeteria manager surveyed the students in a middle school to find out if they would buy Brand X pizza on Friday if the manager sold it. She made a circle graph to display the results of her survey.

NO

YES

Based on the results of the survey, answer the following questions:

1.1. What fraction of students would buy Brand X pizza on Friday?What fraction of students would buy Brand X pizza on Friday?

2.2. What percent of students would buy Brand X pizza on Friday?What percent of students would buy Brand X pizza on Friday?

3.3. There are 1200 students in this school. How many students will There are 1200 students in this school. How many students will buy Brand X pizza on Friday if the manager’s survey is accurate?buy Brand X pizza on Friday if the manager’s survey is accurate?


Ordering Pizza (Revised Item)Maria surveyed the students in her school to find out if they liked pizza on Friday. She made a circle graph to display the results of her survey.

NO

YES

1.1. What What fractionfraction of students said “yes”? of students said “yes”?

2.2. What What percentpercent of students said “yes”? of students said “yes”?

3.3. There are 1200 students in Maria’s school. There are 1200 students in Maria’s school. How manyHow many students said “yes”?students said “yes”?


Suppose a test item requires a student to read an Aesop’s fable

What could this item measure?– Decode text?– Comprehend extended passages?– Extent of vocabulary?– Understand the moral or point of the fable?– Discuss the common elements of any fable?– Compare and contrast fables with news reports?– Articulate the relationship between the fable and

the overall culture? – Anything else? (National Center on Accessing the Curriculum, 2003)


Something to think about


The “door” we need to go through: What is meant by the construct of ‘reading’ What about ‘literacy?’

The answer(s) to these questions will determine the universal design

or accommodations that can be used when certain content

standards are taught and assessed!


Modes of print interaction: with examples of accommodations

• Visual • Tactile (feeling print)• Auditory (listening to

printed messages)• Multi-modal (using

any combination of the above modalities)

• Printed text; ASL• Text in Braille and

Nemeth Codes• Listen to taped text• Computer-based

“assistive” reading/ viewing programs


Individualized accommodation decisions should be linked to the standard, the construct assessed, the nature of instruction or the assessment, and the student’s characteristics

Purposeful reading – reading to select

and apply relevant information for a given task

Does this allow different modes of print interaction? And, what are the implications of these different modes for accommodations?


Issues:• Universal design does not eliminate the

need for accommodations.

• Universal design requires the systematic application of good test development – more attention needs to be given at the same time that we need to develop more assessments.

• Little research exists on universally designed assessments.


Promising Practices:

• Item developers who know disability issues and their interaction with test formats and procedures.

• “Bias” or “sensitivity” review committee with at least one member who knows disability issues.

• Field testing that includes students with disabilities who use accommodations during the field test.


What is computer-based assessment

• Does NOT mean adaptive testing

• DOES mean making use of technology for accommodations

• DOES mean grappling with very, very important issues


Opportunities

• Efficient administration• Preferred by students• Improved writing performance• Built-in accommodations• Immediate results• Efficient item development• Increased authenticity


Challenges

• Use of technology cannot take the place of content mastery

• Issues of equity and skill in computer use

• Added challenges for some students

• Technological challenges

• Security of online data

• Lack of expertise in designing accessible Web pages

• Prohibitive development cost


Example: Kentucky Online Assessment

• Web-based, individualized assessment :– Students with IEP or 504 Plan that specifies need for

"reader" as an instructional and assessment accommodation;

– Students who require and routinely use text-reader or screen-reader technologies to access printed material in classroom instruction and assessment;

– Students who have accessed and used the CATS Online Practice Area.

• Based upon success of pilot studies, 16 districts, 31 Schools, & 204 students participated in “live” CATS Online in the spring of 2003


CATS Online


Visit: education.umn.edu/nceo

or Search for NCEO

Check us out!

Warlick, Quenemoen, Rigney 2004 Options for including students with learning disabilities in...

Documents

Transcript of Warlick, Quenemoen, Rigney 2004 Options for including students with learning disabilities in...