How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The...
-
Upload
caitlin-nash -
Category
Documents
-
view
218 -
download
1
Transcript of How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The...
![Page 1: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/1.jpg)
How a Theoretical and Data-based Modification Process Can Help
Students Eligible for an AA-MAS
The Consortium for Alternate Assessment Validity and Experimental Studies
(CAAVES Project)
Presented by Ryan J. Kettler on April 15, 2009 at the annual meeting for the National Center for Measurement in Education San Diego, CA
The CAAVES project is funded by theUS Department of Education, Office of Elementary and Secondary Education
![Page 2: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/2.jpg)
2
CAAVES Project Research Questions
We sought to answer the following questions about item modifications:
1) Will modifications in testing conditions change the reliability of measurement?
2) Will taking the test under modified conditions change the comparability of scores for eligible students?
3) What specific modifications are the most helpful for making the scores of eligible students comparable to the scores of ineligible students?
![Page 3: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/3.jpg)
Multi-State Sample
Students without
Disabilities, Ineligible
Students with
Disabilities, Ineligible
Students with
Disabilities, Eligible
Arizona 28 25 21
Hawaii 20 16 18
Idaho 62 43 59
Indiana 159 152 152
![Page 4: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/4.jpg)
Typical Item in Original Condition
![Page 5: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/5.jpg)
Reading Item in Original Condition
![Page 6: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/6.jpg)
Item Modification Strategies
Universal Design Principles
Research on item answer choices
Cognitive Load Theory
Cognitive lab results
![Page 7: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/7.jpg)
CAAVES Item Modification Strategies
For all items: Removed least effective distractor Increased white space
For many items: Bolded key vocabulary terms Simplified language in the directions, stimulus,
stem, and answer choices Reorganization of layout Added graphic support
![Page 8: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/8.jpg)
Reading Item in Modified Condition
![Page 9: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/9.jpg)
Mathematics Item in Modified Condition
![Page 10: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/10.jpg)
Reliability and Concurrent Validity with a Proficiency Test
![Page 11: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/11.jpg)
Mean Item Difficulty by Group and Condition in Reading
* Student abilities were equated using a Rasch model.
![Page 12: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/12.jpg)
Mean Item Difficulty by Group and Condition in Math
CAAVES Modified Achievement CCSSO 2008 12
* Student abilities were equated using a Rasch model.
![Page 13: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/13.jpg)
Upcoming Analyses: Difficulty Change on Individual Items
Students without Disabilities, Ineligible
Students with Disabilities, Eligible
![Page 14: How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MAS The Consortium for Alternate Assessment Validity and Experimental.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649f455503460f94c6793e/html5/thumbnails/14.jpg)
Individual Modification Analysis
Reviewed items based on differential boost Nine well-modified items were identified Six poorly-modified items were identified
Patterns were revealed for two modifications One modification was used in 7 of 9 well-modified
items, and only 1 of 6 poorly-modified items Another modification was used on all 3 poorly-
modified reading items, and 0 of 4 well-modified items