Adding Value to Accommodations Decision-Making (AVAD) 2006 Enhanced Assessment Instruments Grant...
-
Upload
ronnie-galloway -
Category
Documents
-
view
213 -
download
0
Transcript of Adding Value to Accommodations Decision-Making (AVAD) 2006 Enhanced Assessment Instruments Grant...
Adding Value to Accommodations Decision-Making (AVAD)
2006 Enhanced Assessment Instruments Grant Project
Elizabeth Jones and Christopher Webster, South Carolina Department of Education
2006 Enhanced Assessment Instruments Grant Award
The South Carolina Department of Education received a 2006 Enhanced Assessment Instruments Grant award to undertake the AVAD project.
Overall goal of AVAD:
To ensure the valid, effective use of accommodations designed to increase access to large-scale assessments for English language learners and students with disabilities
AVAD Project Partners
South Carolina Department of Education, lead state and fiscal agent
LEP SCASS ASES SCASS Michigan State University: Rebecca Kopriva University of Oregon: Gerald Tindal University of Nebraska/Psychometric
Inquiries: Barbara Plake Phoebe Winter, project evaluator
AVAD Project Period
Current project period from October 1, 2006 to March 31, 2008
One-year no-cost extension will be requested SCASS grant activities will begin in
spring/summer 2007 SCASS grant activities will be scheduled
during or immediately before/after regularly scheduled SCASS meetings
AVAD’s Objectives:
1) to validate and enhance the Selection Taxonomy for English Language Learner Accommodations (STELLA) and publish the program on the SDE’s Web site
2) to create and validate a decision-making taxonomy for the Accommodation Station (AS) based on the ASES Accommodations Manual and publish the program on the SDE’s Web site
AVAD’s Objectives, cont.:
3) to develop 20 prototypes of computer-based access enhanced science items for grade four, and
4) to disseminate results through reports, the SDE’s Web site, User and Technical Manuals for STELLA and AS, the ASES Professional Development Guide video, and Web-Ex teleconferences
What role does the LEP SCASS play? LEP SCASS will serve as expert panel to provide input and
feedback on STELLA and computer-based access-enhanced prototype items
Opportunity to participate in a subgroup of states within the SCASS focusing on accommodations issues
Accommodations subgroup will commit to participation in AVAD meetings/conference calls
Updates on AVAD will be provided to the larger SCASS group
(ASES SCASS will play a similar role in Accommodation Station design and development)
What is STELLA?
STELLA=Selection Taxonomy for Testing English Language Learners an online system to assist educators in
assigning testing accommodations to English language learners (ELL) so that individual students receive appropriate accommodations
developed by a 2003 Enhanced Assessment Instruments grant
Data Collection Teacher Form Parent Form Records Form
Preloaded in System Student Information Variables Test Accommodations Conversion and Consolidation Rules Decision-Making Rules
Output Student Profile Accommodation Decisions for Each Student
Associated Materials Data Collection Manual Output Interpretation Manual Technical Manual
STELLA Overview
Current Accommodations loaded in STELLA system
Pre-Test Best Practice Accommodations Family Assessment Night Tailored Classroom Support
Forms Standard or some Universal Design Forms Access-based Form L1 or Side-by-Side forms as available
Tools Bilingual word list, general or test specific Picture-word dictionary Problem solving tools
Current Accommodations loaded in STELLA system
Administration Small group Individual Administration Oral English Oral L1 Language Liaison Extra time More frequent breaks
Response Written L1 or code switching Oral English Oral L1 or code switching Demonstrated or modeled response
English and L1 proficiency (L1 includes ‘don’t know’ option)
Standardized score accuracy and judgments about reasons for inaccuracy
Student’s experience with standard test formats
Student’s perceived purpose of standardized testing
Classroom test condition options
Condition options that help student on classroom tests, evaluations
Teacher Form
System Output
Individual Student Profile• English language proficiency, 4 domains • L1 proficiency, 4 domains • Cultural proximity, 3 variables:
1. Time in U.S.2. Experience with testing procedures3. Schooling proximity, native country to U.S. experience
System Output: STELLA, Teacher Recommended and SEA/LEA Allowed
Since different states allow different accommodations, STELLA program output is set up to identify the subsetof allowable accommodations for each student.
Non allowable accommodations selected for each student based on research and other best practice literature are also displayed in gray. This provides guidance to teachers and others about additional accommodations that could be useful for ELL students with specified needs.
Test Conversion Rules
Currently, output from 4 English language proficiency tests (including ELDA) is preloaded. There is also the opportunity for personnel to add results from another test.
In all cases score conversion rules place students on a common scale with four levels: Beginner ELL Low intermediate ELL High intermediate ELL Grade-level-competitive ELL
Student Information ConsolidationRules
In several cases, more than one piece of student information is used to make judgments about his or her level on relevant variables. Two types of consolidation rules are part of STELLA. These are:
Consolidation of data from related items Consolidation of information from more than one
source
Time in School = LOW
If the student has been in the US less than 1 academic year.OR
If the student has been in US between 1 and 2 years AND has missed more than two months of school per year for 1 or more years.
ORIf the student has been in US between 2 and 3 years AND has
missed more than two months of school per year for more than one year.
Combining Data from More Than One Question: An Example
Combining Data from More Than One Source: An Example
Consolidation rules are also used to determine L1 proficiency and English language proficiency.
For L1 proficiency, consolidation rules are applied to information from teacher, parent and records forms.
Parents describe student as performing above grade level in L1
Teacher describes student as performing below grade level in L1
Parents trump teacher’s knowledge of L1, but test scores of L1 proficiency trump parents.
What Trumps What? Consolidation Rules Example
Test Accommodation Decision-Making Rules
A beginning set of decision-making rules were developed and tested. These rules take relevant student information and pair it with relevant accommodation factors for individual students. The accommodation factors identify which student needs the specific accommodation was designed to remediate.
In this way, individual students are matched with accommodations appropriate to their particular needs.
LEP SCASS Validation and Enhancement of STELLA Activities
The LEP SCASS accommodations sub-group will validate existing accommodations decision-making rules by participating in a two-meeting and online expert panel process led by Barbara Plake.
Profiles of ELL students will be matched with specific sets of accommodations; accommodations for individual student profiles will be rated for validity and effectiveness; results will be compared to existing STELLA accommodations decision-making rules.
STELLA decision-making structure will be tweaked based on results of expert panel process.
STELLA interface and forms will be reviewed and revised.
Computer-based access-enhanced (CAE) science items review
CAE items aim to reduce barriers to access that English language learners may encounter in typical large-scale assessment items without altering the construct being measured or the rigor of the items
South Carolina will develop 20 prototypes of computer-based access-enhanced science items for grade four and place them online
LEP SCASS will review the CAE items and provide feedback
Benefits of Participation
LEP SCASS member states will have input into the design and development of STELLA
LEP SCASS member states (and any other interested parties) will have free access to the STELLA once it is completed and placed on the SDE Web site
LEP SCASS member states will be able to adapt the STELLA to best fit the policy and needs of their states
ASES SCASS: Parallel Activities The ASES SCASS will serve as an expert panel for the
development and review of the Accommodation Station (AS), an online decision-making model designed to assist IEP teams in assigning valid and appropriate accommodations to individual students with disabilities.
Current AS collects and consolidates information but does not include a decision-making algorithm.
The ASES manual and materials will serve as the basis for an algorithm that will be built and incorporated into the current AS.
A professional development video component will be developed to accompany existing ASES materials.
Sharon Hall of ASES will share additional details.
Questions?
Please contact Therese Gleason Carr at [email protected] or (803) 734-3747
Thank you very much for your collaboration!