Assessment of English Language Learners€¦ · Assessment of English Language Learners Saturday,...

Jamal AbediUniversity of California, Davis

Presented at:The Race to the Top Assessment Program

Public & Expert Input MeetingDecember 2, 2009Denver, Colorado

Assessment of English Language Learners

Saturday, December 7, 13

Assessment results have major impact on ELL students’ academic life more so than on non-ELL students



Classification (for ELLs)




Instruction (assessment before instruction for ELLs)





Accountability (multiple accountability requirements for ELLs)






Promotion (reclassification)






Promotion (reclassification)

Graduation


Assessment challenges for ELL students



• ELL students go through assessment and accountability requirements and challenges twice:




• Title III, the English language proficiency assessment (AMAO1 and AMAO 2)





• Title I assessment in reading/language arts, math, and science





• Title I assessment in reading/language arts, math, and science

• They are faced with the assessment issues in both areas


Focus of this presentation



A. General assessment issues that apply to ELL students as well




B. Assessment issue specific to ELLs




B. Assessment issue specific to ELLs

C. Answers to the RTTT questions regarding ELL assessment and my recommendations


A. General assessment issues that apply to ELL students (Already being discussed at the earlier RTTT presentations by Baker, Bennett, Braun, Darling


A. General assessment issues that apply to ELL students (Already being discussed at the earlier RTTT presentations by Baker, Bennett, Braun, Darling 1. Theory of Action



2. Link between assessment and instruction




3. Using multiple measures





4. Interim & formative assessments






5. Preparation of RFP







6. Use of technology








7. Common content standards









8. Growth measures over time










9. Providing teacher professional development opportunities










9. Providing teacher professional development opportunities

(We will elaborate on some that are more related for ELLs)


2. Link between assessment and instruction (currently, a disconnect)


2. Link between assessment and instruction (currently, a disconnect) • As Darling-Hammond indicated, “..high-achieving nations used open-

ended performance tasks and school-based, curriculum-embedded assessments to give students opportunities to develop and demonstrate

higher-order thinking skills.”





• Instruction should inform development of assessment and assessments should inform instruction






• It is therefore essential to involve state assessment folks and teachers in the process of test item writing and test development







• Teachers should be trained and be involved in all different phases of test development process and use for ELL students including item

writing, scoring, and interpreting the results







• Teachers should be trained and be involved in all different phases of test development process and use for ELL students including item

writing, scoring, and interpreting the results

• The assessment process should help teacher in preparing students for college and career ready standards


3. Using multiple measures• Once a year assessments in Title I and Title III, with all their

limitations particularly for ELL students, may not produce valid and fair outcomes.




• There are concerns over reliability and validity of these instruments for ELLs (e.g., for some content areas reliabilities

are as low as .50)





are as low as .50)

• These assessments include a substantial amount of measurement error which make high-stakes decisions based

solely on their outcomes quite risky





are as low as .50)

• These assessments include a substantial amount of measurement error which make high-stakes decisions based

solely on their outcomes quite risky

• A series of measures from different tests with different format and different tasks, given at different times, would be needed

to make fair decisions about classification and academic performance of ELL students.



• Summative assessments while quite important for accountability purposes may be too little too late to inform curriculum and

instructions




instructions

• They are used mostly for accountability purposes without much instructional values




instructions


• Interim and formative assessments provide teachers with the information needed to help ELL students




instructions


• Interim and formative assessments provide teachers with the information needed to help ELL students

• Outcomes of formative assessments may also help parents of ELL students to identify areas that they children need attention



• As an example of the use of technology in assessment, computerized assessment system can be discussed




• Computerized assessment system has the flexibility and capability of incorporating many accessibility features for

assessments of ELL students






• Examples are: English and bilingual glossary, read aloud of content-based assessment items and providing extra time







• Assigning test items with different levels of linguistic complexities to students at different levels of language

proficiency







• Assigning test items with different levels of linguistic complexities to students at different levels of language

proficiency

• Providing opportunities for students to be tested in a language that produces the most valid assessment outcomes



Growth measures are important for ELLs

English language proficiency (Title III) lends itself well to growth over time

An example of application of the concept of growth model is for the AMAO 2

While it is extremely helpful to think about growth concept and its measurement model, it is also

important to think about its limitations such as: issues concerning baseline changes in the measures overtime the nature of the content being assessed establishing a meaningful vertical scale


B. Assessment issues specific to ELLs



The misconception that ELL students are only quantitatively different from native speakers of English should be discussed




1. Understanding of the two different assessment systems for ELLs, their similarities, differences and interactions





2. Lack of interaction between the two systems, English language proficiency and content assessments






3. Construct-irrelevant sources in measurement of ELL students







4. Applicability of measurement theories in ELL assessments








5. Impact of L1 proficiency on the assessments and instructions for ELL students









6. Classification, reclassification, and misclassification of ELL



• There are two different assessment systems for ELLs, Title III ELP and Title I content assessments




• While they involve different tests based on different content standards and different objectives, they should work together.





• We have made significant progress in the assessment of ELP for ELL students due to the implementation of NCLB, e.g., ACCESS for ELLs,

ELDA, LAS LINK, SELP, etc







• There is a need for substantial work on improving Title I assessments for ELLs








Recommendation:








Recommendation:

Provide support for more improvement in the Title III assessments and support creating more valid assessment system in measuring ELL

content knowledge (RTTT).


2. Lack of interaction between ELP and content assessments



• ELL students must be at a certain level of English proficiency to be able to meaningfully participate in the Title I assessment




• Only students at the proficiency level 4 or above may be able to participate in Title 1 assessments.





• However, there is a disconnect between student’s level of ELP and their participation in content-based assessments.






Recommendation:






Recommendation:

Include ELL students in content assessments in English if they are at the proficiency level to meaningfully participate (level 4 or above);

otherwise, provide valid alternatives such as native language testing, relevant accommodations, etc.



• There are different sources of construct irrelevant variance affecting ELL students assessment outcomes




• Unnecessary linguistic complexity of assessment as a source of construct irrelevant variance adds an additional factor (dimension)

to the assessments for ELLs






• Other sources of construct irrelevant variance such as cultural biases also add additional dimensions to the assessment outcomes

for ELL students







for ELL students

Recommendation:







for ELL students

Recommendation:

Provide ELL professional training to the test item writers and include teachers and linguistic/cultural experts at the item development

process to control for these sources.



• A major assumption underlying classical theory of measurement is uni-dimensionality




• Construct-irrelevant sources for ELL students introduce complexity into the measurement model for these students





• While multidimensional assessment can be handled both in classical and IRT models, those dimension should be clearly

relevant to the assessment







Recommendation







Recommendation

Revisit psychometric principles in light of current research on the assessment of ELLs and make appropriate revisions



• A major oversight in the assessment of ELL students is the lack of attention to their level of native language proficiency




• Proficiency in L1 would help in both instruction and assessment of ELLs





• Native language instruction and assessment could be a great success if students are academically proficient in their native

language.






language.

Recommendation






language.

Recommendation

Include valid and comprehensive measures of ELL students’ level of L1 academic proficiency in all proficiency domains (reading, writing,

speaking and listening) and seriously consider the results in planning their curriculum and assessment


6. Classification, reclassification, and misclassification of ELL students



• If students are not properly classified as ELLs/non-ELLs, instructions, assessments and accommodations for ELL students will

not be productive




not be productive

• Similarly, if ELL students are not properly re-classified as fluent in English when they reach fulency, they may not benefit from proper

instruction and their assessment outcomes may not be valid




not be productive



• There are major concerns on misclassification of ELL students as those having learning disabilities.




not be productive




Recommendation




not be productive




Recommendation

• Multiple reliable and valid criteria should inform decisions about classification/reclassification for ELL students.


C. My answers to the RTTT questions regarding ELL assessment and my recommendations



• Provide recommendations for the development and administration of assessments for each content area that are valid and reliable for English

language learners




language learners

• How do you recommend that assessments take into account the variations in English language proficiency of students?




language learners


• How can technology be utilized to make assessments more inclusive to ELL students?




language learners



• what are the relative merits of developing and administering content assessments in native language?




language learners



• what are the relative merits of developing and administering content assessments in native language?

• What are the technical, logistical, and financial requirements?


Question1a: Provide recommendations for the development and administration of assessments for each content area that are valid and reliable for English language learners



General recommendations:




• ELL students should not be treated differently in the content being assessed; otherwise comparability of assessment outcomes may

become a major concern






• However, assessments in the content-based areas (such as math, science and social sciences) should be free of unnecessary linguistic

complexities and cultural biases








• Multiple measures should be implemented in both Title I and Title III that utilize different measurement approaches








• Multiple measures should be implemented in both Title I and Title III that utilize different measurement approaches

• Provide accommodations that help reduce the effects of construct-irrelevant factors


Question 1b: How would you recommend that the assessments take into account the variations in English language proficiency of students in a manner that enables them to demonstrate their knowledge and sills in core academic areas?



• Provide assessments with varying degree of linguistic complexity




• Provide appropriate accommodations for ELL students to help them with their common needs

of language assistance




• Provide appropriate accommodations for ELL students to help them with their common needs

of language assistance

• Provide native language testing opportunities for students who are academically proficient and are

instructed in their native language


Question 1c: How can technology be utilized to make assessments more inclusive to ELL students?



• Provide computerized assessments that are capable of offering effective and valid accessibility features including

appropriate accommodations





• Provide web-based tutorials for ELL students for having additional opportunities to learn






• Provide web-based training for parents to help their students






• Provide web-based training for parents to help their students

• Provide diagnostic information to teachers, parents, and


Question 2a: In the context of reflecting student achievement, what are the relative merits of developing and administering content assessments in native language?



• If students are proficient in their native academic language and have been instructed in native language then native language

assessments would be productive





• Assessments in native language could be less affected by linguistic and cultural biases

Language is no longer a source of construct-irrelevant variance





• Assessments in native language could be less affected by linguistic and cultural biases

Language is no longer a source of construct-irrelevant variance

• If students have not been instructed in their native language, then these assessments would not help in measuring student

achievement


Question 2b: What are the technical, logistical, and financial requirements?



Logistical:



Logistical:

• Alignment between the language of assessments and language of instructions



Logistical:


• Issues addressing all of the languages spoken in our schools



Logistical:



• Issues concerning translations in different languages



Logistical:




Financial:



Logistical:




Financial:

• Issues with developing, field testing and preparing operational forms for all languages


For more information please contact



Jamal Abedi



Jamal Abedi at



Jamal Abedi at

CRESST/UC Davis



Jamal Abedi at

CRESST/UC Davis

(530) 754-9150



Jamal Abedi at

CRESST/UC Davis

(530) 754-9150or



Jamal Abedi at

CRESST/UC Davis

(530) 754-9150or

[email protected]


Assessment of English Language Learners€¦ · Assessment of English Language Learners Saturday,...

Documents

Transcript of Assessment of English Language Learners€¦ · Assessment of English Language Learners Saturday,...