Making Consistent Decisions About Accommodations for

English Language Learners

– Research Summit –

Texas Comprehensive Center @ SEDLAustin, Texas

March 16–17, 2009

Linguistic Accommodations on Assessments:Linguistic Modification

Edynn Sato, Ph.D.Director, Special Populations for AACC

WestEd

3

Content and Language

What is the relationship?

Content Language

Content Language

LanguageContent

4

What is Linguistic Modification?

• A theory- and research-based approach to clarifying and simplifying the language of the assessment without simplifying the content of the assessment.

• Linguistic modification is not simply “good editing” practice and does not result in “easier” test items.

5

What is Linguistic Modification?

• Linguistic modification involves a process intended to increase English learners’ access to assessment content and subsequently increase the validity of the assessment results.

– Access is the minimization or removal of conditions (i.e., sources of construct-irrelevant variance, such as aspects of presentation/format of test information, aspects of response requirements, sociocultural contexts or references that may disadvantage certain students, etc.) that may interfere with students’ ability to meaningfully engage with content or demonstrate their content knowledge and skills.

– Appropriate access does not significantly change the targeted construct

– Providing students the opportunity to learn, exposing them to grade-level content, or applying broadly the principles of Universal Design do not necessarily address the particular access needs English learners adequately.

– Lack of appropriate access has consistently posed a threat to validity.

6

Why Linguistic Modification?

• Relative to other strategies such as portfolios and native language translations, linguistic modification lends itself more readily to standardization and broader application across language groups.

7

Linguistic Modification: Key Points

• Addresses the linguistic needs of ELLs

• Increases the linguistic accessibility of the test item/task

• Does not change the content/construct tested

• Does not alter essential grade-level content-related language

8

Assessment of English Learners:General Considerations

Critical and interdependent factors include:• Population

– Characteristics (diverse)

– Access needs (linguistic)

• Content

– Language skill vs. content skill (how interrelated are the language and content skills)

– Construct

– Language demands

– Proficiency expectations

• Context

– Assessment vs. instruction

– Design/Presentation

9

Linguistic Modification: Considerations

• Item context

• Item graphics

• Item vocabulary/wording

• Item sentence structure

• Item format

…and, the interactions among these dimensions

10

Linguistic Modification: StrategiesCurrent Literature/research-based Additional

Item context

•Familiar to students; no cultural or linguistic bias

•Sometimes context is added to: remove passive voice construction, past tense and conditionals; break stem into shorter, less complex sentences (create a familiar story line)

•Helps make abstract/highly-generalized language more concrete

Item graphics

•Familiar to students; no cultural or linguistic bias

•Symbols, legends, key vocabulary are relevant to the construct and familiar

•Graphics support understanding

•Clarify: key/central aspects of content/construct; what the student is expected to do (purposeful, meaningful)

•Represents the situation in its totality (not misleading)

•Consistent labeling/naming

Excerpt from Sato (2008)

11

Linguistic Modification: Strategies (continued)

Current Literature/research-based Additional

Item Vocabulary/Wording

•High frequency words

•Common/familiar words

•Relevant technical terms; technical terms defined as appropriate

•No ambiguous words or unnecessary words with multiple meanings

•No irregularly spelled words

•Use of formal proper names are relevant and appropriate

•Naming conventions are consistent

•Consistent with standards/expectations

•“Common” and “familiar” is not necessarily appropriate—consider Academic English necessary

•Repeat key words—providing synonyms within the item may not be helpful

•Avoid words that are both nouns and verbs (e.g., value, cost, carpet)

•Avoid hyphenated and compound words

•Formal proper names should be ones that students are familiar reading

•Relative pronouns (e.g., which) should have a referent

Excerpt from Sato (2008)

12

Study of Linguistic Modification: Overview

• 4,617 seventh and eighth grade students from 13 schools in California

• ELs (Spanish), low reading ability non-ELs, high reading ability non-ELs

• Matched sets of original and linguistically modified versions of mathematics items (number sense, measurement)

• Random assignment of students

13

Study of Linguistic Modification(continued)

Preliminary findings:• Created a set of linguistically modified items that

did not significantly alter the mathematics constructs assessed – expert judgment, cognitive interviews of EL and non-

EL students, analysis of non-EL student performance on linguistically modified item and matched original items, analyses of non-EL student performance on statewide academic achievement tests and on the two versions of items developed for the study

14

Study of Linguistic Modification(continued)

Preliminary findings:• Positive effect, though not significant, of linguistic

modification of test items on the mean scores of EL students and on the mean scores of non-EL students with low-reading ability

15

Framework for understanding language demands on ELLs

Language Demands• A language demand is categorized as either a

Linguistic Skill or an Academic Language Function based on whether the demand is fundamental to the development and use of language or is a contextual application of language.

From: Sato, Lagunoff, Worth, Bailey, and Butler (2005); Bailey, Butler, and Sato (2007)

16

Language Demands

From Sato (2008, June)

17

Language Complexity—Density

Low HighLength ranges from a word to paragraphs    No/little variation in words and/or phrases in sentences/paragraphs; consistent use of language    Repetition of key words/phrases/sentences reinforces information    Language is used to present critical/central details    No/little abstraction; language reflects more literal/concrete information; illustrative language is used; language is used to define/explain abstract information     Graphics and/or relevant text features reinforce critical information/details

Length ranges from a word to paragraphs   Some variation in words and/or phrases in sentences/paragraphs   Repetition of key words/phrases/sentences introduces new or extends information   Language is used to present critical/central details, but non-essential detail also is presented   Some abstraction; language may or may not be used to define/explain abstract information; illustrative language may or may not be used; technical words/phrases are used   Graphics and/or relevant text features may or may not reinforce critical information/details

From Sato (2008, June)

18

Language Complexity—Construction

Simple ComplexMostly common/familiar words/phrases; no/few uncommon words/phrases, compound words, gerunds, figurative language, and/or idioms   Language is organized/structured   Mostly simple sentence construction   No/little passive voice   Little variation in tense   Mostly one idea/detail per sentence   Mostly familiar construction

(e.g., ’s for possessive; s and es for plural)   Mostly familiar text features

(e.g., bulleted lists, bold face)

Some common/familiar words/phrases; some uncommon words/phrases, compound words, gerunds, figurative language, and/or idioms   Language may or may not be organized/structured    Varied sentence construction, including complex sentence construction   Some passive voice   Variation in tense   Multiple ideas/details per sentence   Some less familiar/irregular construction   Some less familiar text features

(e.g., pronunciation keys, text boxes)

From Sato (2008, June)

19

Types of “Expertise” Needed

• Knowledge of language development and language acquisition

• Knowledge of English language learners

• Knowledge of content (English language arts, mathematics, science, social studies)

• Knowledge of (large-scale) assessment

• Knowledge of instruction

References• Bailey, A. L., Butler, F. A., & Sato E. (2007). Standards-to-standards

linkage under Title III: Exploring common language demands in ELD and science standards. Applied Measurement in Education, 20(1), 53–78.

• Sato, E., Lagunoff, R., Worth, P., Bailey, A. L., & Butler, F. A. (2005). ELD standards linkage and test alignment under Title III: A pilot study of the CELDT and the California ELD and content standards (Final Report to the California Department of Education). San Francisco: WestEd.

• Sato, E. (2008). Linguistic modification: Part II — A guide to linguistic modification: Increasing English language learner access to academic content. Washington, DC: LEP Partnership.

• Sato, E. (2008, June). Language as Content and Language of Content. What's the Difference? — Defining the Language Construct: Strategies and Implications. Presentation at the annual conference of the Council of Chief State School Officers National Conference on Student Assessment. Orlando, FL.

20

21

For more information:

Edynn Sato, Ph.D.

Director, Special Populations

Assessment and Accountability Comprehensive Center (AACC)

esato@wested.org

To learn more about the AACC:

www.aacompcenter.org