The Ecology of Language

Assessment

Carl Falsgraf, Director CASLS

The Language Education EcosystemThe Language Education EcosystemThe Language Education EcosystemThe Language Education Ecosystem

CurriculumCurriculum

InstructionInstruction

Formative AssessmentFormative Assessment

Summative AssessmentSummative Assessment

REFLECTI

ON

REFLECTI

ON

RE

FLEC

TION

REFLECTION

Can one assessment do it all?

ASSESSMENT CONTINUUM

Teacher administered

Customized

Curriculum dependent

+ high validity

+ supports instruction

- low reliability

- context dependent

Expert administered

Standardized

Curriculum independent

+ high reliability

+ context independent

- low validity

- divorced from instruction

STAMP?

ProficiencyProficiency

PerformancePerformanceAchievementAchievement

OPI

Oral quiz

Service learning project

Weekly quiz

Final exam

STAMP?

Specifications of Intended Test Use (SITU)

(Norris, 2003)

•Who are the test takers?

•Who are the test score users?

•What is the purpose of the test?

•What information should the test provide?

•What are the intended consequences of using the test?

Norris (2000)

STAMP?

0

0+

1

1+

2

2+

3

3+

4

4+

5

ILR

Novice-High

Novice-Mid

Novice-Low

Intermediate-High

Intermediate-Mid

Intermediate-Low

Advanced Plus

Advanced

Superior

Benchmark 1

Benchmark 2

Benchmark 3

Benchmark 4

Benchmark 5

Benchmark 6

ACTFL

BENCHMARKS

Ass

essm

ent

STAMP 101• • By teachers, for teachers By teachers, for teachers

• Designed to inform, not just measureDesigned to inform, not just measure

• • Online: Anywhere, anytime, anybodyOnline: Anywhere, anytime, anybody

• • Computer adaptive reading/listeningComputer adaptive reading/listening

• • Computer assisted speaking/writingComputer assisted speaking/writing

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

STAMP LISTENING

STAMP READING

STAMP READING

NOELLA SPEAKING What is happening in the picture? Tell as much as you can.

G4-IV-2-ED-1 SubQ 2

0

20

40

60

80

100

1 2 3 4 5

Group Score Rank

Percent chosen

ABCD

Example of a good item

G4-IV-2-SG-7

0

20

40

60

80

100

1 2 3 4 5

Group Score Rank

Percent chosen

A

B

C

D

Example of a bad item

Level 3

Level 4

CLASS REPORT

PROGRAM REPORT

SLA Web

National 3rd/4th YR Japanese Data

Proficiency Levels by Years (Spanish - High School - 7500 students)

0% 10% 20% 30% 40% 50% 60%

Novice-Low

Novice-Mid

Novice-High

Intermediate-Low

Intermediate-Mid

0% 10% 20% 30% 40% 50% 60%

Novice-Low

Novice-Mid

Novice-High

Intermediate-Low

Intermediate-Mid

0% 5% 10% 15% 20% 25% 30% 35% 40%

Novice-Low

Novice-Mid

Novice-High

Intermediate-Low

Intermediate-Mid

0% 5% 10% 15% 20% 25% 30% 35%

Novice-Low

Novice-Mid

Novice-High

Intermediate-Low

Intermediate-Mid

1 year of study 2 years of study

3 years of study 4 years of study

NEEDS

• AUDIT

• CERTIFICATION

• PROGRAM EVALUATION

• PROGRESS CHECK

• DIAGNOSIS

• IMPROVE INSTRUCTION

• MOTIVATE STUDENTS

NCLB/STAMP

AP/OPI

STAMP

MIDTERM

LINGUAFOLIO

LINGUAFOLIO

LINGUAFOLIO

ASSESSMENTS

DEVELOPING AN ASSESSMENT PLAN

• CLARIFY GOALS

• DEFINE PURPOSES/USES

• RESEARCH EXISTING TOOLS

• DEFINE INTERNAL TOOLS

• INITIATE ASSESSMENT LITERACY PROFESSIONAL DEVELOPMENT

SAMPLE ASSESSMENT PLAN