Performance on the Performance on the 10th-Grade WASL: 10th-Grade WASL:

Summary of Findings to DateSummary of Findings to Date

Presentation to theHouse Education Committee

January 19, 2007

Robert (Barney) Barnoski, Ph.D.Washington State Institute for Public Policy

2 of 25

Who We AreWho We Are

Washington State Institute for Public Policy

• Created by the 1983 Legislature• Conducts non-partisan research on topics assigned by

the Legislature or board of directors

Representative Helen Sommers Robin Arnold-Williams, DSHS Representative Fred Jarrett Victor Moore, OFMRepresentative Phyllis Kenney Sandra Archibald, University of WashingtonHouse position pending Andrew Bodman, Western Washington University Senator Karen Fraser Les Purce, The Evergreen State CollegeSenator Jeanne Kohl-Welles Robert Rosenman, Washington State UniversitySenator Pam Roach Ken Conte, House Office of Program ResearchSenator Mark Schoesler Richard Rodger, Senate Committee Services

3 of 25

Conduct a statistical analysis of WASL data to:

Increase understanding of students who did not meet standard,

Identify the characteristics of those students, and

Identify possible barriers to success.

[SSB 6618]

Evaluate the Promoting Academic Success program for students who did not meet standard.

[ESSB 6386]

Study Direction From the 2006 LegislatureStudy Direction From the 2006 Legislature

4 of 25

OutlineOutline

1. Overall WASL findings Strands

Subject-area results

Open-ended vs. multiple-choice

2. Characteristics of students who did not meet standard on the spring 2006 WASL

3. Evaluation of Promoting Academic Success (PAS) in summer 2006

Participation in PAS

Results from the August retake

5 of 25

Percentage of StudentsPercentage of Students Achieving Achieving Proficiency in MATH StrandsProficiency in MATH Strands

30%

35%

40%

45%

50%

55%

60%

65%

1999 2000 2001 2002 2003 2004 2005 2006

Content 1 Content 2Content 3 Content 4Content 5 Process 1Process 2 Process 3

6 of 25

The percentage of students who are proficient in reading and math strands varies considerably from one year to the next.

This variability diminishes the use of strand results to diagnose areas in need of improvement.

Variability in strand-level results does not diminish the overall reliability of the WASL.

Subject-Area Strand ResultsSubject-Area Strand Results

7 of 25

Correlations Among Reading, Writing, Correlations Among Reading, Writing, and Math Subject-Area Scoresand Math Subject-Area Scores

The correlations among subject-area results are strong:

0

No Association

1.0

Perfect Association

.10 .90.80.70.60.50.40.30.20

WRITINGand

MATH0.62

READING and

WRITING0.64

READINGand

MATH0.69

8 of 25

Average Math Score by Reading Score Average Math Score by Reading Score

300

350

400

450

500

550

300 350 400 450 500 550

Reading Score

Mat

h S

core

Math Score = Reading Score

10 point increase in reading = 7.5 point increase in math

Students who do well in math also do well in reading and writing, but the converse is not necessarily true:

9 of 25

Correlations Between Raw Scores on Correlations Between Raw Scores on Open-Ended and Multiple-Choice QuestionsOpen-Ended and Multiple-Choice Questions

0

No Association

1.0

Perfect Association

.10 .90.80.70.60.50.40.30.20

MATH0.83

READING0.65

10 of 25

Most students who do well on multiple-choice questions also do well on open-ended questions, and vice versa.

That is, multiple-choice and open-ended questions require a similar ability level, especially in math.

Performance on Open-Ended and Performance on Open-Ended and Multiple-Choice QuestionsMultiple-Choice Questions

11 of 25

Summary: Summary: Overall WASL FindingsOverall WASL Findings

• Strand results are unreliable and cannot be used to diagnose student performance

• Reading, writing, and math results are strongly correlated

• Raw scores on open-ended and multiple-choice questions are strongly correlated

12 of 25

Factors Affecting WASL PerformanceFactors Affecting WASL Performance

Student Characteristics

ClassroomInstruction

CurriculumAlignment

Student Performance

on WASL

ExtraAssistance

School Environment

UnknownFactors

13 of 25

Student Characteristics Associated With Student Characteristics Associated With WASL PerformanceWASL Performance

Lower met-standard rates

• Non-Asian minorities

• Poverty

• English language learners

• Students with disabilities

• Male students (reading and writing)

Higher met-standard rates

• Asian American

• European American

• Grade point average

• Parents’ educational attainment

• Male students (math)

14 of 25

WASL Met-Standard Rates by Race/EthnicityWASL Met-Standard Rates by Race/Ethnicity

88 91

7970

76

88 88

7567

75

63 61

3730 27

0

20

40

60

80

100

Asian(7.9%)

European(71.6%)

Native(2.5%)

Hispanic(10.3%)

African(4.8%)

Reading Writing Math

15 of 25

WASL Met-Standard Rates by PovertyWASL Met-Standard Rates by Poverty

91

73

89

7163

34

0

20

40

60

80

100

Not free or reduced lunch(70.5%)

Free or reduced lunch(29.5%)

Reading Writing Math

16 of 25

WASL Met-Standard Rates by WASL Met-Standard Rates by Primary Language SpokenPrimary Language Spoken

8882

53

65

8681

49

66

5763

17

37

0

20

40

60

80

100

English(91.4%)

Asian (1.9%)

Spanish(4.4%)

Other (2.3%)

Reading Writing Math

17 of 25

Relative Association Between WASL Met-Standard Relative Association Between WASL Met-Standard Rates and Students Grouped by CharacteristicsRates and Students Grouped by Characteristics

-9

-4

-9

-18

+23

+2

-6

-22

-31

Reading

+3Male

-9Hispanic American

-13African American

-16Poverty

+26

+5

-5

-8

-27

All Three

Asian

Parents’ Education

Native American

Non-English

Disabilities

+7

-9

-13

-16

+5

+26

-5

-7

-27

Math

-25

-3

-7

-17

+6

+21

-5

-21

-29

Writing

18 of 25

Factors Affecting WASL PerformanceFactors Affecting WASL Performance

Student Characteristics

ClassroomInstruction

CurriculumAlignment

Student Performance

on WASL

ExtraAssistance

School Environment

UnknownFactors

19 of 25

Participation in Summer 2006 PAS Participation in Summer 2006 PAS by Level of WASL Performanceby Level of WASL Performance

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

Reading Writing Math

Eligible (did not meet standard)

Level 2 PAS participants

Level 1 PAS participants

32,144

11,44710,137

20 of 25

Summer 2006 PAS: ReadingSummer 2006 PAS: Reading

Did not meet standard in spring 2006

(10,137)

Summer 2006 PAS

(897)

Retake August 2006

(532)

Met Std. August 2006

(221)

Met Std. August 2006

(586)

Retake August 2006

(1,333)

No summer 2006 PAS

(9,240)

8.8% of eligible

students

91.2% of eligible

students

59.3% of PAS

participants

41.5% of WASL

retakes

14.4% of non-

participants

44.0% of WASL

retakes

21 of 25

Summer 2006 PAS: WritingSummer 2006 PAS: Writing

Did not meet standard in spring 2006

(11,447)

Summer 2006 PAS

(1,035)

Retake August 2006

(602)

Met Std. August 2006

(394)

Met Std. August 2006

(1,032)

Retake August 2006

(1,655)

No summer 2006 PAS

(10,412)

9.0% of eligible

students

91.0% of eligible

students

58.2% of PAS

participants

65.4% of WASL

retakes

15.9% of non-

participants

62.4% of WASL

retakes

22 of 25

Summer 2006 PAS: MathSummer 2006 PAS: Math

Did not meet standard in spring 2006

(32,144)

Summer 2006 PAS

(4,218)

Retake August 2006

(2,986)

Met Std. August 2006

(843)

Met Std. August 2006

(1,412)

Retake August 2006

(5,817)

No summer 2006 PAS

(27,926)

13.1% of eligible

students

86.9% of eligible

students

70.8% of PAS

participants

28.2% of WASL

retakes

20.8% of non-

participants

24.3% of WASL

retakes

23 of 25

Percentage Who Met Standard on the Percentage Who Met Standard on the August 2006 WASL RetakeAugust 2006 WASL Retake

* Met-standard rates for non-participants adjusted using logistic regression

42%

65%

28%

42%

59%

22%

0

20

40

60

80

100

Reading (n.s.)

Writing (p<.01)

Math (p<.01)

PAS participants

Non-participants *

24 of 25

Percentage of Students By Level Who Met Percentage of Students By Level Who Met Standard on the August 2006 WASL RetakeStandard on the August 2006 WASL Retake

1724

4

45

72

39

15

31

3

47

64

31

0

20

40

60

80

100

Reading Writing Math

Level 1

Level 2

PAS Not-PAS PAS PASNot-PAS Not-PAS

24

72

15

47

25 of 25

Summary:Summary:Summer 2006 Promoting Academic SuccessSummer 2006 Promoting Academic Success

• Few students who were eligible for PAS in summer 2006 participated (approximately 1 in 10).

• Writing and Math: PAS participants outperformed non-participants by 6 percentage points on the August retake.

• Reading: PAS participants have same performance as non-participants on the August retake.