ANDREW PARR, WASHINGTON STATE BOARD OF EDUCATION GREG LOBDELL, CENTER FOR EDUCATIONAL EFFECTIVENESS...

A N D R E W PA R R , WA S H I N G T O N S TAT E B O A R D O F E D U C AT I O N

G R E G L O B D E L L , C E N T E R F O R E D U C AT I O N A L E F F E C T I V E N E S S

N AT I O N A L C O N F E R E N C E O N S T U D E N T A S S E S S M E N T

J U N E 2 4 , 2 0 1 5

Achievement of Former-ELL Students and their Impact on the Washington Achievement Index

Washington State Board of Education


Presentation Roadmap

Former ELL subgroup performance

Impact of the Former ELL subgroup on the Index

Preliminary findings from new data

English Language Learners (ELLs) and Former ELLs are the fastest growing subgroup in Washington - we will continue this research to

inform all who are working with these students.


Key Questions

Joint effort between the SBE and Greg Lobdell at the Center for Educational Effectiveness

1. What does the academic performance of the Former ELL subgroup look like through the Index?

2. How does the Former ELL subgroup impact the Index school ratings?

3. Are the earlier results being confirmed by the more recent analyses?


Achievement Index as a Data Source

Building is the unit of analysisContinuously enrolled students onlyPerformance Indicators:

Proficiency: reading, math, writing, science percent of students meeting standard

Median Student Growth Percentiles: reading and math Graduation Rate: 5-year Adjusted Cohort method


Washington Comprehensive Assessment Program

Through 2013-14 Grades 3-8 Measures of Student Progress in reading, math, science,

and writing Grade 10 high School Proficiency Exams in reading and writing End of Course assessments (Math 1, Math 2, & Biology) 200 to 600 point scale – 400 cut-score = Meets Standard

Beginning in 2014-15 Smarter Balanced Assessments Grades 3-8, ELA , math, and science Grades 5 & 8 Grade 10 HS SBAC ELA as a graduation requirement Grade 11 ELA, math, and Biology EOC


Achievement Index

School Index rating: proficiency and growth for elementary and middle schools Proficiency, growth, and CCR for high schools


Achievement Index

Washington State Board of Education 8

AI – Growth Measures


9


Disaggregated Subgroup Performance


DATA FROM THE INDEX

Former ELL Students


Former ELL Group Outperforms the ALL Group

60.2

%

63.9

%

63.8

%

20.7

%

23.2

%

26.9

%

61.9

%

67.5

%

71.4

%

0%10%20%30%40%50%60%70%80%90%

100%

2011 2012 2013 2011 2012 2013 2011 2012 2013

All Students ELL Students Former ELL Students

Math Proficiency: All Grades


Former ELL Math for Elementary Schools

Former ELL subgroup outperforms the All Students group

62.2

%

64.8

%

64.2

%

24.9

%

26.2

%

29.2

%

71.0

%

72.2

%

75.8

%

0%10%20%30%40%50%60%70%80%90%

100%

2011 2012 2013 2011 2012 2013 2011 2012 2013


Elementary Math


Former ELL Math for Middle Schools

Similar performance between the Former ELL group and the All Students group

55.6

%

58.6

%

59.1

%

12.1

%

14.9

%

16.7

%

51.8

%

55.9

%

58.1

%

0%10%20%30%40%50%60%70%80%90%

100%

2011 2012 2013 2011 2012 2013 2011 2012 2013


Middle School Math


Former ELL Math for High Schools

The Former ELL group underperforms compared to the All Students groups

66.8

% 75.4

%

79.1

%

21.1

%

19.0

%

37.0

%

55.3

% 70.5

%

76.9

%

0%10%20%30%40%50%60%70%80%90%

100%

2011 2012 2013 2011 2012 2013 2011 2012 2013


High School Math


Comparison Summary

-60%

-40%

-20%

0%

20%

40%

60%

Grad

e 3

Grad

e 4

Grad

e 5

Grad

e 6

Grad

e 7

Grad

e 8

Grad

e 10

Grad

e 3

Grad

e 4

Grad

e 5

Grad

e 6

Grad

e 7

Grad

e 8

Grad

e 10

Reading Math

3-Year Proficiency Gap: Former-ELL compared to ALL


Reading SGP Comparison

Elementary Middle High30

40

50

60

70

50.3 49.8 49.3

52.8 53.1

47.3

Reading3-Year Median Growth Percentile

All Students

Former ELLs

School Level

Me

dia

n S

GP


Math SGP Comparison

Elementary Middle High30

40

50

60

70

52.050.1

47.5

55.953.4

48.0

Math3-Year Median Growth Percentile

All Students

Former ELLs

School Level

Me

dia

n S

GP

High proficiency and high growth

Moderate proficiency and

high growth

Low proficiency and low growth


5-Year Graduation Rate

Former ELL Schools All Schools0

20

40

60

80

100

5-Year Graduation Rate

All Students

Former ELLs

Per

cen

t G

rad

uat

ing


IMPACT TO THE ACHIEVEMENT INDEX

Former ELL Students


How Many Schools Were Impacted?

Most schools benefited when the Former ELL group is included in the Index calculations.

Schools School Ratings

Year Rated Former ELL

No Change

Increased Decreased

2011 1817 553 8 513 32

2012 1861 753 6 717 30

2013 1886 819 9 770 37

Comp AI 1801 866 4 826 24


How Were the School Ratings Impacted?

A small average gain when the Former ELL group is used in the Index calculations.

2011 2012 2013 Comp AI

Average 1 2 3 4-0.25

0.25

0.75

1.25

1.75

Impact to the Index Ratings

Ind

ex

Ra

tin

g


How Were the Indicators Impacted?

Modest average rating point gains for all indicators when the Former ELL group in included.

Proficiency Growth Graduation

Average 1 2 3

-0.75

-0.25

0.25

0.75

1.25

1.75

Impact to the Indicators (2013 AI)

Ind

ica

tor

Ra

tin

g


Impact to Indicators by School Level

Elementary Schools Middle Schools

Proficiency Growth

Average 1 2

-0.75

-0.25

0.25

0.75

1.25

1.75

Impact to the Indicators (2013)

Ind

icat

or

Rat

ing

Proficiency Growth

Average 1 2

-0.75

-0.25

0.25

0.75

1.25

1.75

Impact to the Indicators (2013)

Ind

icat

or

Rat

ing


New Disaggregation Capacity

Historically, in the achievement records each student was ELL=Yes or No

In 2014 we have, for each student: Never-ELL ELL Transitioned within 2 years Transition more than 2 year ago

You probably have a pretty good guess as to how the groups’ performance compare?

3rd Grade Reading

ALL

Curre

nt E

LL

Never

ELL

Form

er E

LL ≤

2

Form

er E

LL >

20

10

20

30

40

50

60

70

80

90

100

Roadmap Districts3rd Grade Reading

Pe

rce

nt

Me

eti

ng

Sta

nd

ard

6th Grade Reading

ALL

Curre

nt E

LL

Never

ELL

Form

er E

LL ≤

2

Form

er E

LL >

20

10

20

30

40

50

60

70

80

90

100

Roadmap Districts6th Grade Reading

Pe

rce

nt

Me

eti

ng

Sta

nd

ard

10th Grade Reading

ALL

Curre

nt E

LL

Never

ELL

Form

er E

LL ≤

2

Form

er E

LL >

20

10

20

30

40

50

60

70

80

90

100

Roadmap Districts10th Grade Reading

Pe

rce

nt

Me

eti

ng

Sta

nd

ard


New Data Received in Late May 2015

2012-13 School Year Assessment Records Student Demographics SGP Records School Enrollment Records Special Programs Records WELPA Records ELL Status File

Data to still acquire Language level at entry TBIP program type

Student-levelPre-SBAC Field Test

Eliminates potential biasing from SBAC Field Test

Voluntary participation 1/3 of students/schools


Years in Bilingual Education

The most common home languages in Washington

On average: Asian students spend

fewer years in Bilingual Education

Hispanic/Latinos are among the highest on this measure

Language Years in ELL

Chinese-Mandarin 2.1Japanese 2.2Chinese-Unspecified 2.3Korean 2.4Chinese-Cantonese 2.7Tagalog 2.7Vietnamese 2.8Arabic 2.9Punjabi 3.0Cambodian 3.2Russian 3.2Somail 3.2Ukrainian 3.4Spanish 3.5


Former ELL Students by Home Language

Years in Bilingual Education is important

15 percent of Spanish speaking Former ELL students spent more than 5 years in Bilingual Education.

58 percent of Korean speaking Former ELL students spent less than 2 years in Bilingual Education.

Home Language

% by LanguageTotal

Former ELL

Short-Term ELL

Long-Term ELL

Spanish 58.7 27.7 15.3Vietnamese 6.5 44.5 6.9Russian 6.5 31.6 9.2Ukrainian 3.6 25.2 11.1Korean 2.4 58.3 3.3Tagalog 2.0 50.4 6.4Chinese-Cantonese 1.7 50.5 6.1Somali 1.5 36.1 12.7Punjabi 1.5 40.3 8.9Cambodian 1.2 35.6 10.7Chinese-Mandarin 1.0 69.0 3.0Chinese-Unspecified 1.0 61.5 2.6Arabic 0.9 42.2 9.3Japanese 0.8 64.5 1.9Amharic 0.6 54.2 3.9


More on Long-Term ELLs

Long-Term ELLs are more likely to qualify for FRL and participate in Special Education.

Student Characteristic

MeanP

Not Long-Term ELL Long-Term ELLFRL Eligibility 77.9 89.8 <0.001Special Ed 7.3 24.3 <0.001Migrant 6.1 8.9 <0.001Homeless 2.3 1.8 <0.001Gifted 1.0 0.1 <0.001Title III Immigrant 14.8 8.1 <0.001Section 504 1.0 1.0 0.281


Former ELL Performanceby Long-Term ELL Status

Former ELL=Yes and SWD=No

Former ELL=Yes, SWD=No, FRL=Yes, & Home Language=Spanish

Reading SS Math SS

Math EOC1 SS

Reading % Proficient

Math % Proficient

Not Long-Term ELL 414.9 421.3 414.2 82.4% 73.5%

Long-Term ELL 399.9 391.4 385.1 63.5% 52.0%

Reading SS Math SS

Math EOC1 SS

Reading % Proficient

Math % Proficient

Not Long-Term ELL 409.6 408.5 400.0 76.2% 64.3%

Long-Term ELL 398.8 388.5 381.8 61.1% 48.5%


Former ELL – Never ELL Scaled Score Performance Gap

In reading, the Former ELL group outperforms the Never ELL group up to the 5th grade and underperforms after that. See the higher performance in all grades for math.

Gra

de 3

Gra

de 4

Gra

de 5

Gra

de 6

Gra

de 7

Gra

de 8

Gra

de 1

0

Gra

de 3

Gra

de 4

Gra

de 5

Gra

de 6

Gra

de 7

Gra

de 8

Gra

de 1

0

-10

-5

0

5

10

15

20

2013 Scaled Score Performance Gap (MSP/HSPE) Former ELL vs. Never ELL

Scal

ed S

core

Poi

nts

Reading Math


Former ELL – Never ELLProficiency Performance Gap

At all grade levels, the proficiency rate in math for the Former ELL group exceeds the rate for the Never ELL group. See the lower proficiency rates in reading beginning in the 8th grade.

Gra

de 3

Gra

de 4

Gra

de 5

Gra

de 6

Gra

de 7

Gra

de 8

Gra

de 1

0

Gra

de 3

Gra

de 4

Gra

de 5

Gra

de 6

Gra

de 7

Gra

de 8

Gra

de 1

0

-5%

0%

5%

10%

15%

20%

2013 Proficiency Performance Gap (MSP/HSPE)Former vs. Never ELL

Perc

enta

ge P

oint

s

Reading Math


Scaled Score Gap Proficiency Gap

Former ELL – Never ELL Math EOC 1 Performance Gaps

Gra

de 7

Gra

de 8

Gra

de 9

Gra

de 1

0

-4.0

-2.0

0.0

2.0

4.0

6.0

8.0

2013 Scaled Score Gap Math EOC 1Former ELL vs. Never ELL

Scal

ed S

core

Poi

nts

Gra

de 7

Gra

de 8

Gra

de 9

Gra

de 1

0

-4%

-2%

0%

2%

4%

6%

2013 Proficiency Gap Math EOC 1Former ELL vs. Never ELL

Perc

enta

ge P

oint

s


FELL 0 FELL 1 FELL 2 FELL 3 All FELL Never ELL350

375

400

425

450

3rd Grade Reading and Math Scaled Score AveragesFormer ELL vs. Never ELL

Read SS Math SS

ELL Status

Scal

ed S

core

3rd Grade Performance

Controlling for SWD (No) and FRL (Yes) status

All Former ELL groups perform as well or better than the Never ELL group

FELL 0 FELL 1 FELL 2 FELL 3 All FELL Never ELL0%

20%

40%

60%

80%

100%

3rd Grade Reading and Math Proficiency AveragesFormer ELL vs. Never ELL

READ_PRO MATH_PRO

ELL Status

Perc

ent M

eetin

g St

anda

rd


4th Grade Performance

The Former ELL groups outperform the Never ELL group in Math

All Former ELL groups perform as well or better than the Never ELL group

FELL 0 FELL 1 FELL 2 FELL 3 FELL 4 All FELL Never ELL375

400

425

450

475

4th Grade Reading and Math Scaled Score AveragesFormer ELL vs. Never ELL

Read SS Math SS

ELL Status

Scal

ed S

core

FELL 0 FELL 1 FELL 2 FELL 3 FELL 4 All FELL Never ELL

0%

20%

40%

60%

80%

100%

4th Grade Reading and Math Proficiency AveragesFormer ELL vs. Never ELL

READ_PRO MATH_PRO

ELL Status

Perc

ent M

eetin

g St

anda

rd



Three years after reclassifying, Former ELL students outperform the Never ELL group.


FELL 0 FELL 1 FELL 2 FELL 3 FELL 4 FELL 5 All FELL Never ELL

350

375

400

425

450


Read SS Math SS

ELL Status

Scal

ed S

core

FELL 0 FELL 1 FELL 2 FELL 3 FELL 4 FELL 5 All FELL Never ELL

0%

20%

40%

60%

80%

100%


READ_PRO MATH_PRO

ELL Status

Perc

ent M

eetin

g St

anda

rd




In the third year after reclassification, Former ELL student groups outperform the Never ELL student group.

FELL 0 FELL 1 FELL 2 FELL 3 FELL 4 FELL 5 FELL 6 All FELL Never ELL

350

375

400

425

450


Read SS Math SS

ELL Status

Scal

ed S

core

FELL 0 FELL 1 FELL 2 FELL 3 FELL 4 FELL 5 FELL 6 All FELL

Never ELL

0%

20%

40%

60%

80%

100%


READ_PRO MATH_PRO

ELL Status

Perc

ent M

eetin

g St

anda

rd




Three years after reclassification, Former ELL students perform as well or better than Never ELL students.

FELL 0

FELL 1

FELL 2

FELL 3

FELL 4

FELL 5

FELL 6

FELL 7

All FELL

Never E

LL375

400

425

450

475


Read SS Math SS

ELL Status

Scal

ed S

core

FELL 0

FELL 1

FELL 2

FELL 3

FELL 4

FELL 5

FELL 6

FELL 7

All FELL

Never E

LL0%

20%

40%

60%

80%

100%


READ_PRO MATH_PRO

ELL Status

Perc

ent M

eetin

g St

anda

rd




Five years after reclassification, Former ELL students begin to outperform Never ELL students.

FELL 0

FELL 1

FELL 2

FELL 3

FELL 4

FELL 5

FELL 6

FELL 7

All FELL

Never E

LL350

375

400

425

450


Read SS Math SS

ELL Status

Scal

ed S

core

FELL 0

FELL 1

FELL 2

FELL 3

FELL 4

FELL 5

FELL 6

FELL 7

All FELL

Never E

LL0%

20%

40%

60%

80%

100%


READ_PRO MATH_PRO

ELL Status

Perc

ent M

eetin

g St

anda

rd


Former ELL vs. Never ELL

Former and Never ELL SWD = No FRL = Yes

Former ELL Students Reclassified in 1st Grade Home Language = Spanish

All things being equal, the Former ELL group reclassified

in the 1st grade achieves a higher average scaled score

on all assessments.

Reading Scaled Score

Math Scaled Score

GradeTested

FELL Years

Former ELL

Never ELL

Former ELL

Never ELL

3 FELL 2 421.7 411.8 415.5 406.7

4 FELL 3 412.9 407.6 429.1 409.8

5 FELL 4 419.3 411.7 420.7 406.3

6 FELL 5 413.1 408.7 413.5 402.9

7 FELL 6 408.6 407.6 417.8 410.5

8 FELL 7 411.0 405.0 408.0 396.4


Preliminary Data

FRL=Yes, SWD=No, Home Language=Spanish

FELL 2 FELL 3 FELL 4 FELL 5 FELL 6 FELL 7MSP 3 MSP 4 MSP 5 MSP 6 MSP 7 MSP 8

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2012-13 Reading Proficiency Rate by ELL Status and Years in Bilingual Education Program (Reclassified in 1st Grade)

ELL 0.9 Yrs ELL 1.9 Yrs ELL 2.9 Yrs Never ELL

Perc

ent M

eetin

g St

anda

rd


Same Preliminary Data


FELL 2 FELL 3 FELL 4 FELL 5 FELL 6 FELL 7MSP 3 MSP 4 MSP 5 MSP 6 MSP 7 MSP 8

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2012-13 Reading Proficiency Rate by ELL Status and Years in Bilingual Education Program (Reclassified in 1st Grade)

ELL 0.9 Yrs ELL 1.9 Yrs ELL 2.9 Yrs Never ELL

Perc

ent M

eetin

g St

anda

rd


FELL 1 FELL 2 FELL 3 FELL 5MSP 6 MSP 7 MSP 8 HSPE 10

0%10%20%30%40%50%60%70%80%90%

100%

2012-13 Reading Proficiency Rate by ELL Status and Years in Bilingual Education Program (Reclassified in 5th Grade)

ELL 0.9 Yrs ELL 1.9 Yrs ELL 2.9 Yrs ELL 3.9 Yrs ELL 4.9 Yrs ELL 5.9 Yrs Never ELL

Perc

ent M

eetin

g St

anda

rdEnd of 5th Grade Reclassification


Former ELL Average = 50%



End of 7th Grade Reclassification


FELL 1 FELL 3MSP 8 HSPE 10

0%10%20%30%40%50%60%70%80%90%

100%

Reading Proficiency Rate by ELL Status and Years in Bilingual Education Program (Reclassified in 7th Grade)

ELL 0.9 Yrs ELL 1.9 Yrs ELL 2.9 Yrs ELL 3.9 YrsELL 4.9 Yrs ELL 5.9 Yrs ELL 6.9 Yrs Never ELL

Perc

ent M

eetin

g St

anda

rd Former ELL Average = 37%



Reading Student Growth

26.5% 28.3%32.5% 30.4% 29.7% 30.7% 31.7% 33.9% 33.5%

40.4%37.0% 34.7% 35.0% 36.2% 36.7% 36.1%

31.7% 33.5%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Former ELL: 0 years

Former ELL: 1 years

Former ELL: 2 years

Former ELL: 3 years

Former ELL: 4 years

Former ELL: 5 years

Former ELL: 6 years

Former ELL: 7 years

Never ELL

Reading Growth Levels

Low Growth Med Growth High Growth


Math Student Growth

25.2%29.2% 29.6% 28.5% 28.9% 30.7% 30.4% 32.0% 33.9%

43.4%38.7% 37.5% 38.3% 38.0% 36.4% 35.5% 37.0%

33.1%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Former ELL: 0 years

Former ELL: 1 years

Former ELL: 2 years

Former ELL: 3 years

Former ELL: 4 years

Former ELL: 5 years

Former ELL: 6 years

Former ELL: 7 years

Never ELL

Math Growth Levels



Reading Student Growth

31.6% 29.9%34.4% 33.5%

34.9% 36.5%32.2% 33.7%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Former_Elem Former_Middle Former_High Never

Reading Growth Levels by Grade Level



Math Student Growth

27.2%31.1% 33.5%

39.7% 36.1% 33.7%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Former_Elem Former_Middle Never

Math Growth Levels by Grade Level



Accountability

Allows Former ELL students to be included in the ELL student group for up to two years after reclassification.

The underlying assumption is that Former ELLs for two years are expected to perform similarly to but higher than Never ELL students (which is not necessarily true in all cases)

Our work shows: many Former ELL student groups do not perform similarly to Never ELL students for many

years and some never reach the levels of the Never ELL group

Washington is looking for new ELL accountability measures

How do your states include Former ELL students in accountability?


TBIP Task Force

Transitional Bilingual Instructional Program (TBIP) accountability

Just finished Phase I work with recommendations forthcoming

Expecting recommendations to enhance district- and school-level TBIP effectiveness Predictive analytics for ELLs progressing toward reclassification Monitoring ELL and Former ELL Learning Assistance Program (LAP)


Policy Implications

Schools with reportable Former ELL groups are demographically different and those differences are reflected in student outcome

measures.

Do you think current funding and staffing policy effectively addresses these differences?

Do you think current PD policy is adequate to address the learning needs of this diverse group?

Do you think that expanded learning opportunities would benefit this diverse group?

Other thoughts?


Ongoing Research

Academic performance disaggregated deeper by Primary language Years in (and out of) TBIP program TBIP model Socioeconomic status Other student characteristics (mobility, migrant, etc.)

Potential or Potential ImpactsTBIP accountabilityWashington Achievement AwardsGap reduction work Index revisions


Questions

For questions and other information, contact:

[email protected]

[email protected]

mailto:[email protected]

mailto:[email protected]

ANDREW PARR, WASHINGTON STATE BOARD OF EDUCATION GREG LOBDELL, CENTER FOR EDUCATIONAL EFFECTIVENESS...

Documents

Transcript of ANDREW PARR, WASHINGTON STATE BOARD OF EDUCATION GREG LOBDELL, CENTER FOR EDUCATIONAL EFFECTIVENESS...