Do Race/Ethnicity-Based Student Achievement Gaps Grow …So, what do value-added models tell us...

Do race/ethnicity-based

student achievement gaps grow over time?

ACT Research and Policy

Research ReportsEducator Reports

Policy ReportsIssue/Information Briefs

March 2012Issue Brief

www.act.org/research

© 2012 by ACT, Inc. All rights reserved. 18035

Last year, Hispanic and African American high school graduates met ACT’s College Readiness Benchmarks in English, reading, mathematics, and science at substantially lower rates than did Asian and White graduates.1 For example, the rate of White graduates who were college ready in English (77%) was twice that of their African American peers (35%). Unfortunately, evidence of such academic achievement gaps—as measured by College Readiness Benchmarks and other measures of academic achievement—is all too common and persistent.2

While much attention has been paid to achievement gaps, growth gaps have been largely overlooked. This brief investigates race/ethnicity-based growth in achievement gaps for ACT-tested 2011 high school graduates. First we look at growth in achievement gaps among the four largest racial/ethnic groups (i.e., African American, Asian, Hispanic, and White), and then highlight the gaps of the two largest racial/ethnic minorities in the United States—African American and Hispanic students.3

Two methods of calculating academic growth are used in this brief, each measuring growth in academic achievement by race/ethnicity across three aligned and standardized academic achievement assessments—EXPLORE®, PLAN®, and the ACT®. 4 In Part 1, growth is measured simply as the difference between the test scores taken between 8th and 12th grades for each of four subjects: English, reading, mathematics, and science. Part 2 looks at growth using a value-added model. The value-added approach accounts for student background and prior achievement levels to compare growth across racial/ethnic groups, so that the gaps reflect differences in relative achievement growth during high school.

Data are from a sample of 123,541 students in the high school graduating class of 2011 who took EXPLORE in 8th grade, PLAN in 10th grade, and the ACT in 11th or 12th grade. (See Table A1 in the Appendix for more information on the sample used).5

Issue Brief

2

Part 1: Finding achievement gaps between minority subgroups—A simple descriptive approach

Our first analysis—the descriptive approach—helps us understand the nature of the race/ethnicity gaps.

The achievement gaps for all subgroups, found in Figure 1 below,6 show two important findings: the gaps begin early and grow over time.

1. Academic achievement gaps are evident before high school.

· Asian students score the highest in all four subjects in grade 8, with White students close behind.

· Science, on average, shows the smallest gap between the subgroups.

2. The gaps widen over time.

· Asian students start out with the highest scores and show the largest gains across grades in all four subjects.

· African American students exhibit the opposite pattern. They start with the lowest scores and show the smallest gains.

Bottom line: Asian and White

students start with the highest

scores and grow at the fastest

pace; African American and

Hispanic students start with the

lowest scores and grow at the

slowest pace.7 These trends are

consistent across English, reading,

mathematics, and science.

Figure 1: Average EXPLORE, PLAN, and ACT Scores, by Race/Ethnicity Asian White Hispanic African American ACT College Readiness Benchmarks

12.0

16.0

20.0

24.0

28.0

Grade 8 Grade 10 Grade 11 or 12

Sco

re

English

12.0

16.0

20.0

24.0

28.0


Sco

re

Reading

12.0

16.0

20.0

24.0

28.0


Sco

re

Mathematics

12.0

16.0

20.0

24.0

28.0


Sco

re

Science

Issue Brief March 2012

3

Taking a closer look at the achievement gap between African American and Hispanic students, we find:

1. This gap is evident before high school, with Hispanic students performing slightly better on average than African American students.

2. Hispanic students also make more progress over time.

· The gaps in mathematics, reading, and science accelerate over time, while it widens at a more constant rate in English.

3. On average, neither Hispanic nor African American students meet the ACT College Readiness Benchmarks.8

· The one exception was that the average for Hispanic students just met the English Benchmark.

Part 2: Understanding growth gaps between minority subgroups—A value-added approach

You might be wondering, why bother with a value-added model? After all, didn’t the Part 1 analysis just show that these gaps are real? While simple gains are informative, a better approach for understanding growth differences between racial/ethnic groups is to use value-added models. Why? Because unlike the simple descriptive approach taken in the first analysis, value-added models can account for characteristics of students or schools to help understand what might lead to such gaps (e.g., race/ethnicity, family income, prior achievement level in all subject areas, a high-performing school).

So, because of their ability to help us understand why these gaps are occurring, we also examined the results of a value-added model predicting achievement scores while accounting for 1) all four earlier subject scores, 2) the span in months between the two tests, 3) parent income, and 4) parent education.9

The value-added model provides some additional insight into race/ethnicity-based academic achievement gaps by statistically controlling for the variables listed above. Figure 2 shows the model’s average value-added estimates from EXPLORE to PLAN and PLAN to the ACT, by race/ethnicity. The average value-added scores represent the extent to which each student subgroup’s average growth differs from the average growth among all students; value-added growth scores above 0.0 indicate above average growth from one assessment to the next, while value-added growth scores below 0.0 indicate below average growth.

Bottom line: A slight Hispanic–

African American achievement

gap is apparent in 8th grade that

widens over time. Nevertheless,

average achievement for both

groups is not on par with college

readiness levels.

Issue Brief

4

Bottom line: Hispanic and African

American student growth is

below average, and considerably

lower than that of Asian

students, even when accounting

for prior academic achievement

in all subject areas, length of

time between assessments,

and student background

characteristics.

So, what do value-added models tell us about race/ ethnicity-based academic achievement gaps?

With the value-added model, we find that overall:

1. The relative growth of students differs by race/ethnicity, even when accounting for the four factors listed above.

· Asian (+0.2 to 1.1 points, depending on subject) and White (+0 to 0.1) students show greater growth relative to average growth across the entire sample.

· African American (-0.4 to -0.2) and Hispanic (-0.2 to -0.1) students show below average growth.

2. The differences in the relative growth of students grow larger over time.

· African American students’ relative growth values from EXPLORE to PLAN are approximately 0.2 points below average, whereas their relative growth values from PLAN to the ACT are approximately 0.3–0.4 points below average. This pattern appears to hold for Hispanic and White students as well.

· Asian students, however, continue to show above-average relative growth rates over time.

Figure 2. Value-Added Average Growth from EXPLORE to PLAN and PLAN to ACT, by Race/Ethnicity Asian White Hispanic African American

-0.4

-0.2

0.0

0.2

0.4

0.6

EXPLORE to PLAN PLAN to ACT

Ave

rag

e G

row

th

English

-0.6

-0.4

-0.2

0.0

0.2

0.4


Ave

rag

e G

row

th

Reading

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

1.2


Ave

rag

e G

row

th

Mathematics

-0.4

-0.2

0.0

0.2

0.4

0.6


Ave

rag

e G

row

th

Science


5

Bottom line: Hispanic students

grew more than African American

students, even when accounting

for other background factors.

Results from the value-added models are consistent with the descriptive results and suggest that parent’s income and education levels also explain some of the differences in student growth.10

With respect to the African American-Hispanic achievement gap, our analysis indicates:

1. The African American-Hispanic achievement gap is evident before 8th grade and continues through 12th grade.

2. This growth gap widens in later grades as Hispanic students’ growth rates are higher relative to those of African American students.

3. Some, but not all, of the difference in growth is related to differences in family income and levels of parental education.

How might these gaps be closed? Policy and practice suggestions for boosting the academic achievement of all students

This analysis indicates a seemingly intractable aspect of academic achievement gaps—higher entering achievement levels are related to higher growth rates, resulting in greater race/ethnicity-based achievement gaps over time. Despite family income and parental education levels being related to differences in student academic achievement, these factors inherently do not suggest an immediately workable set of policy or practice recommendations. In fact, the reasons for such academic gaps are numerous and, admittedly, are not all explored within this brief analysis (e.g., student course-taking patterns, educational and career aspirations, and other institutional factors). Nevertheless, the following may be some approaches schools, states, parents, and students can take to increase student academic achievement, and, perhaps, shrink or close the academic achievement gaps.

1. High expectations for all: Establish and uphold the same high academic expectations for all students, regardless of their initial level of academic achievement, post-high school plans, or demographics. All students have the right to expect that, provided they put in the work, what they learn throughout their K–12 careers will prepare them adequately with the skills and knowledge needed to be successful after graduation.

2. Early monitoring: Effective formative and summative assessments can help K–12 educators identify when and with which learning topics students begin to fall behind. Combined with in-class teacher observations and other assessments, such monitoring tools should be used regularly so that academic achievement gaps for individual students can be noticed and rectified as early as possible.

Issue Brief

6

3. Targeted interventions: Because there are likely many diverse reasons why individual students fall behind academically, the interventions for closing individual learning gaps must also be varied. Applying targeted interventions at the individual student level, based on the specific reasons each student falls behind, has the potential to realize the most effective outcomes for individual students.

4. Minimize summer learning loss: There are indications that academic achievement gaps widen over the summer, perhaps because disadvantaged students have less access to quality educational resources outside of school.11 Schools, districts, communities, and states can examine, develop, implement, and support summer learning opportunities targeted precisely for lower achieving students as a way to slow the growth of academic achievement gaps.

Appendix

Table A1 contains the number, percentages, and average ACT scores of the ACT-tested graduating class of 2011 by subject area and by race/ethnicity. Table A2 is a subset of the students in Table A1 who also took EXPLORE in 8th grade and PLAN in 10th grade, and was the sample of interest in this study. Comparison of Tables A1 and A2 indicate that the study sample had slightly higher scores, a higher proportion of White students, and a lower proportion of Hispanic students than the ACT-tested graduating class of 2011. The differences were not large, indicating that the two groups of students are reasonably similar.

Table A1. N-Counts, Percentages and Average ACT Scores, by Race/Ethnicity

All Students

White

African American

Hispanic

Asian

N 1,623,112 981,585 223,383 200,661 66,650

Percent 65.0 13.8 12.4 4.1

English 20.6 22.2 15.9 17.6 22.8

Reading 21.3 22.7 17.1 18.9 23.0

Mathematics 21.1 22.1 17.2 19.2 25.1

Science 20.9 22.1 17.2 18.8 23.1

Note: The percentages do not sum to 100% because not all racial/ethnic groups are represented.


7

Table A2. Average EXPLORE, PLAN, and ACT Scores and Score Gains, by Race/Ethnicity

All Students

White

African American

Hispanic

Asian

N 123,541 80,362 16,562 11,288 4,264

Percent 60.5 13.4 9.1 3.5

Mean Gain Mean Gain Mean Gain Mean Gain Mean Gain

English

EXPLORE 15.7 16.4 13.5 13.9 16.7

PLAN 18.1 2.3 18.8 2.4 15.6 2.1 16.3 2.4 19.6 2.9

ACT 21.3 3.2 22.4 3.6 17.3 1.7 18.3 2.0 24.7 5.0

Total Gain 5.6 6.0 3.8 4.4 8.0

Reading

EXPLORE 15.5 16.1 13.3 14.2 16.4

PLAN 17.8 2.3 18.5 2.4 15.2 1.9 16.0 1.9 19.5 3.0

ACT 21.5 3.7 22.6 4.0 17.7 2.4 19.0 2.9 24.1 4.7

Total Gain 6.0 6.5 4.4 4.8 7.7

Mathematics

EXPLORE 16.5 17.1 14.3 15.2 18.2

PLAN 18.8 2.3 19.5 2.5 15.8 1.6 17.1 1.8 21.7 3.5

ACT 21.2 2.4 22.1 2.6 17.5 1.7 19.2 2.2 25.5 3.8

Total Gain 4.8 5.1 3.3 4.0 7.3

Science

EXPLORE 17.3 17.8 15.5 16.3 18.4

PLAN 19.1 1.7 19.7 1.9 16.7 1.1 17.6 1.3 20.9 2.5

ACT 21.4 2.3 22.3 2.6 17.9 1.2 19.3 1.7 24.4 3.5

Total Gain 4.1 4.5 2.4 3.0 5.9

Note: The percentages do not sum to 100% because not all racial/ethnic groups are represented. Gains may not sum to total gain due to rounding.

Issue Brief

8

Table A3 provides a closer look at the achievement gap between African American and Hispanic students. Table A3 contains the differences in mean EPAS scores between African American and Hispanic students, and the corresponding differences in gains between tests. Comparison of the differences in means reveals that the Hispanic-African American achievement gap increased over time across all four subject areas. Comparison of the differences in gains shows that, with the exception of English, not only was the gap increasing, but it was increasing at a greater rate over time. In English, the gap was increasing, but at a constant rate over time.

Table A3. Gap between African American and Hispanic Students from EXPLORE to PLAN and PLAN to ACT

Difference in Means Hispanic—African American

Difference in Gains Hispanic—African American

English

EXPLORE 0.4

PLAN 0.7 0.3

ACT 1.0 0.3

Total Gain Difference 0.6

Reading

EXPLORE 0.9

PLAN 0.8 -0.1

ACT 1.3 0.5


Mathematics

EXPLORE 1.0

PLAN 1.2 0.3

ACT 1.7 0.5


Science

EXPLORE 0.7

PLAN 0.9 0.2

ACT 1.3 0.4


Note: Positive means and gains indicate scores were higher for Hispanic students than for African American students. Gains may not sum to total gain due to rounding.


9

1 ACT, The Condition of College & Career Readiness (Iowa City, IA: Author, 2011), 5.

2See, for example, Thomas D. Snyder and Sally A. Dillow, “Percentage of students at or above selected reading score levels, by age, sex, and race/ethnicity: Selected years, 1971 through 2008 [Table 128],” in Digest of Education Statistics 2010, (Washington, DC: National Center for Educa-tion Statistics, 2011), 195; Thomas D. Snyder and Sally A. Dillow, “Percentage of students at or above selected mathematics proficiency levels, by age, sex, and race/ethnicity: Selected years, 1971 through 2008 [Table 141],” in Digest of Education Statistics 2010, (Washington, DC: National Center for Education Statistics, 2011), 209; Susan Aud, Mary Ann Fox, and Angelina KewalRamani, Status and Trends in the Education of Racial and Ethnic Groups (Washington, DC: U.S. Department of Education, National Center for Education Statistics, 2010).

3Karen R. Humes, Nicholas A. Jones, and Roberto R. Ramirez, An Overview: Race and Hispanic Origin and the 2010 Census (March 2011), http://2010.census.gov/2010census

4Cross-sectional achievement gap studies can examine differences in test scores for different cohorts of students (e.g., is the achievement gap in 4th grade mathematics decreasing over time?). However, longitudinal analyses of students over time can provide information about the presence and magnitude of achievement gaps as the same students progress through grades (e.g., does the achievement gap in mathematics decrease between 4th grade and 8th grade?). For example, two important findings come out of a Phillips, Crouse, and Ralph (1998) longitudinal study that in-vestigated changes in the Black-White achievement gaps. Not only do Black students start school academically behind their White peers, but they continue to fall further behind as they progress through school. See Meredith Phillips, James Crouse, and John Ralph, “Does the Black-White test score gap widen after children enter school?,” in The Black-White Test Score Gap, eds. Christopher Jencks and Meredith Phillips (Washington D.C.: The Brookings Institution Press, 1998), 229–272.

5The sample of students used is a subset of the ACT-tested high school class of 2011. Students were excluded in our analysis if they attended a high school for which there were fewer than 30 students or less than 50% of the graduating class of 2011 took all three assessments. American Indian and Pacific Islander subgroups are not presented due to small sample sizes. The ACT-tested graduating class of 2011 consisted of 1,623,112 students with an average ACT Composite of 21.1; the EPAS sample consisted of 123,541 of those students and had an average ACT Composite of 21.5. While students who took all three assessments were similar to the ACT-tested popula-tion, both groups likely differ to some extent from the overall United States K-12 population. More information on EXPLORE, PLAN, and the ACT can be found in their technical manuals available at: http://www.act.org/research/researchers/techmanuals.html

6For a more detailed description, see Tables A2 and A3 in the Appendix.

7The average span in months between tests were similar from EXPLORE to PLAN (23 months) and PLAN to the ACT (22 months); although the time span between PLAN and the ACT was more vari-able than the span from EXPLORE to PLAN.

8ACT, What are ACT’s College Readiness Benchmarks? (Iowa City, IA; Author, 2011).

9Value-added scores at the individual level can be interpreted as the estimated difference between a given student’s actual and expected test scores, where expected scores are based on the relationship between prior and current scores observed in the overall sample. The average value-added scores across students within each race/ethnicity subgroup represent the extent to which each group’s average growth differs from the average growth across all students. By definition, the overall average residual for each model is zero. A random intercept model was used to account for variation in school effects on growth in the value-added model.

10 Data were only available from 8th grade to the time that students last took the ACT in 11th or 12th grade; longitudinal data following students from elementary school through high school would provide a fuller picture of the achievement gaps and growth rates of students from different racial/ethnic backgrounds.

11Harris Cooper, K. Charlton, J.C. Valentine, and L. Muhlenbruck, “Making the most of summer school. A meta-analytic and narrative review,” Monographs of the Society for Research in Child Development 65, 1, no. 260 (2000): 1–118.

ACT Research and Policy www.act.org/research

ACT is an independent, not-for-profit organization that provides assessment, research, information, and program management services in the broad areas of education and workforce development. Each year we serve millions of people in high schools, colleges, professional associations, businesses, and government agencies, nationally and internationally. Though designed to meet a wide array of needs, all ACT programs and services have one guiding purpose—helping people achieve education and workplace success.

Do Race/Ethnicity-Based Student Achievement Gaps Grow …So, what do value-added models tell us...

Documents

Transcript of Do Race/Ethnicity-Based Student Achievement Gaps Grow …So, what do value-added models tell us...