The Bubble Bursts - CPRE : Consortium for Policy Research · PDF file ·...

Working Paper September 2016 WP #2016–8

The Bubble BurstsTHE 2015 OPT-OUT

MOVEMENT IN

NEW JERSEY

Jonathan SupovitzFrancine StephensJulie KubelkaPatrick McGuinn Hannah Ingersoll

THE 2015 OPT-OUT MOVEMENT IN NEW JERSEY

About the Consortium for Policy Research in Education

The Consortium for Policy Research in Education (CPRE) brings together education experts from renowned research institutions to contribute new knowledge that informs PK–20 education policy and practice. Our work is peer-reviewed and open-access. Our institutions include:

University of Pennsylvania

Teachers College, Columbia University

Harvard University

Stanford University

University of Michigan

University of Wisconsin-Madison

Northwestern University

The Consortium for Policy Research in Education (CPRE) is headquartered at the Graduate School of Education at University of Pennsylvania.

Stay connected and join the conversation online!Consortium for Policy Research in Education

@CPREresearch | CPRE.ORG

Subscribe to CPRE Insights eNewsletter: bit.ly/CPREinsights

Jonathan SupovitzFrancine StephensJulie KubelkaPatrick McGuinn Hannah Ingersoll

The Bubble BurstsTHE 2015 OPT-OUT MOVEMENT IN NEW JERSEY

Working Paper September 2016 WP #2016–8

IV | CONSORTIUM FOR POLICY RESEARCH IN EDUCATION | CPRE.ORG


AbstractThe Bubble Bursts: The 2015 Opt-Out Movement in New Jersey analyzes the scope, factors, and context of the opt-out movement that occurred in New Jersey in the spring of 2015. Using test participation data released in February 2016 by the New Jersey Department of Education, we found that approximately 135,000 students did not take the state assessment in the spring of 2015. Depending on how it was calculated, this represented between 11-19% of the population of students eligible for testing in grades 3 to 11 in the state. There was also a positive correlation between higher district opt-out rates and wealthier districts. We found that several factors contributed to these trends. Predominant amongst these were an accumulated skepticism with high stakes testing in general and the new PARCC assessment in particular, concerns from the Common Core State Standards rollout, teacher union opposition to premature teacher accountability, and confusion in the messages of state policymakers about graduation requirements. These explanatory factors were based upon interviews with over 30 state policymakers, professional education association representatives, advocacy group leaders, school administrators, teachers, parents, and students.

FundingThis project received no external funding from any source.

Suggested CitationSupovitz, J., Stephens, F., Kubelka, J., McGuinn, P., & Ingersoll, H. (September, 2016). The Bubble Bursts: The 2015 Opt-Out Movement in New Jersey. Working Paper (#WB 2016–8). Philadelphia, PA: Consortium for Policy Research in Education, University of Pennsylvania.

The views and opinions expressed in this report are solely those of the author and other contributors. These views and opinions do not necessarily represent those of the Consortium for Policy Research in Education (CPRE).

Copyright © 2016 by CPRE. All rights reserved.

For informaiton about CPRE and copies of our publications, please visit: cpre.org.

AcknowledgementsThis paper would not be what it is without invaluable contributions from many people. Bobbi Newman at CPRE was an instrumental partner in this work, conducting interviews, managing the project, and commenting on drafts. Many thanks to Garrett Baker and Oluwatosin Shenbanjo who helped coordinate interviews and transcripts. A very special thanks to Bridget Goldhahn for providing her design talent to the report layout and design.


Contents

I. Executive Summary ................................................................................................................................................ 1

II. Overview .................................................................................................................................................................. 3

Report Organization ........................................................................................................................................... 4

III. The Magnitude of Opt-Outs in New Jersey in 2015 ............................................................................................. 5

Overall Opt-Out Rates by Subject .................................................................................................................... 6

Opt-Out Rates in Mathematics by District Factor Group ............................................................................... 7

Opt-Out Rates Merging in District Enrollment Figures ..................................................................................... 8

Correlations Between Opt-Out Rates and District Factor Groups .............................................................. 10

IV. Opt-Out Backdrop: National Education Policy Maelstrom Feeds Testing Consternation ............................. 11

The Common Core and the Backlash Against Common Standards ......................................................... 17

National Government Efforts to Push State Implementation of Standards, Tests, and Accountability .. 13

V. New Jersey State Factors that Contributed to the Spring 2015 Climate ......................................................... 14

Adopting the CCSS and Getting an NCLB Waiver ....................................................................................... 14

New Jersey Teachers’ Union Opposes PARCC Testing Primarily Due to Its Link to Teacher Evaluation .. 16

StateGraduationRequirementsMadethePARCCSuperfluousforManyEleventhGraders ................ 17

Parent Groups and Anti-Testing Advocacy Groups Mobilized Against the PARCC ................................ 18

State Communications to Districts Sent Mixed Messages, Fueling Uncertainty ....................................... 19

VI. The Local Context ................................................................................................................................................. 20

Test Administration Challenges Result in Implementation Issues ................................................................. 21

Test Anxiety ........................................................................................................................................................ 22

Role of High School Students ........................................................................................................................... 23

VII. The Role of Social Media in the Opt-Out Movement ........................................................................................ 23

VIII. Policy Implications ................................................................................................................................................ 26

IX. Epilogue ................................................................................................................................................................. 29

References .................................................................................................................................................................... 31

Appendix A: Research Design .................................................................................................................................... 34

Appendix B: Opt-Out in ELA by District Factor Group ............................................................................................... 36

VI | CONSORTIUM FOR POLICY RESEARCH IN EDUCATION | CPRE.ORG


List of Abbreviations

ARRA American Recovery and Reinvestment Act

CCSSO CouncilofChiefStateSchoolOfficers

CSSS Common Core State Standards

DFG District Factor Group

ELA English Language Arts

ESEA Elementary and Secondary Education Act

ESSA Every Student Succeeds Act

HSPA HighSchoolProficiencyAssessment

NCLB No Child Left Behind

NGA National Governors Association

NJASK New Jersey Assessment of Knowledge and Skills

NJDOE New Jersey Department of Education

PARCC Partnership for Assessment of Readiness for College and Careers

PTA Parent Teacher Association

RTTT Race To The Top

SBAC Smarter Balanced Assessment Consortium

TEACHNJ Teacher Effectiveness and Accountability for the Children of New Jersey Act


WORKING PAPER |1

I. Executive SummaryFor decades, public schools have marched to the drumbeat of state-sponsored testing. Each spring, for two weeks or more, schools across America are dedicated to assessing students in the core subjects of reading, writing, and mathematics. In many schools the actual testing time is the culmination of months of build-up devoted to test preparation, as teachers put away their regular teaching materials and increasingly focus their lessons on preparing students to take the state test. Time spent preparing for the test far exceeds testing time. This trend follows the growing use of high-stakes testing as a policy tool intended to direct the emphasis of the education system towards academic outcomes and to apply pressure on educators to shape their behavior and hold them accountable for student performance.

The swelling pressure of years of testing and accountability policies unexpectedly burst in the spring of 2015. While there had always been pockets of anti-testing sentiment, 2015 saw an unprecedented nationwide movement of parents deciding to opt their children out of state testing. Hundreds of thousands of parents and their children from acrossthenationprotestedagainsttheinfluenceoftestingineducationby withholding their participation and opting out of their state test. In Colorado, almost 70 percent of the eleventh graders in the 20 largest districts in the state opted out (Engdahl, 2015). In the state of Washington, nearly 30 percent of eleventh graders opted out of the state test (Ujifusa, 2015). In New York State, 20 percent of all students opted out (Harris, 2015).

What caused a small but persistent anti-testing movement to grow into a national phenomenon? Was it a narrowly focused antipathy towards the utility of annual testing as a measure of school worth and student accomplishment? Was it a symptom of a larger disquiet with the American education system? Is it likely to persist in the coming years? These are questions that we take up in this analysis of the 2015 opt-out movement in New Jersey.

Based on an investigation of state data (New Jersey Department of Education, 2015a), we found that about 135,000 students in New Jersey across grades 3–11 did not take the State’s test in the spring of 2015. While there could be several reasons for registered students to not take the state test, including absence, medical emergencies, and other reasons1, the rates were substantially higher in 2015 than in previous years. For parsimony, we hereafter refer to all not-tested cases as opt-outs. District opt-out rates reached as high as 70 percent, and averaged 19 percent

1 According to the New Jersey Statewide Assessment Reports (2015), “The Not Tested count includes all students who did not receive a valid scale score. This includes students who were absent, students with medical emergencies, students who were enrolled to admin-ister the Partnership for Assessment of Readiness for College and Careers (PARCC) but were administered Dynamic Language Maps (DLM) or World-class Instructional Design and Assessment (WIDA), students who refused to take the assessment, parents who refused to have their child take the assessment, voids, and other issues that prevented students from completing the PARCC assessment and receiving a valid score.” Retrieved from http://www.nj.gov/education/schools/achievement/15/parcc/excel.htm


2 | CONSORTIUM FOR POLICY RESEARCH IN EDUCATION | © 2016

across the 644 districts that reported data to the state. Further, we found that opt-outs were substantially higher in high schools than in elementary and middle schools in both mathematics and English language arts (ELA). From the state data, there was no relationship between district socio-economicstatus(asidentifiedthroughDistrictFactorGroups,orDFG)andopt-out rates, except in high schools, where districts with higher socio-economicstatushadsignificantlyhigheroptoutrates.

The data provided by the state also had a substantial amount of missing data 2 – almost 40 percent of the districts did not report data to the state onthenumberofstudentsregisteredtobetested,whichmadeitdifficultto produce an accurate picture of opt-out rates across the state. When we replaced these missing registered to test numbers with enrollment data reported elsewhere (New Jersey Department of Education, 2015b), we found that the average opt-out rate across the state declined to about 11 percent. Therefore, ironically, the incomplete data reported by the state initsaccountingofopt-outratesresultedininflatedestimatesofstudentsnot tested. On the other hand, the replacement of the missing data with enrollment data revealed a strong correlation between district socio-economic status and opt-out rates across elementary, middle and high schools–withhigherDFGdistrictshavingsignificantlyhigheroptoutratesacross the board.

In order to investigate the motivations underlying these trends, we interviewed educators at the state, district, and school levels, as well as parents and advocacy group leaders. Our analyses of these data suggestedthattheopt-outmovementemergedfromaconfluenceoffactors, some persistent and others circumstantial. These included:

• an accumulated skepticism of the merits of a high-stakes testing system, which has come to dominate a big chunk of the school year and educators’ time and energy;

• a spillover effect from the unremitting furor over the Common Core State Standards (CCSS), which had been roiling the education waters across the country for the past several years, raising concerns about the direction of education policy;

• the perception and concern over the federal government’s strong-armed push—embodied in the Race To The Top (RTTT) competition—to encourage states to implement large-scale changes in their education systems in a relatively short time, including more ambitious standards, aligned assessments, and teacher accountability;

• concerted teachers’ union opposition to the use of student growth techniques as measures for teacher accountability, particularly so soonaftertheimplementationoftheCCSSandinthefirstyearofPARCC;

• the concern of local educators about the new PARCC assessment,

2 According to the New Jersey Statewide Assessment Reports (2015), “Data is suppressed to protecttheconfidentialityofthestudents.”Retrievedfromhttp://www.nj.gov/education/schools/achievement/15/parcc/excel.htm

WORKING PAPER |3


which was exacerbated by substantial implementation problems in schools and districts;

• the outsourcing of the New Jersey state test to the Partnership for Assessment of Readiness for College and Careers (PARCC) for the firsttime;

• confusion in the messages from state policymakers about which testscouldfulfilltherequirementsforhighschoolgraduation.

Collectively, these concerns coalesced to create the conditions that fueled the opt-out movement in New Jersey. These were also likely catalysts in similar movements in other states. Driven by these factors, the opt-out movement is a wake-up call to education policymakers about the appropriate role of testing and accountability in our education system. There is an urgent need to explore how instruments can be used productively, while minimizing their negative externalities to productively shape the experiences of students and educators. If the opt-out movement spurs a recalibration of the role of assessment and accountability in school improvement, we may look back at 2015 as a watershed moment in the history of American education.

II. OverviewThis report analyzes the scope, factors, and context for the opt-out movement that occurred in New Jersey in the spring of 2015. It examines the magnitude of enrolled students not taking the spring PARCC test and the national, state, and local contexts that contributed to parental opt-out decisions. Our investigation was focused on addressing three essential questions:

1. What was the scope of students opting out of the PARCC test in grades 3 to 11 in ELA and mathematics in the spring of 2015 in New Jersey, and were opt-out patterns different across the diverse communities in the state?

2. What were the major national, state, and local factors that contributed to the opt-out phenomenon?

3. What are the policy implications for New Jersey and other states?

To address these questions, the research team studied data from a range of sources. First, we analyzed New Jersey Statewide Assessment Reports made publicly available in February 2016 by the New Jersey Department of Education (NJDOE). These data provide a sense of the magnitude of the movement, as well as an independent accounting of its scope. Because there was a substantial amount of missing data on students who were registered to test, we merged enrollment data from a separate state data source to arrive at closer estimates of opt-out rates for the state.

Second, we conducted over 30 phone and in-person interviews with a range of state and local New Jersey actors in September and

4 | CONSORTIUM FOR POLICY RESEARCH IN EDUCATION | CPRE.ORG


October of 2015. Interviewees included state policymakers, professional education association representatives, advocacy group leaders, school administrators, teachers, parents, and students.

Third, we examined publicly available Twitter data from a few of the central advocacy groups and relevant hashtags about opting out shared by parents to investigate how New Jersey’s opt-out movement was communicated about and represented on social media. Our analysis included about 5,000 tweets and the major hashtags related to Twitter conversations and information-sharing surrounding the opt-out movement. And a more detailed description of our research design, sampling, data collection, and analysis methods are provided at the end of this report in Appendix A.

Report Organization

Inthefirstsectionofthisreport,weusedatafromtheNJDOEtodescribethe magnitude of the opt-out phenomenon in the state. We examine trends across socio-economic status bands in the state by subject (ELA and mathematics) and grade level (elementary, middle, and high schools). The second section describes the national backdrop and the educational context in New Jersey and other states as new standards and assessments were being implemented in a politically contentious environment. The third section outlines the state-level factors that contributed to the patterns of opt-outs in New Jersey in the spring of 2015. Section four focuses on the sentiment within the local educational context about parents choosing not to have their children take the state exam. This includes perspectives from districts, schools, and parents. The fourth section highlights the role of social media in facilitating the movement. Thefinalsectionsummarizesthestudyfindingsanddiscussesimplicationsfor state and local policymakers moving forward.

WORKING PAPER |5


III. The Magnitude of Opt-Outs in New Jersey in 2015To assess the scope of the test opt-outs in New Jersey in the spring of 2015, we used data released by the NJDOE in February of 2016. These included district level data for each grade (3 to 11) in mathematics and ELA. The data contained the number of students in each district in the state—organized by grade and subject—who were (a) “registered to test” and (b) who received “valid scores.” We used these data to produce a percentopt-out,whichwedefinedasthenumberofstudentswithvalidscores as a proportion of the students who were registered to test.

The data set contained data for 539 districts, 85 charter schools, and 20 vocational schools, and we report numbers for each of these entities. Where we report overall numbers, we treated the charter and vocational schools as districts; where we disaggregated, we report them separately. A large number of districts had missing data for the “registered to test” variable in one or more grade levels. Therefore, our aggregations to levels (elementary, middle, and high) often included only a subset of the data. In many districts we could not calculate an opt-out rate at all, because all the data were missing.

Overall, based on the data reported by the state, the average opt-out rate in the state was 19 percent (with a standard deviation of 11 percent, which indicates skewness towards the high end). Opt-out rates exceeded 65 percent in a few districts, and 20 districts had 40 percent or more of their students opting out. Additionally, opt-out rates were much higher in high school than they were in middle or elementary schools. Interestingly, the numbers not reported declined from elementary to middle to high school. Based on the NJOE dataset, the rates of opting out were not correlated with district socioeconomic status overall, but they were correlated with socioeconomic status in high school, with lower poverty districts having higher opt-out rates in ELA and Algebra II. We could not calculate opt-out rates for about 40 percent of the districts as they did not report the number of students who were registered to test. Those with missing data included a substantial number of charter schools, with only 10 percent of charter schools reporting students registered to be tested in the state database. These and other results are reported in more detail below.

Inthefiguresthatfollow,wereportdatabydistrictsoverallanddecomposed by the tested grades in elementary schools (tested grades 3-5), middle schools (tested grades 6-8) and high schools (tested grades 9-11). We have also chosen to present the opt-out rates in four bands. The lowest band contains districts with opt-out rates less than six percent. Since No Child Left Behind (NCLB) is the prevailing law through 2017 (when the Every Student Succeeds Act (ESSA) goes into effect), and requires that 95 percent of students be tested, we have elected to use a similar cut-off for the lowest band. The second band consists of the number of districts with 6 to 15 percent of students opting out. The third



band comprises districts with more than 15 percent and less than 25 percent opting out. The fourth and highest band includes districts with greater than 25 percent of students opting out. Although these bands are somewhat arbitrary, they effectively display the distribution of opt-out rates in the state of New Jersey in 2015.

Overall Opt-Out Rates by Subject

The opt-out rate for districts across the state of New Jersey was 19 percent. The opt-out rates were similar in ELA (19.3%) and mathematics (19.7%). The overall opt-out rates by subject and grade range are presented by bands inFigure1.Severalimportantfindingscanbeseeninthesedata.First,asshown in the leftmost two bars, in both ELA and mathematics, about 15 percent of districts had more than 25 percent of their enrolled students opting out; about 20 percent of districts had 15 to 25 percent of students opting out; about a quarter of districts had between 6 to 15 percent opting out. About 40 percent of the districts did not report enrollment data and therefore were not represented in the New Jersey Statewide Assessment Reports provided by the state.

The second notable trend was that opt-out rates increased from

elementary to middle to high school. For example, only 10 percent of districts had opt-out rates exceeding 15 percent in elementary school. In middle school the number jumped to 24 percent. In high school, more than half of the districts had opt-out rates exceeding 15 percent. More specifically,inELA57percentofdistrictshadhigherthan15percentopt-out rates. In geometry, which is largely a tenth grade subject, 38 percent of districts had more than 15 percent of their students opting out. In Algebra II, 59 percent of districts had greater than 15 percent opt-out rates.

98(15%)

111(17%)

158(25%)

23(4%)

254(39%)

ELA All(n=644)

16(2%)

88(14%)

129(20%)

156(24%)

255(40%)

Math All(n=644)

11(2%)

57(10%)

43(8%)

107(19%)

342(61%)

ELAElementary

(n=560)

10(2%)

58(10%)

43(8%)

103(18%)

346(62%)

MathElementary

(n=560)

27(5%)

43(8%)

92(16%)

134(24%)

265(47%)

ELA Middle(n=561)

21(4%)

44(8%)

92(16%)

127(23%)

277(49%)

Math Middle (n=561)

17(6%)

125(40%)

53(17%)

79(26%)

35(11%)

ELA High

(n=309)

11(2%)

86(17%)

82(16%)

101(20%)

221(44%)

Math Algebra I(n=501)

11(3%)

105(30%)

56(16%)

74(21%)

104(30%)

Math Geometry

(n=350)

8(3%)

128(41%)

55(18%)

50(16%)

71(23%)

Math Algebra II(n=312)

100%

75%

50%

25%

0%

Figure 1. District Opt-Out Rates by Grade Band and Subject

Nu

mb

er a

nd

Pe

rce

nt

of D

istric

ts in

Ea

ch

Ba

nd

No. & Pct. of Districts with more than 25% opting out

No. & Pct. of Districts with between 6–15% opting out

Districts Not Reported


No. & Pct. of Districts with less than 6% of students opting out

ElementaryAll Middle High School

WORKING PAPER |7


Third, high schools (considering ELA, geometry, and Algebra II) had the highest opt-out rates. About 40 percent of districts had greater than 25 percent of enrolled students opting out of the PARCC test; about 17 percent of districts had between 15 to 25 percent opt-out rates. Over a third of the districts in the state reported opt-out rates of at least 15 percent at the high school level. Finally, the prevalence of missing data is also notable, with about 40 percent of the districts overall not reporting their data on the number of students tested to the state.

Opt-Out Rates in Mathematics by District Factor Group

As noted, the opt-out rates across ELA and mathematics were fairly similar (see comparisons in Figure 1). Therefore, in the next analysis we present data only on mathematics, although the patterns are very similar to those in ELA (shown in Appendix B). In this section we decompose opt-out rates by districts with different levels of socioeconomic challenge, called district factor groups (DFGs) in New Jersey. DFGs are a measure of a community’s socioeconomic status, based on census data3.

Figure 2 shows the opt-out rates for districts in different DFGs, as well as for charter schools and vocational schools. In this Figure, we aggregate all the different levels of schools (elementary, middle, and high) into a single district-level opt-out rate.

When looking at district-wide numbers, opt-out rates were fairly consistent across the different DFG rates of districts, with about 10 to 20 percent of districts at each DFG having very high opt-out rates (greater than 25 percent); about 20 to 30 percent having moderately high levels of opt-out

3 To produce District Factor Groups, the state used decennial census data to categorize districts into groups based on six variables: (1) percent of adults with no high school diploma; (2) percent of adults with some college education; (3) residents’ occupational status; (4) unemployment rate; (5) percent of individuals in poverty; and (6) median family income.

DFG A(n=38)

DFG B(n=65)

DFG CD(n=65)

DFG DE(n=82)

DFG FG(n=88)

DFG GH(n=73)

DFG I(n=103)

DFG J(n=25)

Charter(n=85)

Vocational(n=20)

100%

75%

50%

25%

0%

Figure 2. Opt-Out in Mathematics by District Factor Group

Nu

mb

er a

nd

Pe

rce

nt

of D

istric

ts in

Ea

ch

Ba

nd

98(15%)

111(17%)

158(25%)

23(4%)

254(39%)

By Grade Range Charter & Vocational All

All(n=644)

4(11%)

7(18%)

13(34%)

3(8%)

11(29%)

5(8%)

11(17%)

19(29%)

3(5%)

27(42%)

10(15%)

16(25%)

14(22%)

25(38%)

17(21%)

13(16%)

23(28%)

29(35%)

12(14%)

21(24%)

22(25%)

2(2%)

31(35%)

13(18%)

17(23%)

22(30%)

3(4%)

18(25%)

16(16%)

27(26%)

31(30%)

2(2%)

27(26%)

3(12%)

8(32%)

3(12%)

1(4%)

10(40%)

6(7%)

4(5%) 1

(1%)

74(87%)

2(10%)

5(25%)

2(10%)

8(40%)

3(15%)

88(14%)

129(20%)

156(24%)

16(2%)

255(40%)








rates (between 15 to 25 percent); and another 20 to 30 percent of districts having moderate opt-out rates (between 6 to 15 percent). The overall correlation between DFG and opt-out rates (treated as a continuous variable, not collapsed within bands) was small (.078) and not statistically significant,whenexcludingthedistrictsthatdidnotreportenrollmentdata. The charter schools are notable for not reporting their registered to test numbers to the state. Almost 9 in 10 charter schools did not have registered to test in the state data.

Opt-Out Rates Merging in District Enrollment Figures

The high number of districts not reporting the number of students who were registered to test resulted in 40 percent of the data being missing in the New Jersey Statewide Assessment Reports provided by the state. For this reason, we were concerned that the available data were producing an inaccurate representation of the true opt-out rates in New Jersey. Therefore, we downloaded 2014-2015 district enrollment data by grade from a separate state database4 and merged these data with the New Jersey Statewide Assessment Reports set, thereby replacing the missing registered-to-test data with enrollment data. We only did this substitution for grades in which districts did not report their number of students who were registered to be tested. The advantage of this approach is that it removes much of the missing data from the data set and allows us to include almost all of the districts in the state in the analysis. The disadvantageisthatenrollmentdataareconstantlyfluctuatingandtheremay be some slight differences between students enrolled at the time of spring testing and the numbers reported in the enrollment data. To test this assumption, we compared the enrollment data reported by the state with the registered-to-test in the New Jersey Statewide Assessment Reports set and found that 99 percent of the registered-to-test data fell within ± 2 percent of the enrollment data. We therefore felt it was reasonable to proceed.

We conducted this merging of data for ELA, but not for mathematics. We could not apply this technique to the mathematics data, because the data were reported for each grade in K-8 but then by subject matter (Algebra I, geometry, and Algebra II) beginning in eighth grade. This made it impossible for us to distinguish between grade-level enrollment and subject testing. For example, some eighth graders took the eighth grade test, while others took Algebra I. Similarly, ninth and tenth graders in high school could have taken either the Algebra I or geometry test, or even in some cases Algebra II. From the state data, it was impossible for us to know how many of the enrolled students took which test, and therefore impossible to arrive at accurate opt-out rates in mathematics in middle and high schools using the merged enrollment data. Therefore, we report these data for ELA only.

4 The state database is publicly available at http://www.nj.gov/education/data/enr/enr15/stat_doc.htm

WORKING PAPER |9


The overall average opt-out rate using the merged data was 10.9 percent, with a standard deviation of 10.7 percent. The opt-out rates withthemergeddistrictenrollmentfiguresforELAareshowninFigure3.They are shown for grade ranges (elementary, middle, and high schools), as well as for the different DFGs. The overall average opt-out rate in ELA, which includes 99 percent of the districts in the state, is 11 percent. Interestingly, this is substantially lower than the 19 percent opt-out rate we calculated using the state’s New Jersey Statewide Assessment Reports dataset5.

You can also see from Figure 3 that the opt-out rates increase from the elementary grades to the middle grades to high school, just as they did in Figure 1. The major difference is that districts with unreported data tended to fall into the lower two opt-out rate bands when their enrollment data were included. This explains the drop in the overall average opt-out rate between the two approaches.

The next set of bars in Figure 3 shows the opt-out rates in ELA by DFG just for the high schools in the sample. This shows there were high opt-out rates in high schools across all DFGs. There was also a general pattern of higher highschoolopt-outratesinthewealthierDFGs.ThefinaltwobarsinFigure3 show that after merging enrollment data, charter high schools had opt-out rates that were similar to the overall average ELA opt-out rates across the state; and vocational high schools were slightly higher.

5Acarefulreadermightfinditparadoxicalthatthenumberofdistrictsinthehighestopt-out band in ELA declined missing registered to test data. When these were merged into the data, the number of districts in this category, and others, shifted.

ELA All(n=644)

ELA Elementary

(n=560)

ELA Middle(n=561)

ELAHigh

(n=308)

HS DFG A(n=26)

HS DFG B(n=31)

HS DFG CD(n=30)

HS DFG DE(n=43)

HS DFG FG(n=42)

HS DFG GH(n=39)

100%

75%

50%

25%

0%

Figure 3. Opt-Out Rates in ELA using Merged District Enrollment Figures

Nu

mb

er a

nd

Pe

rce

nt

of D

istric

ts in

Ea

ch

Ba

nd

98(15%)

111(17%)

158(25%)

23(4%)

254(39%)

By District Factor GroupBy Grade RangeAll

HS DFG I(n=43)

Charter & Vocational

HS DFG J(n=11)

HS Charter(n=24)

HS Vocational

(n=20)

68(11%)

102(16%)

207(32%)

6(1%)

261(41%)

41(7%)

169(30%)

339(61%)

7(1%)

4(1%)

36(6%)

90(16%)

186(33%)

7(1%)

242(43%)

129(42%)

47(15%)

87(28%)

45(15%)

12(46%)

5(19%)

7(27%)

2(8%)

10(32%)

7(23%)

8(26%)

6(19%)

14(47%)

5(17%)

5(17%)

6(20%)

21(49%)

4(9%)

15(35%)

3(7%)

15(36%)

8(19%)

12(29%)

7(17%)

18(46%)

5(13%)

12(31%)

4(10%)

23(53%)

6(14%)

11(26%)

3(7%)

7(64%)

2(18%)

2(18%)

3(13%)

5(21%)

8(33%)

8(33%)

6(30%)

2(10%)

7(35%)

1(5%)

4(20%)








Correlations Between Opt-Out Rates and District Factor Groups

InourfinalanalysisoftheNewJerseyStatewideAssessmentReportsdata, we calculated the correlations between opt-out rates and district factor groups. For these analyses, we used the actual opt-out rate for each district, rather than grouping them into bands. We also removed charter and vocational schools, who have no DFG attached to them. We conducted two sets of correlational analyses, one using the original data from the state, which removed the 40 percent of missing districts. The second analysis used the dataset where we incorporated enrollment data for the districts with missing data in the state New Jersey Statewide Assessment Reports dataset.

The correlations between opt-out rates of different subgroups of the data and the district socio-economic status indicator or DFG are shown inTable1.Twomajorfindingsemergefromanalyzingthedatahere.First,that using the original data, there is no overall correlation of either ELA or mathematics with DFGs. In fact, with the original data, there was only a correlation in high schools between opt-out rates and DFGs.

However, as shown in the full data set that includes the merged data, therearesignificantcorrelationsbetweendistrictsoverallandDFGs,andin each of the subgrade levels and DFGs. That is, there was a positive

andsignificantcorrelationbetweenopt-outratesinELAandDFGsattheelementary, middle, and high school levels. Collectively, these empirical analyses raise questions about what national, state, and local factors contributed to parental decisions to opt their children out of PARCC testing. This is the focus of the remainder of this report.

Table 1. Correlations between Opt-Out Rates and DFGs

Subgroup DFG (original data)

DFG(with merged

enrollment data)

All Districts ELA (grades 3–11) .095 .110**

ELA Elementary Schools (grades 3–5) .079 .143**

ELA Middle Schools (grades 6–8) .015 .119**

ELA High (grades 9–11) .196** .258**

All Math (grades 3–11) .078

Math Elem (grades 3–5) .076

Math Middle (grades 6–8) .050

Math Algebra I (mostly grades 9 and some grade 8) .020

Math Geometry (mostly grade 10 and some 9 ) .103

Math Algebra II (mostly grades 11 and some grade 10) .135*

**p<.01 *p<.05

WORKING PAPER |11

IV. Opt-Out Backdrop: National Education Policy Maelstrom Feeds Testing ConsternationTwo powerful and related national factors contributed to the climate in which the opt-out movement exploded across the country in the spring of 2015. First, the pitched battle over the states’ adoption of the CCSS increased the partisan hyperbole surrounding education policy (Supovitz, Daly, & Del Fresno, 2015). Second, a strong federal push for states to adopt a system of standards, testing, and accountability alienated a range of constituencies that were traditionally in favor of reforms that sought to improve instructional processes (Supovitz & Spillane, 2015).

The Common Core and the Backlash Against Common Standards

The CCSS is the latest, and most concerted, effort to use standards to leverage improvement across the American education system (McDonnell & Weatherford, 2013; Supovitz & Spillane, 2015). Using standards, or expectations for student performance as a fulcrum of policy change can be traced throughout American education reform (Gamson, 2015). A recent example was the systemic reform effort of the 1990s that was built around three general principles. First, ambitious standards were developed by each state to provide a set of targets of what students ought to know and be able to do at key grade junctures. Second, states measured progress toward standards by developing aligned assessments that combined rewards and sanctions for holding educators accountable tothestandards.Thethirdcomponentwaslocalflexibilityinorganizingcapacity to determine how best to meet academic expectations (Smith & O’Day, 1991; Vinovskis, 1996).

This structure of clear goals (standards), measures (assessments), and incentives (accountability) at the state level, combined with implementationautonomy,fitwithAmerica’shistoricalconceptionof education as a locally organized effort. This was supported by the redirecting of federal dollars to aid state efforts to raise academic standards (McGuinn, 2006). As each state developed its own standards and assessment systems, a lot of variation emerged in the quality and rigor of state educational systems across the country. This contributed to a perception of disappointment with the standards-based reform movement of the 1990s (Hamilton, Stecher, & Yuan, 2008).

The Evolution of Assessment in State and Federal Education Policy

1970s Minimum competency testing as a way to set base expectations for student performance led to low expectations for achievement.

1980s Alternative and performance assessment movement as a trend to move away from a narrowing of instruction associated with standardized testing.

1990s States develop standardized assessments aligned with state standards.

2000s No Child Left Behind expansion of state testing and ratcheting up of test-based accountability systems.

2010s Multi-state test consortia developed to align assessments to Common Core State Standards.




The 2000s gave rise to a much stronger emphasis on testing as the driving force for holding schools and districts accountable for meeting standards. Test-based accountability is seductive to policymakers because it is a relatively inexpensive way to direct the behaviors of district and school leader, and teachers. Attaching stakes to tests incents teachers to align instruction to standards and provide more instructional time to cover the material that will be on the test (Hamilton, 2003). The 2001 passage of the NCLB Act inaugurated an expansion of testing by requiring states that receive federal funding (which all do) to assess students annually in all grades between third and eighth, and one year in high school. NCLB pressed states to develop plans to have all schools make adequate yearlyprogresswithatargetof100percentproficiencyby2014—anendeavor that would prove impossible. The research on schools that were the targets of test-based accountability showed both productive and unproductive responses. There was a rise in time spent on low-level test preparation activities and more attention to students just at the cusp of passing the test. Disproportionate attention was paid to tested subjects at the cost of other important developmental needs of children (Jennings & Rentner, 2006). Some states gamed the system by creating tests that moststudentscouldeasilypassandtherewereseveralhigh-profilecasesof systematic cheating by educators in school districts and schools. The test-based accountability movement can be seen as an attempt to tighten the linkages in the theory of standards-based reform by increasing student performance expectations via high-stakes testing to hold schools accountable for meeting standards (Hamilton, 2003; Koretz, 2008; Supovitz, 2009). But this experience left many policymakers convinced that although pressure was important, we could not just squeeze higher performance out of the system without an infrastructure to support it (Cohen&Moffit,2009).

The CCSS incorporated a number of lessons learned from the earlier standards-based reform movement (McDonnell & Weatherford, 2013). The new standards were named the “Common Core” because they sought to eliminate the variation in the quality of state standards experienced in the 1990s, by designing a set of uniform standards. They were developed at thebehestofthestategovernorsandchiefstateschoolofficerstoavoidthe charge of federal intrusion. The development of the CCSS began in 2009 in a collaborative effort between the National Governors Association (NGA)andtheCouncilofChiefStateSchoolOfficers(CCSSO).TheCCSSset forth what students should know and be able to do in mathematics and ELA at each grade level from kindergarten to twelfth grade. Advocates argued that high, uniform academic standards would improve the academic performance of American students and better prepare them for college and careers.

In a remarkable and short-lived moment of bi-partisanship, the CCSS were adopted by the legislatures in 46 states and the District of Columbia in 20106. Since then, the CCSS have become increasingly controversial, with Indiana and Oklahoma backing out of the CCSS and several other states

6 Alaska, Texas, Virginia, and Nebraska did not adopt the CCSS, preferring their own state standards. Minnesota adopted the Common Core standards in ELA, but not in mathematics.

WORKING PAPER |13


(including Missouri, New Jersey, Tennessee, and West Virginia) developing new standards to replace the CCSS. Opponents of the CCSS have made a range of arguments that critiqued the standards themselves (not developmentallyappropriate,reducedemphasisonclassicalfiction,attended to academic priorities at the expense of social and emotional needs), but primarily attacked the CCSS on cultural and ideological grounds(federaloverreach,dataprivacy,corporateprofitingoffofapublic good).

National Government Efforts to Push State Implementation of Standards, Tests, and Accountability

Despite this evolution of a set of common standards implemented in a decentralized manner, national circumstances and the eagerness of federal policymakers to prod implementation forward led to a series of bellwether decisions that have haunted implementation moving forward. Two federal policies in particular contributed to the national context. First, the Obama administration used funding from the American RecoveryandReinvestmentAct(ARRA),an$800billiondollarfiscalstimulus bill passed in 2009 that earmarked $97.4 billion to the Department of Education. To push states to adopt the standards, the Department of Education, under Secretary of Education Arne Duncan, used $4.35 billion to create the grant competition Race to the Top (RTTT). The 19 state awardees received RTTT grants of a total of $4.1 billion to adopt a range of policies including new standards and assessments, build data systems to measure student growth, and develop teacher and principal evaluation systems.

The Department of Education also used stimulus funding to award two comprehensive assessment system grants in September 2010. PARCC with 26 member states received 170 million in federal funds and Smarter Balanced Assessment Consortium (SBAC) with 31 member states received 160 million7. The grant tasked the testing consortia with developing the next generation of assessments.

The second important policy decision was the waivers that the Department of Education instituted in 2011 as a placeholder due to Congress’s inaction in reauthorizing the Elementary and Secondary Education Act (ESEA) that expired in 2007. The waivers sought to reduce theNCLB’sburdenonstates.Thewaiversofferedstatesflexibilityfromadozen requirements of NCLB in exchange for the states’ commitment to four target areas: adopt college- and career-ready standards; develop new accountability systems based on reading and math assessments, graduation rates, and student growth over time; implement teacher and principal evaluation systems based on multiple factors with student growthbeingasignificantfactor;andreduceadministrativeandreporting requirements that are burdensome to states.

7 For more details about the award, see http://www2.ed.gov/programs/racetothetop-as-sessment/index.html



The CCSS movement, propelled forward by federal initiatives in the form of RTTT dollars and waivers from the looming requirements of NCLB, had the effect of states and federal policy critics viewing these acts as coercion to implement standards, assessments, and accountability without adequate time to phase them in.

V. New Jersey State Factors that Contributed to the Spring 2015 Climate The national context played into a series of events and decisions in New Jersey, contributing to the environment that produced the opt-out dynamic. First, New Jersey adopted the CCSS and competed for — and eventually won — RTTT funds. However, the teacher evaluation requirement that was part of RTTT alienated the state’s teachers’ union, The New Jersey Education Association (NJEA), and resulted in strong NJEA opposition to the PARCC test. Second, New Jersey adopted the CCSS-alignedPARCCtest,and2015wasthefirstyearofitsadministration.TheonlinenatureofthePARCC—afirstforstatewidetestinginNewJersey— and confusion about its requirement for graduation further eroded test participation. Third, there was a political twist to the dynamic of state testing, as the state’s governor, Chris Christie, was running for president and sought to shore up his Republican candidacy by publicly opposing, and eventually dropping the CCSS, while maintaining state support for PARCC.Inthissectionwefollowthesetrendsandhowtheyinfluencedconditions in the state.

Adopting the CCSS and Getting an NCLB Waiver

Joining 46 other states, the New Jersey State Board of Education adopted the CCSS in 2010. Along with CCSS adoption, New Jersey joined the PARCC testing consortium in the spring of 2010 and became a PARCC governing state in the spring of 2011, allowing it to have a voice in the development of the next-generation assessment system.

In the midst of the recession of 2009, New Jersey competed in all three rounds of RTTT. In 2010, New Jersey came in eighteenth of the 40 states that competed. In the second round, also in 2010, New Jersey came in eleventh of the 36 states in the competition, just tantalizingly outside of the10statesbeingawardedfunding.Thefinalistsinthesecondroundwere encouraged to revise their proposals and seven of them were awarded grants. In this last RTTT cohort, New Jersey received $38 million in December 2011, and agreed to support the transition to higher standards and improved assessments and implement both teacher and principal evaluation systems.

WORKING PAPER |15


In 2011, the NJDOE also submitted a waiver application to the US Department of Education (USDOE) for relief from certain provisions of NCLB. The comprehensive waiver allowed the NJDOE to replace a provision of NCLB with a new accountability system, which centered on providing support and intervention to the state’s lowest-performing schools and those with the largest in-school gaps between subgroups of students.

Following the approval of New Jersey’s 2011 waiver application, New Jersey State Senator and chair of the Senate Education Committee, Teresa Ruiz (D), proposed a teacher and principal evaluation/tenure bill, the Teacher Effectiveness and Accountability for the Children of New Jersey Act (TEACHNJ) that was supported by the NJEA and signed into law by Governor Christie in August 2012.

The law, which went into effect in the 2014–15 school year, put into place a yearly evaluation system for teachers and principals that included students’ performance on annual statewide assessments to be considered as a “predominant factor” in an educators’ annual performance rating. According to the law, elementary and middle school ELA and mathematics teachers would have 30 percent of their ratings based on student progress. In July of 2014, due to widespread concern from educators about the proportion of performance connected to a single measure, Governor Christie signed an executive order lowering that amount to 10 percent.

NewJerseyalsoparticipatedinthePARCCfieldtestinthespringof2014and administered PARCC in the spring of 2015, along with 11 other states and the District of Columbia. Joining the PARCC consortium brought several changes to state standardized testing in New Jersey. First, the PARCC tests are computer-based, a contrast to New Jersey’s previous statewideassessments,theHighSchoolProficiencyAssessment(HSPA)and the New Jersey Assessment of Knowledge and Skills (NJASK), which were completed using paper and pencils. Second, unlike these previous assessments, the PARCC tests were designed to measure students’ higher order thinking skills and problem solving and include more short- and extended-response questions. To accomplish this, the test was administeredintwotimeperiodsseparatedbysixweeks.Thefirst,inMarch, focused on performance tasks and short answer questions to capture more authentic representations of student capabilities, while the second, about a month later, had more multiple-choice questions to assure domain coverage. Additionally, in the 2014–15 school year, the total testing time for the PARCC test was longer than that of the HSPA and NJASK. PARCC was administered in two windows with the total testing time amounting to approximately 8.25 hours for grades 3 to 7 and 9.7 hours for grade 11 (NJDOE, 2015). Third, the PARCC test promisedtoprovideparentswithmorespecificfeedbackabouttheirchild’s test performance by including in the score report comparisons of a child’s performance to the average scores of the school, district, state, and PARCC consortium states, as well as information about the child’s academic growth from year to year (NJDOE, 2015).



New Jersey Teachers’ Union Opposes PARCC Testing Primarily Due to Its Link to Teacher Evaluation

Although the state’s teachers’ union, the NJEA, supported the CCSS and initially supported TEACHNJ, the union staunchly opposed the new PARCC test. The NJEA’s opposition to the PARCC was in large part motivated by the state’s policy to use the test results as a factor in teacher evaluation. The tight, and immediate, alignment between standards and accountability was particularly problematic because teachers were just beginning to adjust to the new way of teaching to meet the CCSS. The union also believed the test had several additional shortcomings. The expandedtimerequiredforthetestadministrationsignificantlyreducedinstructional time. Also, the tests were inequitable for both teachers and studentsasnotalldistrictsweresufficientlyequippedwithtechnologyfor the online assessment and not all students had equal access to technology as a regular part of their educational experiences.

In response to the state’s testing and evaluation plans, the NJEA launched a multi-million dollar ad campaign against the PARCC. The NJEA’s strategy was to use television, radio, billboard, print, and online advertisements, as well as social media, to raise awareness and concerns with parents and the public about the PARCC exams. Members of the NJEA were also active in the winter of 2014 and spring of 2015 in attending town hall events, rallies, and board meetings across the state and voicing their views.

The NJEA did not formally declare that parents and students should refuse to take the PARCC test. However, their efforts to spread anti-PARCC messages made the union a key player in the opt-out movement. In describing their strategy, a union representative stated, “We believe that parents should know what the test was about, and we made no secret ofthefactthatwethinkthetestisdeeplyflawedandisbeingusedforpurposes that it shouldn’t be used for.” To further press their point, the NJEA conducted and publicized focus groups and polls of parents and voters about their attitudes toward testing in November and December of 2014. According to the union representative, “The results revealed that parents, even to a greater extent than we thought, [and the] public … was really frustrated and upset about what was happening with this testing.”

Interviewees at both state and local levels felt that the NJEA outreach strategy was effective and that their tactic of having union members introduce themselves and speak not as educators, but as parents, was shrewd. There was a general consensus amongst those we talked to that theNJEA’smessagesinfluencedparents’decisionstoholdtheirchildrenout of state testing. A Parent Teacher Association (PTA) member in a high-poverty district explained, “All the negative press that the test was getting from the NJEA, which had that whole ad on TV, really impacted people. I was getting calls and text messages in response to the ads,” she said.

The NJEA also developed collaborative relationships with parental opt-out groups and anti-standardized testing groups to share information about the misuse of standardized testing with parents and other community

WORKING PAPER |17


members. They jointly sponsored PARCC information events across the state where, according to the union representative, members of the NJEA’s local associations “would work with parent groups and other educationgroupsintheircommunitiestoshowafilmandhaveadiscussion about standardized testing and what the effects of it are.” On social media, the NJEA also shared messages and information with opt-out advocacy groups. Additionally, these groups worked together to collect data and publicize the number of students who opted out of the PARCC test in districts across the state.

SomestateofficialsbelievedthattheNJEAspreadmisinformationand misrepresented the PARCC test in their advertisements. One advertisement that several interviewees mentioned featured “the parent of[a]firstgraderwhowascryingbecause[his]kid[was]undersomuchstressbecauseofPARCC,[however]firstgradeisnoteveninvolvedin PARCC.” Another participant said that the NJEA also erroneously communicated that high school teachers would have student growth percentile scores calculated as a part of their annual evaluation.

Many people we interviewed commented on the connection between the opt-out movement and the state’s teacher evaluation policy. According to one school administrator, “The fact that teachers were suddenly being held more accountable for test results became a huge political piece… They [the state] should have studied the test more before tying it to evaluation.” A representative from an educational advocacy organization believed that the anti-testing movement has been increasing over the years in response to federal and state policies that misused test results a variety of ways. “This has been building over years,” he said, noting that the number of weeks of testing had increased from two to three, the testing results were increasingly driving policy, and school closures in urban districts were being based heavily on test performance. “So the stakes attached to the tests as well as the impact ofthetestsonthecurriculumhasbeenincreasingforthepastfiveyears,”he said.

State Graduation Requirements Made the PARCC Superfluous for Many Eleventh Graders

Another important factor that contributed to the rise in students opting outofthestatetestwasconfusionaboutwhattestqualifiedastudentto meet the state graduation requirements. When the PARCC was administeredforthefirsttimeinthespringof2015,thestateannouncedthat, although in future PARCC results would be tied to graduation, other competency tests would be allowed to substitute for the PARCC as part of the transition. Achieving a certain score on other tests—the SAT, ACT, PSAT, ASVAB, and Accuplacer—would demonstrate the necessary proficiencyandallowstudentstograduate.

Havingtheoptiontotakeotherassessmentstofulfillthestategraduationrequirement led to higher opt-out rates in high schools because students



and parents saw little reason to sit for meaningless tests. As one district administrator noted, “When parents, especially of students who were academically doing well, saw that their child had already met a graduation requirement through the PSAT, there was less motivation to take the PARCC test because of the number of hours students were going tomissfrominstruction.”Astateofficialfeltthattherangeofalternativesled some school counselors to interpret the PARCC as optional and for kids to think, “Oh, I don’t need to take the test because I’ve already got the SAT score I need.” The state’s range of graduation requirements resulted in the PARCC being seen as optional by many.

The confusion around the state’s graduation requirements also contributed to the differences in opt-out across communities with different socio-economic status. According to a high DFG parent and PTAmember,parentsinhighDFGshadthetimeandresourcestofind,understand, and communicate about the state policies that dictated graduation requirements. “The state didn’t make it readily available for parents to know that if students take the SAT or ACT they can also use that for the testing option to graduate from high school,” the parent said. Thus information about the options for students to meet the state’s graduation requirement was likely to have been unevenly available across communities in the state.

Parent Groups and Anti-Testing Advocacy Groups Mobilized Against the PARCC

The three groups advocating opting out that were mentioned most frequently in our interviews were Save Our Schools New Jersey, United Opt OutNewJersey,andCaresAboutSchools.ManyparticipantsidentifiedSave Our Schools as the most involved in leading the opt-out charge.

Save Our Schools was founded in Princeton, New Jersey by parents who were concerned with charter expansion and more recently had become involved in advocacy around school funding and high-stakes testing. United Opt Out New Jersey was a state chapter of the United Opt Out National group that was established in response to NCLB. According to a special interest group representative, United Opt Out New Jersey serves as “a clearinghouse for people interested in opting out, small organizations that have kids opting out, or just organizations that wanted equity in schools.” The Cares About Schools groups are located in various communities across the state and organize around education issues in their communities.

These three parent-led groups collaborated with each other to spread information about the PARCC test and opting out in a transpartisan partnership where issue trumped ideology. Several interviewees mentioned that the liberal leaning opt-out groups like United Opt Out and Save Our Schools worked across ideological lines with conservative groups like the Eagle Forum to raise awareness about opting out and supported legislation and policy recommendations allowing opting out.

WORKING PAPER |19


A representative from a special interest group described the coalition of opt-out advocacy groups as “bipartisan; it was parents united for local school education.” A school administrator from a lower DFG district describedthemessagingfromSaveOurSchools,“Whenitfirststarted,Save Our Schools was really talking about the concern with testing, the amount of testing. That started maybe two to three years ago. When PARCC came, their message moved from this push from all of this assessment for students, to PARCC being a bad assessment.”

Opt-out advocacy groups relied substantially upon social media to disseminate information about the PARCC test and opting out and as a tool to organize their members. A special interest group representative mentioned that the Facebook sites of opt-out groups contained “robust discussion and lots of sharing of information, lots of coordination about how to present this issue to local school boards and form resolutions or different kind of policy decisions.” Many study participants agreed that social media was important to spurring opting out. A principal from a high poverty district claimed that, “The community has a Facebook page and that’s where opting out gained traction.” Several interviewees also mentioned that leaders of the opt-out groups posted opt-out form letters on social media sites that parents could submit to their schools, thereby facilitating the process. A few of these groups were also able to raise funds to put up billboards with anti-PARCC messages. Lastly, these groups were active in public meetings and forums. Members of these groups attended school board meetings to raise their concerns. A special interest group representative said, “We also went around from town to town and met with parents to answer questions [about opting out] and share our experiences.”

By contrast, the groups who supported annual testing were far less visible. OnegroupthatrespondentsidentifiedwasacoalitioncalledWeRaiseNew Jersey that formed in response to the grassroots parent organizing. Members of the coalition include the Jersey Campaign for Achievement Now, the New Jersey School Boards, the state PTA, and principals’ and administrators’ associations. The coalition’s strategy to promote the PARCC test included a TV advertisement campaign, raising awareness on social media, and holding public forums.

State Communications to Districts Sent Mixed Messages, Fueling Uncertainty

As the state and school districts prepared for the test administration and awareness of opting out grew, the NJDOE issued a number of statements about the testing. At the beginning of the 2014–15 school year, New Jersey Commissioner of Education David Hespe sent a memo to school leaders stating that participating in testing was required for compliance with NCLB (Hespe, Oct 2014). As the test administration period approached, the Department of Education announced that district administrators would determine how to address test refusals. Interviewees from districts and schools characterized the state’s communication of



opt-out policies as unclear. A principal from a high DFG district explained: “The state was very wishy-washy throughout the ordeal. They said that the principals would be the ones to determine what to do. They did a very poor job handling this.” District administrators also found the state’s communication of testing directions problematic. One principal we interviewed said, “We were halfway through the testing, and [the state was] changing the protocol. We were thinking at the time: we’re supposed to be starting in three weeks, and still don’t know how this is going to work.”

The NCLB policy that was in effect the spring of 2015 clearly required states to have 95 percent of students participate in state tests. This was a central tenet of the equitability of the law so that schools and districts could not manipulate test performance by having low performing students stay away during testing periods. The opt-out movement challenged this premise but New Jersey’s position regarding the consequences for districts with high rates of opting out was unclear. A member of a special interest group remarked that the memo released by the commissioner of education “restated the fact that the federal [law] required the state to administer the tests but said that the policy of dealing with test refusals was up to the districts.” An administrator from a low DFG discussed the state’s changing communication about consequences for districts that did not meet the 95 percent participation rate. The administrator said, “Originally we were told that funding could be impacted. Now it’s looking more like a corrective action plan … The corrective action plan could be additional communication to parents in the community and doing other things to increase participation. It’s a little bit fuzzy, but that’s what’s been loosely communicated to districts recently.” Caught between federal directives and a bottom-up movement, the state’s wavering messages contributed to the unevenness of local response.

VI. The Local ContextTheflurryofopt-outsandtherapidriseintheprofilesurroundingtheissue surprised principals and district leaders, who were caught in the awkward position between parents having a legitimate right to recuse their children from testing and state policy requirements for districts to advocate that all students take the PARCC test. The newness of the test, which was expanded to occur over three weeks rather than two, the new procedures for the PARCC, and the online administration, all added to the challenge. The overall picture that emerged from our interviews with local administrators was one of leaders trying to adjust to an evolving situation withinasysteminflux,withuncleardirectionsfromthestate,amidstahostof implementation challenges.

WORKING PAPER |21


Test Administration Challenges Result in Implementation Issues

Several district administrators we interviewed talked about test administration challenges. They noted that preparing for the PARCC was more time consuming than they had anticipated. Several administrators felt that Pearson — one of PARCC’s vendors for test administration — and the NJDOE had not fully resolved all the issues with test administration before disseminating it to the schools. Districts spent much time and effort preparing to administer the exam, including ensuring that there was adequate technology, determining a testing schedule that maximized instruction time but adhered to PARCC guidelines, and doing teacher professional development on test preparation and administration guidelines. “Literally all of our staff meetings were spent practicing for the test…and three to four [meetings] were devoted to administration,” said anassistantprincipalinanaffluentschooldistrict.

A principal from a district in another county reported that the testing administration rules from the state were not clear, “The rules about how you could implement the testing for high schools were extremely complicated. Not at all commensurate with how high schools naturally operate — anything from lunch schedules to bus schedules to mechanics ofhowmanyMacbookswereavailable,totechnicaldifficultiesattimeswith kids at different parts of the test not being able to see parts of the test,” she said.

Administrators at urban and suburban schools discussed technology challenges, including gaining access to adequate computers as well as training teachers and students to navigate the online test. Students were not used to testing on computers and the interface was not user-friendly, according to both school administrators and students. We heard of cases where computers crashed, forcing students to restart the test, and proctors who had to constantly log in students who had been logged out for spending too much time on one page. In some schools, weak wireless connections led to the test taking much longer than intended.

Due to shortage in computers, some high schools had to test students in waves. As a result, different students were missing on different days and in some schools it took three full weeks before classes went back to normal. Schools with larger student populations required more administrators to oversee the test as well as the continuation of the school day for those not testing, meaning that high schools had the biggest burden of adjusting lunch, bus, and class schedules and hiring substitutes to staff classrooms for teachers who were also test administrators.

One example of an unexpected obstacle that a high school principal told us about, was preparing testing tickets. Shortly before testing began, every student was assigned a testing ticket with unique login information and a password, ensuring a secure test. However, the state software had glitchescausingthefirstpageofeachsettohavedifferentsizedmargins.As the tickets did not all line up, it was impossible to cut the tickets out simultaneously on a paper cutter and so “school principals were spending



Sunday afternoons on their dining room table with an industrial size paper cutter because the state didn’t make tickets that were easy to print and slice,” the principal bemusedly told us.

Teachers and administrators feared that the messy implementation could affect the validity of scores, yet the test results would still have consequences for teacher and school performance measures. In terms of preparation, principals and superintendents mentioned that they “don’t know if more time was spent on the mechanics of it than the actual content” and that “entire weekends were devoted to pulling this off.” Parents and school-level administrators believed that implementation issues resulted in increased opt-out rates for the second testing window.

Test Anxiety

Parents and PTA leaders who represented parents, expressed a range of parental concerns during interviews, including the burdens of over-testing, the anxiety produced by testing, and concerns about developmental appropriateness of the tests. Several interviewees felt that the amount of testing in schools was excessive, in particular for students who were taking AP courses and college admissions tests. Similar opinions were expressed by teachers as well as parents. For example, a high school teacher from a high DFG district explained, “We have some juniors that are going from SAT tests on the weekend, to PARCC test, to two weeks of AP testing, all withinamatteroffourorfiveweeks.Theyjusthaditwithtestingandtheirparents were also tired of it.”

Another reason mentioned by parent organizations was that the test was nerve-wracking for students. “This [testing] was bringing on undue stress and anxiety in their children,” said a local PTA organizer. She further emphasized that test anxiety was a particular issue with parents of special education students, who were concerned their children would be required to take the tests without accommodations. A high school English teacher from a high DFG district stated that “kids are stressed with the college application process, the competition in the grade schools, the SAT scores, the AP scores, so I think parents are sensing all that.” Parents of younger children were concerned about the length of the test. According to a teacher, “A lot of parents who have younger kids are saying no way their kids are taking the PARCC. They’re not letting their nine-year-old sit through a 90 or 80 minute test,” she said.

Parents were also concerned with the developmental appropriateness of the PARCC test. Two parents who served as PTA members in a low-income district tried a practice test online themselves, after hearing that teachers at their children’s school could not pass the PARCC practice tests. They believedthatthedifficultyofthetestquestionswereinappropriateforthegrade level of their children.

Test opposition was not universal amongst parents, and several parents and educators we interviewed supported, or at least understood, the purpose of annual testing. One principal remarked that the parents who

WORKING PAPER |23


chose to have their children take the test believed that standardized testing “is the way of life” and serves as a measure of a student’s performance. Another principal from a high DFG added that some parents complied with testing even if “they didn’t agree with the test... [Because] they wanted to cooperate and keep the district from getting in trouble for not meeting the 95 percent participation rule [of NCLB].” Other interviewees indicated that some parents did have favorable views of the new PARCC test. A state-level education association representative mentioned that some parents chose to have their children participate in the tests because they were interested in the new assessments and wanted to see the students “assessed on better tests.”

Role of High School Students

Several interviewees discussed the role high school students played in promoting opting out. A parent from a high DFG district explained that two high school student representatives attended a board meeting where the PARCC test and opting out policies were discussed and “after attending this meeting, the two students went back to school and told other students that they could opt out of the test.” Similarly, a principal of ahighschoolinahighDFGdistrictsaidthat,asaresultofthefirstgroupof student testers talking about the technology problems with the test administration, “more students opted out across the days the test was given.” One student we interviewed said that he became involved in the issue after he researched the PARCC test on the Internet and came to believe that “This [PARCC test] was a high-stakes test, and it served no instructional purpose. It was simply a way for companies to gain money,” he told us. Utilizing social media, the student informed his peers about the PARCC test and opting out even though the school administrators had announced that students were not allowed to opt out. As the testing period approached, the student further encouraged opting out by posting an opt-out form on social media that students could have their parents sign and submit to the school.

VII. The Role of Social Media in the Opt-Out MovementMany of the interviewees attributed the success of the opt-out movement to grassroots organizing led by parents who were savvy in their use of social media and represented themselves as parents even if they had a dual role as an educator. One leader of an opt-out advocacy group stated, “Opt-out rates are all about how the parents are organized in their communities.”

Parents from wealthier districts who supported opting out described themselves as “connected”, “informed”, and “concerned about testing.” Interviewees also noted that the parents used social media to share messages about the PARCC test and opting out. A special interest group



representative remarked that “the opt-out parents used [social media] a lot more than the other side because anytime someone said something like ‘our kids have always taken tests, it’s not going to hurt them’ the opt-out groups would attack them on social media.” In presenting the issue, the educators who were members of opt-out advocacy groups took care to speak as parents. An opt-out advocacy group leader revealed, “We swung back and forth standing in shoes as parents and educators. Parents as educators were a huge mover in the beginning of this movement.”

To explore the extent to which social media contributed to the opt-out phenomenon, we examined the Twitter activity of several of the main actors in New Jersey’s opt-out movement. We collected data on the volume of activity and topics of the conversations that these groups engaged in on Twitter using Twitonomy, an online Twitter analytics tool that provides data on users’ tweets, retweets, replies, mentions, and hashtags. Through our analysis we were able to get a sense of the amount of activity on Twitter related to the opt-out movement, the most common topics of conversation, and whom the actors were most frequently acknowledging or addressing.

We used local newspaper coverage of New Jersey’s opt-out movement to identify the major organizations and advocacy groups that were involved. We analyzed the Twitter activity

of four advocacy groups: We Raise New Jersey (@WeRaiseNJ), New Jersey Opt Out (@NJOptOut), the New Jersey Education Association (@NJEA), and Save Our Schools New Jersey (@SavOurSchoolsNJ). The data source consisted of the Twitter activity of the four groups between January 15 and June 30 of 2015, which captures activity about a month and a half before PARCC testing began and across both PARCC testing windows.

As shown in Figure 4, the volume of activity over this time period varied for the four actors. We Raise New Jersey, an organization that supported thePARCCtest,wastheleastactiveonTwitterduringthisfive-monthperiod. We Raise New Jersey, which did not become active on Twitter until late February, had only 276 tweets. Most of We Raise New Jersey’s tweets during this time frame, approximately 157, occurred in late April, which was just before the second round of PARCC testing. In contrast, the opt-out advocacy groups had larger volumes of activity during this timeperiod.OptOutNewJerseyhad905tweetsoverthefivemonths,the NJEA had 1,564 tweets, and Save Our Schools New Jersey had 2,498 tweets. These groups had surges of activity on Twitter across multiple months. A large share of Opt Out New Jersey’s tweets occurred in February and March, in the period of test buildup. The NJEA had a fairly consistent volume of tweets across the testing period. Save Our Schools New Jersey had ramped up its activity on Twitter in January, February, and March, and decreased its activity toward the end of March and

WORKING PAPER |25


early April. All three of these groups appear to have sent out or shared more tweets during the periods before both testing periods. This aligns with comments from interviewees that the advocacy groups and parents ramped up their social media presence in the periods before testing.

There was also evidence of coordination amongst the groups who advocated opting out. NJEA, Save Our Schools New Jersey, and Opt Out New Jersey retweeted and mentioned each other’s tweets and communicatedwithsimilaractorsduringthefive-monthtimeperiodthatwe examined. Both opt-out advocacy groups had the NJEA among its users most retweeted, mentioned, and favorited, which suggested that the these groups were disseminating messages from the teachers’ union to their followers. For the NJEA, Save Our Schools New Jersey was among the top ten users it retweeted and mentioned, so the teachers’ union also appeared to have shared messages from the advocacy group. Thisfindingalignswithstatementsmadebyseveralintervieweeswhorepresented special interest groups, that the union and advocacy groups were sharing messages on social media and working together to inform their followers.

By contrast, there did not appear to be much communication between the groups advocating for opting out and the groups advocating for taking the PARCC test. We Raise New Jersey was not in the top ten users retweeted, mentioned, or replied to for the two advocacy groups and the teachers’ union. Likewise, We Raise New Jersey was not mentioning, or replying to the opt-out advocacy groups or the union frequently during this time frame. We Raise New Jersey tended to retweet and mention users that were also educational advocacy groups or those who are pro-testing including New Jersey Can, the New Jersey Black Alliance for Educational Opportunities, Advocates for Children of New Jersey, Education Trust, the New Jersey Principal and Supervisor’s Association, the Urban League, and the Urban League of Essex County. These organizations did not appear in the top ten most users retweeted, mentioned, or replied to for the opt-out advocacy groups. Although, this data set only included one group that supported the PARCC test, this analysis does highlight the nature of discourse on social media in which users share information with their followers that represent one ideology or faction and rarely engage with individuals or groups that hold the opposite position. In this way, social media allows groups to coordinate their messages and protect their constituents from groups and individuals on the other end of the spectrum.



VIII. Policy ImplicationsApproximately 135,000 students did not take the PARCC assessments in New Jersey in the spring of 2015. Depending on how it is calculated, we estimated this to be between 11 to 19% of the population of students eligible for testing in grades 3 to 11 in New Jersey. The sheer magnitude of this number and the attention drawn by the parents of these children through their actions demands that policymakers take pause and consider the larger trends that led to the bursting of the testing bubble of 2015. What precipitated this rebuke of the way the education system is organized and what can be done to recalibrate the system to better serve the goals of higher quality education for all students?

The circumstances surrounding the 2015 test administration and resulting opt-out movement surfaced a host of frictions within the education system as it is currently constructed. First, the movement calls attention to the need to balance parental rights with the desire to capture a realistic measure of system performance. While no one is contesting parents’ right to recuse their children from testing, it spotlights a tension between parental rights and policymakers’ desire for high participation rates so that all students participate to give an accurate representation of school performance. The 95 percent participation rates and subgroup disaggregationrequirementsthatwerecodifiedintoNCLBcamefromadesire to foster more equitable attention to all students and discourage educators from depressing low performing students’ participation and hiding poor subgroup performance in overall averages. The irony is that currently it is students in wealthier districts who are opting out at larger rates than those in poorer districts, which depresses state average performance. But this may not always be the case. Also, we know little about the within-school distributions of students opting out.

Second, alignment and coherence have been watchwords in the education policy community for at least two decades. States have been seeking to align a variety of policy instruments, including standards, assessments, and accountability, under the theory that incoherence is one of the barriers to improved performance and that alignment between goals (standards), measures (assessments), and incentives (accountability) would move educators in the same direction and create mutually reinforcing synergies that would improve system effectiveness. The CCSS movement and federal and state endeavors to push implementation forward represents a new level of effort to increase coherence. Yet, policy attempts to increase coherence are perceived by local implementers as increasingly prescriptive, removing local latitude andlocalflexibilitytorespondincontext.Theeffortstoalignmaterials,assessments, and accountability underneath a common set of standards are increasingly viewed as a straight-jacket by schools and teachers.

Third, as the experience in New Jersey shows, there can be deleterious and unintended consequences to outsourcing a statewide assessment to an external vendor, because of lack of information sharing about other functions of the education system. This can be seen in the confusions that

WORKING PAPER |27


arose about the windows for the PARCC test and other tests administered by the state. In this case, the increase of testing time required by PARCC’s desire to develop a more robust test, which included a performance component in addition to the more traditional multiple choice, open-endedresponsecomponent—createdconflictswithothertests,particularly in high school. These, and other testing issues may have been firstyearglitches,buttheycontributedtotheperceptionofinconsistencybetween the test implementation and other state policies and practices.

Fourth, the opt-out movement is a reminder to policymakers that there are consequences for not creating a smooth pathway, including adequate time and support, for policies to take root. While the CCSS were firstadoptedbystatesin2010,schoolsanddistrictshavehadfitfultrailsto implementation. Relatively few materials and supports were initially available to schools and teachers and, to this day, experts view relatively few textbooks to be CCSS aligned (Polikoff, 2015). Teacher change and building teacher capacity to instruct to more ambitious standards is also aslowerprocessthatrequiresastableenvironmentandasteadyinfluxof resources, which many strapped school districts do not have. Further, in2015,thePARCCtestwasinitsfirstyearofrollout,butNewJerseywasalready intending to use it for teacher accountability, before students had even experienced the test. One way to interpret the opt-out outburst is that educators felt that accountability was not reciprocal, that they were not given the tools, training, and resources they were promised in order to be legitimately held accountable for the performance of their students.

Amidst these issues, the opt-out movement has surfaced even larger questions about the role of testing in an improving education system. For decades now, America has grappled with the appropriate role of assessment in our education system. In the 1990s, states including Kentucky, Vermont and Maryland dabbled with portfolios and other alternative forms of assessments in the hopes of widening the forms by which students represented their knowledge and skills. Despite greater validity, these efforts faced score reliability concerns, domain coverage challenges,andrelativelyhighfinancialcosts.

Most recently, beginning in 2001 with the passage of the federal NCLB Act, states escalated test-based accountability to be a — if not the — central mechanism for educational improvement. Since then, we have seen a dramatic increase in both the frequency and stakes — and consequently the attention — attached to testing. The frequency of testing increased from key stages (grades 4, 8, and 11) to annual (grades 3 to 8 and at least one year in high school), and the length of testing expanded as well. Accountability was also ratcheted up as schools were judged by the extent of their yearly progress and the unit of accountability moved from the district to the school to the teacher as value-addedmethodshavebecomeincreasinglytechnicallyrefined.Schoolsuccesshasbecomedefinedbyitstestscores.

Test-based accountability is seductive to policymakers because it is a relatively inexpensive way to direct the behaviors of district and school



leaders, and teachers. As abundant research has shown, educators are incredibly responsive to high stakes testing. Attaching stakes to tests incents teachers to align instruction to standards, provide more instructional time in tested subjects, and cover the material that will be on the test (Hamilton, 2003). The effectiveness of testing incentives are plain to see each spring as teachers in millions of classrooms across the country dramatically change their instruction in the months leading up to the state test to focus lessons on test preparation and test taking skills. Without doubt,testinginfluencesbehavior.

However, we also have learned from the NCLB era that the prods of high stakes testing carry also with them a host of undesirable and unproductive responses in addition to those that policymakers intend. Research on the negative effects of attaching heavy consequences to testing show many undesirable effects. These include increased time spent doing low-level test preparation activities, the crowding out of subjects and activities that nurture the development of the whole child, and enough cases of outright cheating across the country to demonstrate that pressure without concomitant capacity can lead to desperation (Nichols & Berliner, 2007).

The national experiment with test-based improvement that we have just experienced should make us realize is that a system that is too heavily dependent on testing as an accountability mechanism distorts the productive behaviors that we seek to incent with such a system. The allure of the “next generation” assessments like PARCC and SBAC is that they would resolve some of the issues that had beset prior assessment regimes. The tests would be more aligned with college and career preparation and they would bring testing into the digital age with online capabilities. Yet these are merely test-based solutions to the bigger problem of how to calibrate pressure and capacity to foster system-wide improvement. The opt-out movement demonstrates that the problem that testing is intended to address — namely the provision of information about the progress of students and the system and a press for educators to focus on academic goals — is actually being exacerbated by over-attention to testing. By focusing so heavily on testing outcomes, the system has lost its balance. The 2015 opt-out movement is a wake-up call for policymakers that it is time to redress the testing imbalance.

WORKING PAPER |29


IX. EpilogueThe analyse of this report are based on 2015 assessment data and interviews conducted in the aftermath of the 2015 testing cycle. Our paperfoundthatinNewJerseytherewerepositiveandsignificantcorrelations between school districts’ opt-out rates in ELA and the DFGs (or measure of district wealth) across grade levels. We also saw that this phenomenon in New Jersey was symptomatic of a larger, national backlash against testing, Common Core, and the shift away from local control to federal oversight of schools. This epilogue is an update on those findings.

In the spring of 2016, several important testing administration and content changes were implemented in New Jersey. For example, the two 2015 PARCC testing windows, one in March and one in April-May, were reduced to a single testing period in April-May of 2016. Also, the PARCC tests were 90 minutes shorter. In addition, the state also worked to release results sooner. While PARCC scores for the 2014–15 school year were not released until mid-November of the following school year, districts received 2016 preliminary data in June and individual student scores in mid-August, with parents expected to receive their students’ score reports inSeptember(StateofNewJerseyOfficeoftheGovernor,2016).

Other state policy changes occurred in 2015–16 in the wake of the opt-outmovement.Specifically,alawwaspassedinNovember2015prohibiting the withholding of state funding from school districts because of low student-participation rates on state assessments. The bill was introducedtopreventdistrictsfrombeingfinanciallypenalizedforhighopt-out rates, which they did not have full control over, as parents had the finalsayintheirchildrenoptingout.Thebillsolelyconcernsstatefunding,meaning that there is still a possibility that federal funds will be withheld as the consequence for districts not meeting the 95 percent assessment-participation requirement stipulated in NCLB/ESSA.

The Education Law Center (ELC) and the American Civil Liberties Union ofNewJersey(ACLU-NJ)filedapetitioninSeptember2015inresponseto the changes the NJDOE made to graduation requirements without using appropriate procedures, namely, by bypassing the legislature. This lawsuit was settled in May 2016 with the NJDOE acknowledging that it did not follow proper procedures in changing graduation requirements, which resulted in 10,000 seniors having to seek portfolio approval — the alternative to passing PARCC or another approved test — in order to obtain a diploma. In response to the recently proposed changes to make PARCC a graduation requirement, the ELC and the ACLU-NJ published comments reiterating their concern with NJDOE creating graduation requirements while circumventing the legislative process, and arguing that the use of the Algebra 1 and tenth grade English assessments do not effectively measure students’ readiness for college and careers, as they are administered early on in high school (ELC and ACLU-NJ, 2016).

Despite their concerns, the New Jersey Board of Education voted on August 3, 2016 to make passing scores on the tenth grade English and



Algebra 1 PARCC exams the requirements for graduation, beginning in 2021. There is also pending legislation requiring school districts and the NJDOE to post student participation rates on their websites. The bill was passed in late May by the Assembly, almost unanimously, and is currently in the Senate Education Committee. And, on August 31, 2016, the NJDOE announced that the PARCC exam will account for 30% of a teacher’s evaluation if their teaching assignment is 4th–8th grade English language arts and/or 4th–7th grade mathematics.

Finally, compared to 2015, the 2016 PARCC results indicated a higher percentage of students met or exceeded expectation on the PARCC exam, and there were more students participating. The state reported that approximately 66,000 more students took the mathematics PARCC exam and 57,000 more students took the English language arts PARCC exam during the 2016 administration. This increase in the number of students testing was most notable in the high school grades, where, for example, approximately 29 percent more students took the Algebra II examin2016thanin2015(StateofNewJerseyOfficeoftheGovernor,2016). However, the released data only shows the number of students who took the test, not participation rates. This means we are unable to determine whether the increase in tested students was due to changes in student enrollment or due to increased participation and to what extent. Therefore we cannot conclude that the PARCC participation rates increased from 2015 to 2016 by looking at these data alone. CPRE plans a follow up study to examine opt-out rates in 2016 compared to 2015.

WORKING PAPER |31


References

Cohen,D.K.,&Moffitt,S.L.(2009).The ordeal of equality: Did federal regulation fix the schools? Cambridge, MA: Harvard University Press.

Education Law Center and the American Civil Liberties Union of New Jersey. (2016). Comments on proposed amendments to N.J.A.C. 6A:8, proposal number PRN 2016-072 submitted by Education Law Center and American Civil Liberties Union of New Jersey. Newark, NJ. Retrieved from http://www.edlawcenter.org/assets/files/pdfs/Newsblasts/Comments%20on%20Proposed%20N.J.A.C.%206A%208%20-%20ELC%20and%20ACLU%20NJ.pdf

Engdahl, T. (2015, July 27). After widespread test protests last fall, opting out spread during spring exams. Retrieved from http://www.chalkbeat.org/posts/co/2015/07/27/after-widespread-test-protests-last-fall-even-fewer-colorado-students-took-spring-tests/#.V1BFDlZgNuZ

Gamson, D. A. (2015). Differences in common: A brief history of American educational standards. In J.A. Supovitz & J. P. Spillane (Eds.). Challenging standards: Navigating conflict and building capacity in the era of the Common Core(pp.133–44).Lanham,MD:Rowman&Littlefield.

Hamilton, L. (2003). Assessment as a policy tool. Review of Research in Education, 27, 25–68.

Hamilton, L. S., Stecher, B. M., & Yuan, K. (2008, December). Standards-based reform in the United States: History, research, and future directions. Santa Monica, CA: RAND Corporation.

Harris, E.A. (2015, August 12). 20% of New York State students opted out of standardized tests this year. New York Times. Retrieved from http://www.nytimes.com/2015/08/13/nyregion/new-york-state-students-standardized-tests.html.

Harris, E.A. (2016, August 3). New Jersey ties graduation to tests aligned with Common Core standards. New York Times. Retrieved from http://nyti.ms/2b31kgK

Hespe, D. (2014, October 30). Student participation in the statewide assessment program [Memorandum]. Trenton, NJ: New Jersey Department of Education. Retrieved from https://education.state.nj.us/broadcasts/2014/OCT/30/12404/Students%20Participation%20in%20the%20Statewide%20Assessment%20Program.pdf

Jennings, J. & Rentner, D. S. (2006). Ten big effects of the No Child Left Behind Act on public schools. Phi Delta Kappan, 88(2), 110–113.

Johnson, R. B., & Onwuegbuzie, A. J. (2011). Mixed research. In R. B. Johnson & L. B. Christensen (Eds.), Educational research: Quantitative, qualitative, and mixed approaches (4th ed., pp. 439–59). Thousand Oaks, CA: SAGE.

Koretz, D. M. (2008). Measuring up: What educational testing really tells us. Cambridge, MA: Harvard University Press.

McDonnell, L. M., & Weatherford, M. S. (2013). Organized interests and the Common Core. Educational Researcher, 42(9), 488-97. doi:10.3102/0013189x13512676

McGuinn, P. J. (2006). No Child Left Behind and the transformation of Federal Education Policy. 1965-2005. Lawrence, KS: University Press of Kansas.

McGuinn, P., & Supovitz, J. (2016). Parallel play in the education sandbox: The Common Core and the politics of transpartisan coalitions. New America, 1-19. Retrieved from https://static.newamerica.org/attachments/12298-



parallel-play-in-the-education-sandbox/common_core_1.13.e146c9a3e44b4dcea030bf4ac1efda59.pdf

New Jersey Department of Education. (2015a). New Jersey statewide assessment reports. Trenton, NJ: New Jersey Department of Education. Retrieved from http://www.nj.gov/education/schools/achievement/15/parcc/excel.htm

New Jersey Department of Education. (2015b). DOE data: 2014-2015 enrollment. Trenton, NJ: New Jersey Department of Education. Retrieved from http://www.state.nj.us/education/data/enr/enr15/stat_doc.htm

New Jersey Department of Education. (2015, August). Year two of PARCC: Parent PARCC questions answered. Trenton, NJ: New Jersey Department of Education. Retrieved from http://www.state.nj.us/education/assessment/PARCCFAQ.pdf

New Jersey Education Association. (2015, February 17). NJEA ads target standardized tests: Parents, teachers push back hard on PARCC. Retrieved from http://www.njea.org/news/2015-02-17/njea-ads-target-standardized-tests

Nichols, S. L., & Berliner, D. C. (2007). Collateral damage: How high-stakes testing corrupts America’s schools. Cambridge, MA: Harvard Education Press.

Polikoff, M. S. (2015). How well aligned are textbooks to the Common Core Standards in mathematics?. American Educational Research Journal, 52(6), 1185–211. doi: 10.3102/0002831215584435

Smith, M. S., & O’Day, J. (1991). Putting the pieces together: Systemic school reform (CPRE Policy Brief, RB-06-4/91). New Brunswick, NJ: Consortium for Policy Research in Education.

StateofNewJerseyOfficeoftheGovernor.(2016).GovernorChristineannouncesimprovedstatewidePARCCresults in year two [Press release]. Retrieved from http://nj.gov/governor/news/news/552016/pdf/20160802a.pdf

Supovitz, J. (2009). Can high-stakes testing leverage educational improvement? Prospects from the last decade of testing and accountability reform. Journal of Educational Change, 10, 211–27.

Supovitz, J., Daly, A., & Del Fresno, M. (2015, Feb 23). #commoncore Project. Retrieved from http://www.hashtagcommoncore.com

Supovitz, J. A., & Spillane, J. P. (2015). Challenging standards: Navigating conflict and building capacity in the era of the Common Core.Lanham,MD:RowmanandLittlefield.

Ujifusa, A. (2015, July 9). Smarter Balanced opt-out rates top 25 percent for Washington State 11th graders. Education Week. Retrieved from http://blogs.edweek.org/edweek/state_edwatch/2015/07/smarter_balanced_opt-out_rates_top_25_percent_for_wash_st_high_school_juniors.html?override=web

U. S. Department of Education. (2014, August 14). Race to the top assessment program. Washington, DC: U.S. Department of Education. http://www2.ed.gov/programs/racetothetop-assessment/index.html

Vinovskis, M. A. (1996). An analysis of the concept and uses of systemic educational reform. American Educational Research Journal, 33(1), 53–85. doi:10.3102/00028312033001053

APPENDICES



Appendix A

Research Design

The purpose of this study was to examine the scope, factors, and context for the opt-out movement that occurred in New Jersey in the spring of 2015. Our overall investigation focused on addressing three essential questions:

1. What was the scope of students opting out of the PARCC test in grades 3 to 11 in ELA and mathematics in the spring of 2015 in New Jersey, and were opt-out patterns different across the diverse communities in the state?

2. What were the major national, state, and local factors that contributed to the opt-out phenomenon?

3. What are the policy implications for New Jersey and other states?

In this study, we employed mixed methods research to understand the nature of the opt-out movement in New Jersey from state, district, and individual perspectives.

Data Collection and Analysis

Interview Data

Researchers conducted one-on-one, semi-structured interviews with individuals on the state, district, and local level in September and October of 2015. Thirty interviewees included state policymakers, professional education association representatives, advocacy group leaders, school administrators, teachers, parents, and students. Researchers used one of two semi-structured interview protocols, one for state level individuals and one for district, school, or student participants. Most interviews were 30–60 minutes in length, digitally recorded, and transcribed.

To identify an interview sample we determined that our interviewees would come from participants representing school districts from either low or high socioeconomic status. We used District Factor Groups (DFGs), the New Jersey Department of Education’s (NJDOE) indicator of district’s socioeconomic status. New Jersey has approximately 539 operating school districts. The NJDOE developed the DFGs for the purpose of identifying districts with similar characteristics in order for districts to be compared in terms of their population and socio-economic status as reported by census data. There are a total of eight DFG categories, the ‘A’ and ‘B’ DFG categories represent districts with the lowest socio-economic status and the ‘I’ and ‘J’ categories are districts with the highest levels of socio-economic status. For the purpose of this report, researchers divided the DFGs groups into low or high. Researchers considered low DFG districts to be A, B, CD, and DE districts whereas highDFGdistrictswereidentifiedasFG,GH,I,andJ.

Researchers employed sequential design analysis, the qualitative and quantitative component were conducted one after the other, for the study (Johnson & Onwuegbuzie, 2011). First, we conducted the qualitative analysis of the interview data. Immediately following each interview, researchers crafted memos basedontheirinterviews.Usingthekeythemesandconceptsidentifiedinthememos,researchersdevelopedacodingframeworktoanalyzeinterviewdata.Codeswerefurtherrefinedthroughaninteractiveprocessofidentification,definition,andrefinement.Fourteammembersusingasecure,cloud-basedplatform,thencoded each interview transcripts.

WORKING PAPER |35


Test Data

The data to calculate opt-out rates were released by the NJDOE in February, 2016 and were downloaded asExcelfilesbyschool,grade,andsubjectlevelfromtheNJDOEwebsite.Wethencleanedtheexcelfilesbyremovingembeddedsubtotalrows.OuranalysisfocusedonthethreefieldsfromtheNJDOEdata:(1)the number of students who had valid scores; (2) the number of students registered to be tested; and (3) the district’s state designated factor group (DFG).

We brought the data into the SPSS statistical analysis software. We then aggregated the data by subject and district overall; and by grade ranges (for ELA grades 3-5, 6-8, 9-11, and for mathematics grades 3-5, 6-8, Algebra I, Geometry and Algebra II). To produce the statewide estimates of students did not take the state test, we simply took the overall number of the total students who were registered to be tested as a proportion of the total number with valid scores. To produce the district distributions, we computed the percent opt-out for each district as the ratio of the number of student who were registered to be tested as a proportion of the total number with valid scores. Before reporting the district data, we collapsed the data into four bands: less than 6% of students opting out, 6-15% of students opting out; more than 15 and less than 25% of students opting out; and more than 25% of students opting out. All schools had students with valid scores, based on the number of students who completed the PARCC test. However, many districts had missing data for the number of students who were registered to be tested. These districts were denoted as having missing data, which accounted for the high levels of missing district data in Figures 1 and 2 and Appendix B.

To run the correlations presented in Table 1, we used the raw opt-out percentage for each district and assigned each DFG an ordinal number (where DFG A=1, DFG B=2, DFG CD=3, DFG DE=4, DFG FG=5, DFG GH=6, DFG I=7, and DFG J=8). We did not include charter or vocational schools in these correlational analyses.

After producing the results for the state provided data, we then downloaded NJDOE 2014-15 state district enrollment data by school. We found that the enrollment data reported by the state was very similar (between ± 2 percent) of the registered to test data. Therefore, we merged the enrollment data only for those schools that had missing registered to test data and reran the entire set of analyses using the more complete dataset.

Twitter Data

We examined publicly available Twitter data from a few of the major organizations and central advocacy groups to determine how New Jersey’s opt-out movement was communicated and represented on social media. We used local newspaper coverage of New Jersey’s opt-out movement to identify the major organizationsandadvocacygroupsthatwereinvolvedintheopt-outmovement.Weidentifiedfouradvocacygroups: We Raise New Jersey (@WeRaiseNJ), New Jersey Opt Out (@NJOptOut), the New Jersey Education Association (@NJEA), and Save Our Schools New Jersey (@SavOurSchoolsNJ). We downloaded the tweets from these groups and the major hashtags related to Twitter conversations as well as the information-sharing surrounding the opt-out movement in New Jersey as made available by Twitonomy. Twitonomy is an online Twitter analytics tool that provides data on users’ tweets, retweets, replies, mentions, and hashtags. The data source from Twitonomy consisted of the Twitter activity of the four groups between January 15 and June 30 of 2015, which captures activity about a month and a half before PARCC testing began and across both PARCC testing windows.

To explore the extent to which social media contributed to the opt-out phenomenon, we analyzed the Twitter activity of four advocacy groups: We Raise New Jersey (@WeRaiseNJ), New Jersey Opt Out (@NJOptOut), the New Jersey Education Association (@NJEA), and Save Our Schools New Jersey (@SavOurSchoolsNJ). Our analysis included about 5,000 tweets and the major hashtags related to Twitter conversations and information-sharing surrounding the opt-out movement in New Jersey as made available by Twitonomy. Through our analysis we were able to get a sense of the amount of activity on Twitter related to the opt-out movement, the most common topics of conversation, and whom the actors were most frequently acknowledging or addressing.Thesedatawerecomparedandorganizedintoagraphthatreflectedactivityoverasix-monthperiod.



Appendix B

Opt-Out in ELA by District Factor Group

A(n=38)

B(n=65)

CD(n=65)

DE(n=82)

FG(n=88)

GH(n=73)

I(n=103)

J(n=25)

Charter(n=85)

Vocational(n=20)

100%

75%

50%

25%

0%

Opt-Out in ELA by District Factor Group

Nu

mb

er a

nd

Pe

rce

nt

of D

istric

ts in

Ea

ch

Ba

nd

By Grade Range Charter & Vocational All

All(n=644)

6(16%)

6(16%)

11(29%)

3(8%)

11(29%)

8(12%)

4(11%)

26(40%) 32

(36%) 18(25%)

26(25%)

10(40%)

3(15%)

254(39%)25

(38%)

6(9%)

28(34%)

75(88%)

4(20%)

18(28%)

13(20%)

23(28%)

23(26%)

23(32%)

34(33%)

3(12%)

8(40%)

158(25%)

15(23%)

12(15%)

17(19%)

16(22%)

25(24%)

7(28%)

3(15%) 111

(17%)

7(11%) 12

(18%) 19(23%)

14(16%)

12(16%)

16(16%)

4(16%)

2(10%) 98

(15%)

6(7%)

2(2%)

2(2%)

4(5%) 2

(2%)

1(4%)

23(4%)

2(2%)






University of Pennsylvania3440 Market Street, Suite 560Philadelphia, PA 19104Tel: 215.573.0700

cpre.org

The Bubble BurstsTHE 2015 OPT-OUT

MOVEMENT IN

NEW JERSEY

The Bubble Bursts - CPRE : Consortium for Policy Research · PDF file ·...

Documents

Transcript of The Bubble Bursts - CPRE : Consortium for Policy Research · PDF file ·...