The struggle to technicise in education policy

Radhika Gorur • Jill P. Koyama

Received: 4 May 2012 / Accepted: 26 August 2013 / Published online: 4 September 2013

� The Australian Association for Research in Education, Inc. 2013

Abstract In contemporary education policy, simplified technical accounts of policy

problems and solutions are being produced with the use of numeric calculations. These

calculations are seen as clear and unbiased, capable of revealing ‘‘what works’’ and

identifying ‘‘best practices.’’ In this piece, the authors use resources from the material-

semiotic approach of actor-network theory to discuss how calculations have begun to

serve as a subtle infrastructure underpinning the way we understand and organise our

world. They demonstrate the usefulness of the approach in tracing the technicisation of

policy by deploying it to qualitative studies of like-school comparisons in the two

unexpectedly linked locations—New York City and Australia. The authors reveal how

technical accounts are precarious and need constant maintenance to endure, even as

they increasingly becoming routine, curtailing the policy imagination and limiting the

spaces of contestation. It is for this reason, they argue, that a deeper understanding and

sustained critique of such accounts is of pressing importance.

Keywords Education policy � Actor-network theory � Accountability �Transparency

Introduction

In a quest to increase the effectiveness of education systems and to gain public

trust and confidence, contemporary education policy is characterised by a strong

R. Gorur

The Victoria Institute, Victoria University, Melbourne, VIC, Australia

e-mail: Radhika.Gorur@vu.edu.au

J. P. Koyama (&)

Educational Policy Studies and Practice, College of Education, University of Arizona, Tucson, AZ,

USA

e-mail: jkoyama@email.arizona.edu

123

Aust. Educ. Res. (2013) 40:633–648

DOI 10.1007/s13384-013-0125-9

desire to generate and use precise and reliable information. As Mulgan (2003)

argues:

Governments have become ravenous for information and evidence. A few may

still rely on gut instincts, astrological charts or yesterday’s focus groups. But

most recognise that their success—in the sense of achieving objectives and

retaining the confidence of the public—now depends on much more

systematic use of knowledge than it did in the past.

This ‘evidence’ often takes the form of numeric and comparative accounts.1

National and international surveys yield a slew of tables, charts and comparative

data, and acute attention is increasingly paid to comparative international measures

of educational attainment (Klenowski 2009). According to Lingard (2011), current

governance and political steering of education policy have elevated the role of

numbers. Huge investments are made into the production of such numerical

accounts.

A technical and quantified ‘evidence-based’ approach to policy and a focus on

‘what works’ has come to be seen as efficient and necessary practice, as well as a

practical morality. Labaree (2011) suggests that weak professions, like education,

concerned with practical and solution-oriented research, are particularly susceptible

to the use of statistics to legitimise claims and generate trust. In education,

statistics—and, in particular, comparative calculations—are used in an attempt to

impose order within a field that is complexified by the inter-related, the local, the

specific and the idiosyncratic. Setting aside such issues as relationships, affect,

inspiration, motivation and classroom ecology, and controlling for factors known to

be associated with poor performance, such as race and poverty, comparative

accounts pursue the task of isolating (often deeply inter-connected) problems in

order to identify discrete and definitive solutions.

Although making decisions and developing policy based primarily on calcula-

tions is admittedly limited, policymakers persist in using such data as straightfor-

ward and logical. Relying on quantifiable measures, they aim to depoliticise and

scientise policymaking (Lather 2005). The processes of standardization, quantifi-

cation, and comparison remove ‘distracting’ (but possibly critically important)

variations to promote simple, clear accounts of policy problems and solutions. The

rationality and apparent universality of ‘unprejudiced’ evidence serve to project

policy as a technical assemblage, based on a high level of expertise that is morally

neutral and undistracted by politics, ideology or prejudice.

While the use of numerical accounts may aim to depoliticise processes and

practices, their use is part of what Rose (1999) argues are powerful political

numbers. Numbers, he reasons, are not merely used in technologies of government,

but rather they are ‘part of the techniques of objectivity,’ in which ‘the apparent

objectivity of numbers, and of those who fabricate and manipulate them, helps

configure the respective boundaries of the political and the technical’ (p. 198).

Technicisation is, in fact, crucial for what Power (1997) refers to as the modern

1 Several terms and words first appear in quotations to designate that they are contested and should not be

taken at face value.

634 R. Gorur, J. P. Koyama

123

‘audit state,’ in which modes of quantification are determined by those who exercise

their public authority. Objectivity is said to be achieved through the systemic

enumeration—or technicisation–of complex processes.

In this article, we highlight the struggles to technicise policy by tracing the

fortunes of ‘like-school comparisons’ in the two unexpectedly linked locations—

New York City (NYC) and Australia. We introduce attempts to establish the

epistemic and moral authority of these numeric comparisons, and the challenges that

are posed to these attempts. We demonstrate how, at each location, techinicised

accounts start to come apart as new actors bring unexpected elements into play,

introducing emotion, challenging expertise, questioning motives and resisting the

simplifications that are produced by comparative calculations. We reveal that not

only are quantitative accounts less clear and not as ‘objective’ as they first appear,

local knowledge, which is (temporarily) displaced by technical accounts of various

groups, rises to challenge and disrupt them.

This paper draws upon two complementary in depth qualitative studies, one

conducted in NYC by the second author between June 2005 and October 2010, and

the other in Australia, between 2007 and 2010, by the first author. Broadly, each

study aimed to answer how evidence-based policies are assembled and rendered

objective, apolitical and authoritative. The link made in 2008 between the policy

problems and solutions in NYC and Australia by Julia Gillard, then Federal Minister

for Education, and Joel Klein, then Schools Chancellor of New York, provided the

specific empirical context which links our two studies.

Framing technicisation in education

Our analysis draws upon the conceptual resources of material semiotics, more

particularly actor-network theory (ANT).2 It is particularly useful in the study of

controversies (Latour 2005; Venturini 2010), characterised by the struggle of

various groups to establish the authority and legitimacy of ideas and practices.

Deployed in policy study, an ANT analysis can trace how policy phenomena

emerge as contingent effects of socio-material practices, how certain policy ideas

come to cohere as more-or-less durable assemblages or networks, and how they are

mobilised, challenged, defended and strengthened. Informed by Jasanoff (2005), we

focus on the current struggles to promote like-school comparisons as authoritative,

technical and apolitical, as well as the publication of these comparisons as an

appropriate register of accountability. We interrogate the value accorded to the

expertise of statisticians and the practice of using data comparisons to guide and

design policy. We also reveal the attempts to challenge and resist these moves.

We trace ‘the specific materializing processes through which policymaking

actually works to animate educational knowledge, identities, and practices’

(Fenwick and Edwards 2011, p. 710). Following Callon and Muniesa (2005), we

2 As Law (2007) reminds us, ANT has been taken up by different researchers in different ways. Rather

than a single, coherent or strong ‘theory’, Law suggests that ANT is ‘a sensibility to the messy practices

of relationality and materiality of the world,’ bringing with it ‘a wariness of the large-scale claims

common in social theory.’ (p. 2).

The struggle to technicise in education policy 635

123

approach ‘materiality’ as pertaining to the investment of observable and measurable

attributes to abstract phenomena, such as quality, in order to render them coherent

and calculable. We consider policy texts, particular devices such as like-school

comparisons, websites, and expertise not only as vehicles which inscribe and

translate human agency into durable and distributed effects, but also as actors. For

instance, like-school calculations serve to cohere and promote certain understand-

ings whilst discounting others. They serve to organise thinking and sort information.

With human investment, they translate the complexities of schooling into a limited

set of discrete entities with observable and measurable attributes.

Enriching our analysis further are some key concepts elaborated by Callon et al.

(2001) who illustrate how the confidence of technical solutions to technicised policy

dilemmas may come to be challenged by diverse and lay actors and reassembled as

socio-technical controversies. In our study, the translation of school quality and

equity into the like-school comparative league tables depends on the work of a small

group of experts in statistics and psychometrics. So complex and specialised is their

expertise that the actual process by which like-school comparisons are produced is a

black box; we are required to accept the result, but the process itself is too technical

for most to understand. Indeed, such calculations are not only inaccessible to non-

experts in terms of comprehending them, but also in challenging them. Yet when

these calculations are made available, the public becomes ‘informed’ and is able to

debate and challenge the use of data in policymaking. The confident technical

accounts begin to unravel, creating productive ‘spaces of uncertainty’ (Callon et al.

2001) where diverse groups bring new ideas and concerns into the policy arena,

elaborate the problem and the range of considerations, and seek better solutions. We

describe the nature and extent of such challenges to the certainty and confidence of

technicisation, and the way these challenges are managed, at our two sites.

Law (2009) describes ANT as an approach rather than a theory—one that prefers

to describe rather than explain. He sees ANT as a ‘toolkit for telling interesting

stories about and interfering in’ relations in assemblages (p. 141–142). ‘More

profoundly,’ he adds, ‘it is a sensibility to the messy practices of relationality and

materiality of the world’, which is ‘wary of large-scale claims’ (p. 142). Latour

(2005), too, argues that ANT is not a theory that is overlaid on empirical sites, but a

sensibility that is deployed in understanding phenomena. Consequently, there is no

one way of doing an ANT analysis. Law contends that it is better to talk of ‘material

semiotics’ rather than ANT, because:

… [I’ve talked of ‘it,’ an actor-network theory, but there is no ‘it.’ Rather it is

a diaspora that overlaps with other intellectual traditions…. [I]t is better to talk

of ‘material semiotics’ rather than ‘actor network theory.’ This better catches

the openness, uncertainty, revisability and diversity of the most interesting

work. Thus the actor network successor projects are located in many different

case studies, practices, and locations done in many different ways, and draw

on a range of theoretical resources. (Law 2009, p. 142)

Accordingly, our analysis here, while informed by ANT concepts drawn from such

thinkers as Latour, Law and Callon, focuses on ‘telling interesting stories’ and

636 R. Gorur, J. P. Koyama

123

attempting ‘interferences’ using a socio-material sensibility rather than ‘applying

ANT’ to the phenomenon under study.

Contexts

Children first in NYC

Complying with US Federal education policy No Child Left Behind (NCLB), NYC

developed an accountability system that assesses students annually (in reading/

language arts, mathematics and science) and then, based on those assessments,

determines whether a school is making adequate yearly progress, and further, how

schools compare to ‘similar’ schools across the district. This involves administering,

scoring and reporting more than 50 million standardised tests annually resulting in

an explosion of data at the school and district levels (Mandinah et al. 2006).

Managing, storing and publicly disseminating the data gleaned from assessments

cost tens of millions of dollars; the purchase and implementation of a data

management system called Achievement Reporting and Innovation System (ARIS)

cost an additional 81 million dollars.

In NYC, managing education policy is a matter of using a generalised

management logic rather than expertise specific to education. The plethora of

numbers makes it possible to ‘govern by numbers’ (Rose 1991). From 2003 to

School Chancellor Klein’s resignation in 2011, Mayor Bloomberg and Chancellor

Klein, indeed, managed by the calculations, reforming the city’s school system

through Children First, a series of NCLB-inspired accountability measures. Like-

school or peer school comparisons became the cornerstone of a transparent,

technicised and apparently neutral accountability system mandated across NYC. In

2007, the City’s Department of Education (DOE) began issuing public schools an

A–F grade, based on its score in three categories: school environment (15 %),

student performance (25 %) and student progress (60 %). According to the DOE

website: ‘Scores are based on comparing results from one school to a peer group of

up to 40 schools with the most similar student population and to all schools

citywide.’3 A list of peer schools is presented in the report to enable students,

parents and the public to hold the DOE and its schools accountable for student

achievement. Not only are schools evaluated in comparative terms based on student

achievement scores, but these scores also participate in evaluating teachers.

MySchool in Australia

Halfway across the world, in 2007, a similar set of reforms, based on standardizing,

testing, comparing and publishing was being conceptualised and launched through

the newly elected Labor government’s ‘Education Revolution,’ promoting a broad

vision of building ‘the best education system in the world’ (Rudd 2007). This

3 http://schools.nyc.gov/Accountability/tools/report/default.htm

The struggle to technicise in education policy 637

123

revolution was explicitly promoted as being based on evidence as opposed to

prejudice:

For over a decade, debates about knowledge and skills in Australia have been

based on the opposite of evidence—prejudice… [This government] was

elected with a mandate to end that approach, with a new emphasis on

evidence-based reform. (Gillard 2008)

As in the US, in Australia too, the responsibility for education rests with the states

rather than the Federation, but several of the decisions of the new government are

centrist in nature. A national curriculum, mandatory for every state and school, is

being developed and implemented. The National Assessment Program—Literacy

and Numeracy (NAPLAN), which tests every student in Australia at four points

during the compulsory years of schooling, has replaced the previous state-wide

tests. National teacher professional standards have been adopted.

When Gillard visited New York in 2008, she was so impressed by the NYC

reforms, in particular the like-school comparisons, that upon her return she actively

canvassed support for similar policy to be introduced in Australia. She invited Joel

Klein to Australia to promote the idea among various stakeholders to garner

support. Finally, in January 2010, and not without opposition from teachers’ unions

and school principals, the controversial MySchool website, with the NYC-inspired

like school comparisons, was launched.4

MySchool is a single-window access to information about each of the nearly

10,000 Australian schools, including details of student demographics, financial data

and, significantly, NAPLAN results in a comparative format. These comparisons are

against 59 other like schools across the country. These like school comparisons are

used to produce what is seen as compelling evidence on the performance of

students—and therefore schools—so as to appropriately reward or hold to account

schools and teachers through such measures as funding incentives and performance

pay. The comparisons were also presented as a way of empowering parents, who

could now ‘vote with their feet’ if their child’s school did not rank well. Later,

Gillard tempered this aspect of comparison, saying her focus was on helping

teachers and schools:

I’m not interested in saying bad teachers or bad schools or anything like that…[T]he motivation here is to identify those schools that need extra assistance.

So it’s not about blame, it’s about making things better. It’s actually about

addressing the problem (Ministers’ Media Centre 2008c).

The apparently unemotional and apolitical comparisons are presented as clear

evidence of best practices of the more successful schools, which are to be replicated

across the board. This nationally standardised account displaces the previously

produced accounts based on a variety of state level and other local assessments and

analyses, which are now to be viewed as ill-founded, non-standard and confused.

4 http://www.myschool.edu.au

638 R. Gorur, J. P. Koyama

123

The trajectory of like-school comparisons

Technicizing the policy problem

In NYC, the persistent, complex and wide-ranging issues associated with

differential academic achievement have increasingly become framed in technical

and numeric terms. Joel Klein explained in a radio interview (Attard 2008) how the

discussions around school improvement became more precise and technical once he

was appointed:

When I came to be Chancellor in the City of New York we talked only about

process, we talked about how much we spent, we’d talk about how big our

classes are, we’d talk about what new teachers we had and now we have

shifted the discussion to one on results because our kids will need, and indeed

right now do need, very different results if they’re going to compete in a

global economy.

Thus, wide-ranging discussions on process, spending, class size, teacher expertise

and other school issues have become replaced by a singular focus on quantifiable

and reportable results.

Like-school comparisons fit neatly into the reductive order, rendering some

factors irrelevant to the discussion. Comments by an NYC Department of Education

senior administrator exemplify the legitimizing of such frames:

We need to be able to measure why some schools have great test scores and

others don’t….The question for us becomes: If some schools are producing

great numbers, why can’t others just like them do the same?…First, we’ve got

to be able to show the schools and the parents the differences in achievement

scores at these comparable schools and then we’ve got to get them to want to

meet or beat their peer schools’ scores.…We’ve got to gather and display the

numbers, and then get to work. (Interview, August 19, 2008, emphasis added)

This administrator echoed a common district discourse: it’s all about the numbers,

and making those numbers public knowledge is the first step in improving the city’s

schools.

The simplified logic of this argument is persuasive, and in Australia, Gillard

echoes it in her bid to introduce similar reforms there:

In New York, they have a system that Joel Klein leads of comparing schools.

Now they don’t compare a rich school on the Upper East Side with one in

Harlem because obviously that would be not a very intelligent comparison.

But they have a way of diagnosing who’s in school and then they compare

like-schools with like-schools, so comparable student populations. And they

then measure attainment and they say, ‘Well, we’ve got two schools,

comparable kinds of kids in those schools. One’s going a lot better than the

other. Why is that?’ And they’re able to work out what is different in terms of

teaching and school leadership and school culture that is making a difference

in the higher achieving school and spread that best practice. They’re also able

The struggle to technicise in education policy 639

123

to bring additional resources to the aid of schools that are falling behind. Now

we don’t have that kind of information to enable us to do that across Australia

(Ministers’ Media Centre 2008c).

Raising student achievement, then, becomes a straightforward matter of having the

information to identify good and bad practices, and then to spread around best

practice. In Australia, the issue initially highlighted was that although Australia did

well in international tests, there were pockets of disadvantage. This, however,

became translated into a focus on the lack of necessary information to accurately

diagnose the problem, so that the generation and publication of comparable

information about schools became imperative for finding a solution, as Gillard

explained:

Well, the problem from my point of view as Federal Minister and I think it’s

also a problem for the nation is we don’t have national comparable

information about schools… so we obviously got [sic] a problem there with

drawing information right across the nation (Ministers’ Media Centre 2008a).

Having comparable information about schools became a prerequisite for raising

school standards and student performance.

Yet, one of the most contentious aspects of the MySchool website has been the

calculation by which the likeness of schools was determined. Many schools were up

in arms about the inappropriateness of some comparisons, and the very basis for the

calculations, the Index of Community Socio-Economic Advantage (ICSEA), was

called into question. In her response to this challenge, Gillard countered:

We have obviously had public debate about the ICSEA index… I do have a

standing offer to any journalist who has read Barry McGaw’s book on meta-

analysis and would like to sit through and work through the regression

equations with him, anybody who wants to do that, a standing invitation to

come to my office for the number of days necessary to get that done

(Ministers’ Media Centre 2010).

The technicality of regression equations, which would take ordinary citizens days to

understand, serves as a reminder of the specialised expertise of statisticians and the

technicised nature of policy. Numbers are used not just to persuade but to

intimidate, so that debate and discussion are truncated and discouraged (Ewing

2011).

Most recently, in both New York and Australia, poor student performance has

been translated into poor teacher performance, with like-school comparisons shining

an accusing spotlight on individual teachers. To place its second bid at $700 million

in federal education grants (named ‘Race to the Top’), the New York State

Department of Education and its teachers’ union agreed to tie teacher evaluations to

student standardized state test scores. Forty percent of teachers’ annual professional

performance reviews (APPR) could be based on students’ standardised test scores,

making them more heavily weighted than other measures. In Australia, too, the bid

to link teacher appraisal to student performance, and use these links in determining a

formula for performance pay continues to invite much debate, as we soon discuss.

640 R. Gorur, J. P. Koyama

123

Technicisation involves a gradual transfer of authority from humans to non-

humans. The machinery of ARIS, APPR, standardized test scores, and numeric

comparisons conveniently summarize the far more complex and messy realities of

the day-to-day work of teaching and governing in schools. Teachers, principals, and

parents no longer speak with authority with regard to what the problems are and

how they are to be addressed—indeed, even ministers or chancellors do not

pronounce judgements; instead, the authority to describe issues and solutions is

given over to the more-difficult-to-refute machinery of numbers.

Informed publics?

A great advantage of technicisation is that data are simplified and rendered

presentable in the form of widely accessible graphs and tables. When parents are

given access to the large amount of data that is produced, it results in the creation of

‘informed’ communities who are able to participate knowledgeably in policy

debates, and moreover to put pressure on underperforming schools. Yet, given the

simplified nature of the data, these communities might arguably be ill-informed or

minimally informed; nevertheless, they are seen as having a right to information,

and there is an expectation that they will use the information to hold schools

accountable.

Klein explains how this was achieved in NYC: ‘[T]he first part of the system and

in my view the critical part … was to get the information publicly available so

parents know, so that the school knows, so that the media knows, so that we can see

how our schools are doing and what the differences are’ (Attard 2008). There is

great confidence in this ‘knowing’—not only are these data seen as being clear and

good information, but also unambiguous, so that all the stakeholders know the same

truth.

In NYC, to facilitate this wide-spread ‘knowing,’ during the 2008–2009 school

year, the school progress reports, quality reviews and surveys already made public

in NYC were enhanced by ARIS, software which made school data and individual

students’ test scores, attendance, credits and graduation trajectories (among other

quantifiables) electronically available to principals, teachers and parents. During an

introductory ARIS workshop, Klein told journalists that ‘for NYC teachers, the

future is now’ and he predicted that NYC would be a national model of data-driven

instruction.5 At a press conference in Brooklyn, U.S. Secretary of Education, Arne

Duncan expressed his hope that states would use the economic stimulus money to

adopt accountability-oriented reforms such as ARIS (Cramer 2009).

In Australia, teachers’ and principals’ fears that parents might not understand the

data and its significance were dismissed by Gillard (Ministers’ Media Centre 2008b)

who said: ‘I absolutely reject the proposition that somehow I am smart enough to

understand information and parents and community members are somehow too

dumb.’ The day before MySchool went live, in her blog, Gillard urged parents, to

log onto MySchool. Discounting their previous knowledge about their child’s

schooling, Gilliard promised: ‘For the first time, parents will be able to see exactly

5 http://insideschools.org/blog/2008/11/17/aris-live-at-last

The struggle to technicise in education policy 641

123

how their child’s school is doing… The worst thing for a student would be if they

were in an underperforming school and no one knew …’6

In this way, the ability to develop valid knowledge was delegated from parents to

the MySchool website. It was only when MySchool was in place that parents could

know, ‘for the first time’ if their child’s school was underperforming. The

apparently neutral and unemotional technicized accounts were presented as more

trustworthy and precise than that the unreliable and imperfect knowledge of

humans.

When the MySchool website finally went live, after much debate and exposure in

the media, it got so many hits that it temporarily crashed. Gratified by the response,

Gillard (Ministers’ Media Centre 2010) noted that everyone was now involved in

informed conversations: ‘Conversations in workplaces and kitchens. Conversations

between parents and school principals. Conversations between teachers in staff

rooms. Conversations between parents and their children.’ However, while Gillard

urged parents to ‘jump online’ and judge for themselves, she dismissed the

dissenting voices of teachers and school principals as stemming from incompetence

and laziness.

Beyond the kitchen table: spaces of uncertainty

As the products of technicisation became more visible, the challenges to

technicisation became more feasible. The widespread publication of data and the

beginnings of public conversations opened ‘spaces of uncertainty’ (Callon et al.

2001), in which the moral and epistemic authority of technical articulations of the

like-school comparisons and related data-driven reforms could be questioned. The

investments in technicising policy were disputed. Educators, parents, and the public

in NYC and Australia began contesting the ways in which the generation,

maintenance and circulation of comprehensive data collections reduced what

teachers and students were actually doing in the classrooms to highly simplified and

easily-accessed datasets (Koyama and Varenne 2012).

NYC teachers have questioned the intent of these calculations. Several sites of

teacher blogs, including www.Educators4Excellence.org and www.GothamSchools.

org, also challenge the technicisation of NYC’s reforms, including standardised

assessments, value-added formulas and like-school comparisons. Parents and

guardians of students have also disputed the value in ‘the growth and development

of data-based systems of inspection and performance management…’ (Ozga 2009,

p. 149). According to a parent, who is active in two parent organizations, most

parents support the wide dissemination and access to school data; however, ‘they

fear that all of the numbers in all the graphs, reports and databases is intended to

placate and lull them into a false sense of participation’ (interview, October 10,

2009). Parents, she explains, have had an uneasy relationship with the Bloomberg

and Klein administration, and engaging with numbers, however sophisticated, was

not satisfying. ‘Parents feel that they no longer have real relationships with the

6 http://blogs.news.com.au/dailytelegraph/yoursay/index.php/dailytelegraph/comments/julia_gillard_

blogs_live_on_my_school_for_parents/

642 R. Gorur, J. P. Koyama

123

teachers and principals,’ says the parent, finding this is ‘not only not satisfying, it is

ludicrous.’ Her sentiments were echoed by several principals, teachers and parents

whose voices had already complicated the technical certainty of the policy

measures.

In Australia, the calculations underpinning like-school comparisons, the

community socio-economic index (ICSEA), came under severe public pressure.

Newspapers highlighted instances in which clearly unlike schools had been deemed

comparable, and a more refined version of ICSEA was eventually developed.

Further, teachers, parents, and several professional organizations, including the

Australian Primary Principals’ Association, voiced concerns about the use of

NAPLAN test data in like-school comparisons, the statistics available on MySchool

website, and the performance-based pay plans for teachers.

One of the important purposes of introducing like-school comparisons was to

enable linking student performance on NAPLAN to teacher performance, which in

turn would enable the introduction of performance pay for teachers. But this plan

was hotly contested. Blog respondents challenging Gillard’s statement about

holding teachers accountable for students’ NAPLAN performance, queried: ‘Which

teacher Ms Gillard? The one that has had them for 3–4 months, the one they had in

Year 1 or 4 or 6 or 8 or Kinder?,’ problematizing the linking of teacher and student

performance.7 Parents’ concerns about the proposed reforms were also voiced in

more formal responses. For instance, The Australian Capital Territory Council of

Parents & Citizens Associations Inc. produced a 56-page report pushing for a senate

enquiry into the conduct and reporting of NAPLAN, saying that the views of parents

had either been disregarded or misrepresented in the administration of NAPLAN

and MySchool.8 They sought changes to the system of testing and reporting that

would ‘[enhance] the educational experiences of Australia’s children and [provide]

parents with meaningful feedback on their child’s and child’s school’s performance

yet does not lead to any undue harm to individual schools or teachers.’ Through

their actions, both organised and individual, parents actively challenged the

authority and certainty of the very information that was designed to be unambiguous

and to quell uncertainty. They disrupted the simplistic input–output model

envisaged by Gilliard and her administration.

Despite the challenges, the 2011 federal budget announced plans to pay a bonus

to ‘top performing’ teachers starting from 2014, assessed in part on students’

NAPLAN results and feedback from parents. The inclusion of parent feedback also

invited controversy. On the Australian Primary Principals’ Association website,

Emeritus Professor Max Angus (2011) wrote:

Requiring feedback from parents is asking for trouble. Some parents may have

an informed view but for many, the impression that they form will be based on

superficial snippets—even gossip. Incorporating mandatory parent feedback

7 http://blogs.news.com.au/dailytelegraph/yoursay/index.php/dailytelegraph/comments/julia_gillard_

blogs_live_on_my_school_for_parents/8 https://senate.aph.gov.au/submissions/comittees/viewdocument.aspx?id=be48789a-6946-4657-be1d-

cd9326e9a0d8

The struggle to technicise in education policy 643

123

has the potential to undermine the relationship between teacher and parent.

This is a policy cooked up in a leafy green suburb.

The capacity of parents to be informed evaluators of schools and teachers was thus

also contested, as debates went beyond the kitchen table to committee meetings of

formal organizations with expertise of their own.

Other voices, such as representatives of teachers’ unions and parents also joined

the debate through such websites as ‘Save Our Schools.’9 Some of these actors

support the new suite of reforms, whilst others challenge them. The Victorian

Employers’ Chamber of Commerce and Industry (VECCI) released a paper in

support of performance pay for teachers.10 The Business Council of Australia also

produced an influential report titled Teaching Talent: The Best Teachers for

Australia’s Classrooms, strengthening the relationship between teacher and student

performance. In November 2011, the Productivity Commission advised the

government not to go ahead with the performance bonus scheme, citing the failure

of such schemes in improving performance in similar experiments overseas and

advising instead that smaller experiments be funded before going ahead with the

scheme. A decision on teacher performance pay is still pending. The publication of

like-school comparisons was expected to unequivocally link teacher and student

performance, replacing prejudice with the hard evidence of numbers—but

unexpected voices arose to displace the certainty of the calculations and generate

controversy.

Overall, in NYC and Australia, technicisation of policy was promoted at

considerable cost and effort, but these mobilisations were never complete and never

secure. At each location, they started to unravel, and their certainty came to be

undone as the public was invited to engage with the published data. The challenges

by parents, organisations, and the general public enjoyed varying degrees of

success, but the assured and certain science of quantifying schooling that was to

provide clear and incontestable information became less certain over time.

Technicisation, contestation and the policy imagination

The account we presented above described an effort to render messy, contestable,

and imprecise realities of schools and teaching and learning, confounded by many

imponderables and competing evaluations, into highly technical, precise and

irrefutable numeric accounts that, it was hoped, would settle controversy. Using

statistical logic to cluster and clump actors into standardised categories, at each site,

there was an attempt to create an orderly world in which activities could be

described in sequences of actions and reactions, causes and effects.

This move to quantify, order, and render manageable is characteristic of what has

been described as ‘governing by number’ (Lingard 2011). Porter (2003) argues that

numbers are used to give governing agents a form of ‘impersonal authority’ that can

9 http://www.soscanberra.com/10 http://www.vecci.org.au/news/Pages/Business_calls_for_performance_-_based_pay_for_teachers.aspx

644 R. Gorur, J. P. Koyama

123

be seen as displacing not only ignorance but also bias. He distinguishes between the

kind of objectivity generated by physical science and that generated through the

bureaucratic use of statistics:

[This is] the world of accounts and budgets, of maps and social surveys, of

classifications of school children, the sick, and prisoners. In this more practical

domain, quantitative objectivity was more bureaucratic than scientific, a

matter of managing populations with numbers and of achieving some kind of

impersonal validity by following the rules.

However, this project of technicisation did not meet complete success at either

location. Challenges to the technical policy instrument of like-school comparisons

in NYC and Australia have come from a variety of quarters, forcing, in some

instances, some changes to the calculations and decisions. The authority of

statistical accounts is not all-encompassing; for example, it cannot entirely displace

parents’ desire for ‘real relationships’ with teachers and principals.

Although the attempts to regulate the work of schools and teachers through these

mechanisms continue to be challenged, the work of imposing order through

techinicisation of policy is on-going and always open to challenge—it is never

complete. There are constant tussles between order and containment on the one

hand, and complexity and overflow on the other as Callon et al. (2001) describe:

Operations that were thought to have been settled definitively are reopened.

Arguments multiply and the project constantly overflows the smooth

framework outlined by its promoters. In the course of the controversy,

unexpected connections are established between what should have been a

simple technical project and a plurality of stakes that are anything but

technical. Thus we have new actors taking up the problem, imposing

unexpected themes for discussion, and redefining the possible consequences of

the project. (p. 15)

Curiously, the calculations in like-school comparisons and evidence-based

measures, themselves, are beginning to provide grounds to challenge both the

epistemic authority and the practical usefulness of such calculations. Like-school

comparisons have been revealed as devices that absorb variance, translating and

aligning disparate elements and actors into apparently commensurate entities. The

certainty of evidence is not merely a matter of the application of sound

methodologies and measurements, but is achieved through negotiations, contesta-

tions and legislations. Every bid to isolate, order, measure, tabulate and establish

certainty was open to challenge not only from various groups, but by the empirical

situation itself. As Latour (1993) suggests, the modernistic zeal for order does not

displace the presence of the complex and the hybrid. The contextual and empirical

complexities of classrooms and societies continue to defy the attempts to make them

amenable to simple forms of measurement.

Despite these challenges, the persistent and widespread use of such data at

multiple governing levels has begun to systematically configure the shape of policy

problems. The formats of such data—single-page rankings, clear graphs, audio-

visual slides and user- friendly websites—offer seductively simplistic

The struggle to technicise in education policy 645

123

representations that could lead to superficial understandings of policy problems and

solutions. Such renderings serve to limit the policy imagination, leading to

ineffective—and possibly harmful—solutions.

Equally importantly, such accounts close off spaces of contestation; as Barry

(2002, p. 272) notes, ‘When situations become calculable it is taken to indicate the

fact that political contestation has ended.’ Although, as we demonstrate, calcula-

tions may come to be challenged, the challenges themselves tend to get organised

around the issues configured by statistical accounts, thus inadvertently reinforcing

them. Because statistical accounts brush aside the particular, the local and the

contextual, operating on a universalising rationale, they are easily networked and

interconnected. The more interwoven calculations become in the policy world, the

harder they become to challenge or displace. Focusing on issues that are outside

these statistical accounts, and appealing to other types of expertise and rationales

becomes more and more difficult as statistical accounts proliferate and become

nested in a range of calculations and policy actions.

Technicisation of policy affects much more than the effectiveness of policy

reforms; rather, the frameworks and calculations come to serve as an infrastructure

underpinning the way we understand and organise our world. Most importantly,

such infrastructures serve as technologies of rationality and universal common sense

around which apparent consensus is developed, but, as Callon et al. (2001) suggest,

consensus is often a mask for hiding relations of domination and exclusion. It is for

this reason that a deeper understanding and sustained critique of such accounts in

educational research must be pursued. As made clear by Gale and Lingard (2010),

evidence-based trends in education not only narrow what we come to accept as

education, they also, if we allow them, limit what is recognised as educational

research, making it less relevant to policymakers and practitioners.

References

Angus, M. (2011). Rewarding ‘‘great’’ teachers? Retrieved January 10, 2012 from http://www.appa.asn.

au/index.php/principals-voice/performance-pay-160511.

Attard, M. (2008). Joel Klein, New York City Schools Chancellor. Retrieved July 14, 2010 from http://

www.abc.net.au/local/stories/2008/08/28/2348955.htm.

Barry, A. (2002). The anti-political economy. Economy and Society, 31(2), 268–284.

Callon, M., Lascoumes, P., & Barthe, Y. (2001). Acting in an uncertain world:An essay on technical

democracy (G. Burchell, Trans.). Cambridge, MI: The MIT Press.

Callon, M., & Muniesa, F. (2005). Peripheral vision: Economic markets as calculative collective devices.

Organization Studies, 26(8), 1229–1251.

Cramer, P. (2009). Duncan: NYC reform initiatives a model for stimuls spending. Gotham Schools. http://

gothamschools.org/2009/02/19/duncan-nyc-reform-initiatives-a-model-for-stimulus-spending/.

Accessed 1 March 2009.

Ewing, J. (2011). Mathematical intimidation: Driven by the data. Notices of the AMS, 58(5), 667–673.

Fenwick, T. J., & Edwards, R. (2011). Considering materiality in educational policy: messy objects and

multiple reals [Article]. Educational Theory, 61(6), 709–726.

Gale, T., & Lingard, B. (Eds.). (2010). Educational research by association: AARE presidential

addresses and the field of educational research. Rotterdam: Sense.

Gillard, J. (2008). Speech. Paper presented at the ACER Research Conference. Retrieved from https://

ministers.deewr.gov.au/gillard/acer-research-conference.

646 R. Gorur, J. P. Koyama

123

Jasanoff, S. (2005). Designs on nature: Science and democracy in Europe and the United States.

Princeton, NJ: Princeton University Press.

Klenowski, V. (2009). Raising the stakes: The challenges for teacher assessment. http://publications.aare.

edu.au/09pap/kle091607.pdf. Accessed 1 Jan 2010.

Koyama, J., & Varenne, H. (2012). Assembling and dissembling: Policy as productive play. Educational

Researcher, 41(5), 157–162.

Labaree, D. F. (2011). The lure of statistics for educational researchers. Educational Theory, 61(6),

621–632.

Lather, P. (2005). Scientism and scientificity in the rage for accountability: A feminist deconstruction.

Paper presented at the First International Congress of Qualitative Inquiry.

Latour, B. (1993). We have never been modern. New York: Harvester Wheatsheaf.

Latour, B. (2005). Reassembling the social: An introduction to Actor-Network-Theory. Oxford: Oxford

University Press.

Law, J. (2009). Actor network theory and material semiotics. In B. S. Turner (Ed.), The new Blackwell

companion to social theory (pp. 141–158). Oxford: Blackwell/Wiley.

Lingard, B. (2011). Policy as numbers: Ac/counting for educational research. Australian Educational

Researcher, 38(4), 355–382.

Mandinah, E. B., Rivas, L., Light, D., Heinze, C., & Honey, M. (2006). The impact of data-driven

decision making tools on educational practice: A systems analysis of six school districts. Paper

presented at the American Edcuational Research Assocation Annual Meeting.

Ministers’ Media Centre (2008a). Transparency, national literacy and numeracy, National Curriculum.

Retrieved August 15, 2008 from http://ministers.deewr.gov.au/gillard/transparency-national-

literacy-and-numeracy-national-curriculum

Ministers’ Media Centre (2008b). Speech: Leading transformational change in schools. Retrieved

November 20, 2008 from http://ministers.deewr.gov.au/gillard/leading-transformational-change-

schools.

Ministers’ Media Centre (2008c). Radio interview 4BC School transparency agenda, rewarding quality

teaching. Media Release. Retrieved August 15, 2008 from http://www.ministers.deewr.gov.au/

gillard/school-transparency-agenda-rewarding-quality-teaching

Ministers’ Media Centre (2010). National press club address: Delivering the education revolution.

Retrieved February 28, 2010 from http://ministers.deewr.gov.au/gillard/delivering-education-

revolution-0.

Mulgan, G. (2003). Government, knowledge and the business of policy-making. National Institute of

Governance Conference, Facing the Future, Canberra, April 23–24.

Ozga, J. (2009). Governing education through data in England: From regulation to self-evaluation.

Journal of Education Policy, 24(2), 149–162.

Porter, T. (2003). Measurement, objectivity, and trust. Measurement, 1(4), 241–255.

Power, M. (1997). The audit society: Rituals of verification. Oxford: Oxford University Press.

Rose, N. (1991). Governing by numbers: Figuring out democracy. Accounting, Organizations and

Society, 16(7), 673–692.

Rose, N. (1999). Powers of freedom: Reframing political thought. Cambridge, MI: Cambridge University

Press.

Rudd, K. (2007). Campaign launch speech, 14 November. Brisbane.

Venturini, T. (2010). Diving in magma: how to explore controversies with actor-network theory. Public

Understanding of Science, 19(3), 258–273.

Author Biographies

Radhika Gorur is a CRN Postdoctoral Research Fellow at the Victoria Institute, Victoria University. Her

research seeks to understand how policy ideas cohere, stabilise, gain momentum and make their way in

the world. Her focus is on how numbers – particularly international comparative data – are being

produced, validated, contested and used in contemporary education policy. She uses assemblage and other

concepts from science and technology studies and actor-network theory as the main analytical and

methodological approaches in her research.

The struggle to technicise in education policy 647

123

Jill P. Koyama is an anthropologist and assistant professor in Educational Policy Studies and Practice at

University of Arizona. Her research focuses on the intersections of social inequities and educational

policy. Her work is situated across three integrated strands of inquiry: the productive social assemblage of

policy; the controversies of globalizing educational policy; and the politics of language policy and

immigrant and refugee education. Her book, Making Failure Pay: High-Stakes Testing, For-Profit

Tutoring, and Public Schools, was published in 2010 by The University of Chicago Press. Her work also

appears in several journals, including American Journal of Education, Anthropology and Education

Quarterly, British Journal of Sociology of Education, Educational Policy, Educational Researcher,

Journal of Education Policy, and Urban Review.

648 R. Gorur, J. P. Koyama

123