Devolution Scotland Wales

7/28/2019 Devolution Scotland Wales

1/24

This article was downloaded by:[Williams College][Williams College]

On: 30 May 2007Access Details: [subscription number 762292902]Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3J H, UK

Policy StudiesPublication details, including instructions for authors and subscription information:http://www.informaworld.com/smpp/title~content=t713442426

EVALUATING THE PERFORMANCE OF LOCALGOVERNMENTDirk Haubrich; Iain McLean

To cite this Article: Haubrich, Dirk and McLean, Iain , 'EVALUATING THEPERFORMANCE OF LOCAL GOVERNMENT', Policy Studies, 27:4, 271 - 293

To link to this article: DOI: 10.1080/01442870601009939URL: http://dx.doi.org/10.1080/01442870601009939

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

This article maybe used for research, teaching and private study purposes. Any substantial or systematic reproduction,re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expresslyforbidden.

The publisher does not give any warranty express or implied or make any representation that the contents will becomplete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should beindependently verified withprimary sources. The publishershall notbe liable foranyloss, actions, claims, proceedings,demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with orarising out of the use of this material.

Taylor and Francis 2007
http://www.informaworld.com/smpp/title~content=t713442426http://dx.doi.org/10.1080/01442870601009939http://www.informaworld.com/terms-and-conditions-of-access.pdfhttp://www.informaworld.com/terms-and-conditions-of-access.pdfhttp://dx.doi.org/10.1080/01442870601009939http://www.informaworld.com/smpp/title~content=t713442426


2/24

w

W

m

M

EVALUATING THE PERFORMANCE OFLOCAL GOVERNMENTA comparison of the assessment regimes in

England, Scotland and WalesDirk Haubrich and Iain McLean

Compared with most other industrialised nations, the UK government places greatest weight on performance assessments of local authorities as a tool to ensure high levels of public servicestandards and efficiency of public spending in areas such as education, social services, housing,culture, and benefits administration. Annual comprehensive performance assessments (CPA), published by the Audit Commission for England, are now an integral part of the central local government nexus. By contrast, Wales and Scotland have embarked on different routes in the post-devolution era and have developed assessment frameworks that are much less prescriptive,less intrusive, and more reliant on self-assessment. Drawing on 20 semi-structured elite interviewswith auditors, auditees, and other stakeholders in the three nations, this article evaluates thelessons learned from the respective assessment regimes. It also assesses critically the plans toreplace the English CPA system from 2008 onwards with a regime that emulates Wales and Scotland-style self-assessments carried out by auditees themselves.

Introduction

The UK government places ever-greater weight on performance assessment of publicbodies as a tool to ensure high levels of public service standards and efficiency of publicspending. Top-Down Performance Management has become one of the four maincomponents of the governments model for public service reform (with market incentives,capability andcapacity improvement, andusers shaping services bottom-upbeing theotherthree). Assessment of the service performance of local government in England in particularhas reached an unprecedented level of sophistication, complexity, formal structure, andprescription. Using hundreds of performance indicators and a multitude of inspection andaudit judgements, the Audit Commission publishes comprehensive performance assess-ment (CPA) ratings for all local authorities in England (counties, districts and unitaryauthorities) that aimto describehoweffectively central governmental funds have been used.Under the heading of earned autonomy, the assessments result in certain freedoms andflexibilities granted to authorities, provided satisfactory rating levels have been achieved.

The UK government oversees local government in England, yet it does not do so inScotland and Wales, where it is the responsibility of the devolved administrations in eachcountry. It comes as no surprise, then, that both countries have gone down different routes

Policy Studies, Vol. 27, No 4, 2006ISSN 0144-2872 print/1470-1006 online/06/04271-23 2006 Taylor & Francis DOI: 10.1080/01442870601009939


3/24


4/24

w

W

m

M

transferred to district councils, with some of the smaller districts being merged. InScotland and Wales this is exactly what was done. In England, however, there was aprocess of local consultation which led to the single-tier model being implemented insome places but rejected (quite frequently) in others. Where single-tier councils wereimplemented, they were called unitary authorities.

Table 1 illustrates that, although the number of authorities was reduced from 520 to441, the resultant structure of local government became anything but clear. Unitaryauthorities are responsible for the complete range of services, from education and socialwelfare to environmental protection, housing, and leisure. In the two-tier system, in turn,the functions are split between the county councils, which provide education, welfare andenvironmental protection, and non-metropolitan (shire) districts, which provide housing,waste collection and leisure. This structure was kept in place after the Labour party gainedoffice in 1997, and has remained largely unchanged ever since (Stoker, 2005, p. 31).

Under the system of devolution that was formalised in Britain in 1999, twoconstituent countries within the United Kingdom, Scotland and Wales, voted for limitedself-government, nominally subject to modifications introduced by the UK Parliament inWestminster. One of the consequences of this process has been that central governmentresponsibility and financial support for local government in England has fallen to theOffice of the Deputy Prime Minister, in Scotland to the Scottish Executive, and in Wales tothe Welsh Assembly Local Government Group. These institutions, all of which we visitedfor interviews, are responsible for devising the assessment frameworks to audit authoritieslocated in their respective jurisdiction. The comparison made in this article, then, isbetween the English local authorities that became subjected to the CPA regime in 2002(marked as groups a, b, c, and f in Table 1), on the one hand, and their counterparts inWales and Scotland (d and e, respectively, in Table 1) on the other. By contrast, group g the English non-metropolitan district councils was subjected to a different CPA exercisethat was not implemented until 2005. These authorities are therefore not included in thisanalysis (for details, see Audit Commission, 2005).

England

England is divided into 388 local authorities, the sizes of which vary considerably:the most populated unitary authority area is Birmingham (a metropolitan district) with

TABLE 1The structure of elected local government in Britain post-1997

Single-tier authorities Two-tier authorities (with split functions)

(a) 46 English unitary councils (f) 34 English county councils(b) 36 English Metropolitan districts (g) 238 English non-metropolitan districts(c) 32 London boroughs(d) 22 Welsh councils(e) 32 Scottish councils (excludes Isle of Scilly; Corp. of London)

EVALUATING THE PERFORMANCE OF LOCAL GOVERNMENT273


5/24

w

W

m

M

980,000 inhabitants, and the least populated is Rutland with 35,000. However, these areoutliers, and most unitary authorities have a population in the range 150,000 300,000. In2000, the UK Parliament passed the Local Government Act 2000 to force councils to moveto an executive-based system, either with the council leader and a cabinet acting as an

executive authority, or with a directly elected mayor. The Council leader cannot do thework of the council himself or herself, and so is responsible for appointment and oversightof officers, who are delegated to perform most tasks. In order to lead these organisations,councils also need to appoint a Chief Executive Officer, with overall responsibility forcouncil employees, and who operates in conjunction with department heads. Employing2.1 million people, local authorities are one of the largest employers in England andundertake an estimated 700 different functions.

These functions are assessed with a performance management system that wasconceived in 2001 by the ODPM, which was in charge of local government at the time, andthe Audit Commission. The Audit Commission had been founded in 1983 to perform thethen unprecedented task of checking not only that public money is spent for authorisedpurposes, but that the investment is done effectively and efficiently. With its 2500employees, the Commission today is the regulatory arm of central government chargedwith responsibilities that range well beyond traditional audits and encompass compre-hensive reviews of the performance of local authorities, a duty confirmed most recently inthe Local Government Act 1999 (HMSO, 1999). The Commissions formal governing boardis made up of several Commissioners and a Chairman, who are appointed by the

ASSESSMENTFRAMEWORK

ABILITY TO IMPROVE

Self-Assessment andCorporate Assessment

CURRENT PERFORMANCE ON SERVICES

Inspection Judgements, auditor judgements,performance indicators, and governmental

Assessments of councils plans

OVERALLASSESSMENT

- Excellent- Good- Fair- Weak- Poor

PUBLICREPORTING

IMPROVEMENTPLANNING

ASSESSMENTFRAMEWORK

ABILITY TO IMPROVE

Self-Assessment andCorporate Assessment

CURRENT PERFORMANCE ON SERVICES

Inspection Judgements, auditor judgements,performance indicators, and governmental

Assessments of councils plans

OVERALLASSESSMENT

- Excellent- Good- Fair- Weak- Poor

PUBLICREPORTING

IMPROVEMENTPLANNING

B e n e

f i t s

S o c

i a l C a r e

E n v

i r o n m e n

t

L i b r a r i e s

& L e

i s u r e

U s e o

f r e s o u r c e s

E d u c a

t i o n

H o u s

i n g

Audit Commission 2002

FIGURE 1The CPA performance management framework

274 DIRK HAUBRICH & IAIN MCLEAN


6/24

w

W

m

M

department responsible for local government (ODPM, in 2006 changed to the Departmentfor Communities and Local Government, DCLG), following consultation with keystakeholders. In a 2001 government White Paper, an assessment framework to that effectwas first formally proposed to the public and consultations commenced with the aim toconceive a final framework (Audit Commission, 2001).

The stated objective of what has since then been dubbed the CPA was, andcontinues to be, the targeting of support to those councils that need it most as well as togrant freedom and flexibilities to the better performing authorities. These incentivesentailed: (a) the elimination of ring-fencing from most central governmental grants to thelocal authority, which allows the latter to spend money on those areas it (rather thancentral government) deems most appropriate; (b) a three-year exemption fromsubsequent audit inspections; (c) an exemption from having a cap imposed on theauthoritys planned expenditure and the level of council tax it is allowed to raise from itstaxpayers; and (d) the freedom not to have to submit detailed service plans to centralgovernment for approval.

The resultant CPA framework was aimed at measuring the effectiveness of councilsin terms of the way they provide services to local people and work in partnership withother local authorities, private corporations, and the voluntary sector, among others. In sodoing, it focused on the leadership, the systems, and processes, as well as the performanceof those services. As Figure 1 depicts, the framework comprises two distinct elements:current performance and ability to improve in the future with numerous transformationsand categorisations applied in the process (for a more detailed exploration, see Haubrichand McLean, 2006, pp. 93 8).

Current Performance

Much of the evidence on service performance gathered in this exercise had beenpublicly available already and was brought together in one place for the first time in CPA.The service areas (or service blocks) assessed are the six core services of education, socialcare for children, social care for adults, housing, environment, and libraries and leisurefacilities. Where available, performance is assessed through already existing judgementsfrom inspectorates and auditors, such as those by Ofsted and the DfES for the educationservice block.

Numerous categorisations and conversions are applied to summarise more than1,000 performance indicators and auditor judgements, with the final result that localities

obtain a score between one and four for each of the service blocks (with one being thelowest, four the highest). The scores are then weighted so that the scores for educationand social services count four times, housing and environmental services count two times,with the remaining blocks counting only once. These are then added up so as to producea performance score of between 15 and 60 points or 12 and 48 points for countycouncils as they do not provide (and are therefore not assessed on) housing or benefitsservices. The performance scores are then categorised so as to produce a performancerating of between one and four for each authority (see Table 2).



7/24


8/24

w

W

m

M

implementation of new performance management frameworks within authorities and thedevelopment of new strategic plans. It has also led to a refocus of attention onorganisational structure, systems, and processes (although apparently at the expense of afocus on outcomes and user satisfaction). Among our interview partners, theseassessments were shared by both auditors and auditees.

CPA has also introduced a new focus for the work done by authorities members of staff. The impact has not necessarily been greatest among top managers, but anunexpected benefit of CPA has been that more junior and frontline members of staff havebeen motivated by CPA, as they identify with the CPA ratings their employer achieves.

Even so, our interview partners stated various areas where the assessment regimerequires improvements. For one, the validity of some indicators used in CPA wasquestioned. For example, the indicator Number of visits to the library in the leisureservices block was said to say little about the authoritys performance if the librarys statedmission is to provide as much information as possible via the Internet. Similarly, theindicator Number of hostels for the homeless in the housing block is of only limited valueif it is the authoritys stated mission to have no hostels at all (but instead provide thehomeless with proper temporary accommodation).

The issue of perverse incentives with regard to indicators was also raised. Forexample, in the education service block, headmasters are assessed inter alia against theattendance rate their school achieves. This produces the perverse incentive for them toclose down the school altogether in times of very bad weather, given the low attendancerates to be expected otherwise. Indicators, so the interviewers stated, should not beabolished altogether, but used in different ways. Whereas indicators themselves are seenas beneficial, the setting of targets for each of them is not in line with a dictum coinedby Charles Goodhart, the former chief economist of the Bank of England, that when ameasure becomes a target, it ceases to be a good measure.

A phenomenon related to the validity of indicators is a type of regulatory failurefrequently referred to in the public policy literature. Interviewees in local authoritiesadmitted to what could be called procedural compliance; that is, going through themotions of producing the documents and data necessary to satisfy the proceduralrequirements for attaining a higher score on a specific indicator, without actuallyimproving the quality of the underlying service. Power (1997) referred to this phenomenonas a decoupling of formal regulatory processes and substantive organisational processes.Under CPA, authorities are focused on improving the CPA rating on each single indicator,

TABLE 3Final CPA ratings

Councils ability toimprove

Councils performance score on core services1 2 3 4

1 Poor Poor Weak N/A2 Poor Weak Fair Good3 Weak Fair Good Excellent4 N/A Good Excellent Excellent



9/24

w

W

m

M

which provides them with an incentive to do as little as possible to move beyond thethreshold of the next rating category on that indicator.

During the interviews it became obvious that this approach is not necessarilyavailable to all authorities: those that are clearly failing will not be able to get away withsuperficial changes only, because the discrepancies in service delivery tend to be so

obvious (and the outside scrutiny so intense) that the regulatee can do nothing but ensurereal service improvements. Yet, it appears to be a frequently used option for authoritiesthat are more or less settled (i.e. want to move up from their present fair or goodcategory ratings).

The interviewees in local authorities also criticised that the initial aim of earnedautonomy, which central government had heralded at the outset as an incentive forauthorities to perform well, has slipped off the agenda. For when authorities succeed inimproving their performance ratings, no greater autonomy or lighter-touch inspectionshas been granted in return. Some interviewees attributed this to the fact that theassessment results are not only influenced by an authoritys real service performance, butalso by the views that central government officials and auditors have formed over manyyears prior to (and irrespective of) any of the recent CPA exercises. These preconceptionsare persistent and do not change merely on account of one years improved CPA rating.For that to happen in the long run, so one interviewee stated, an authoritys Chief Executive and Council Leader need to make sure they exploit every networkingopportunity available that would allow them to portray their authority before officialsas one with which one can do business.

At times, CPA ratings and scores were regarded as not reflecting accurately anauthoritys performance, because it was felt that many good ideas could be learned fromthe badly performing authorities. One interview partner stated that a neighbouringauthority enjoyed beacon status for its performance on race relations, but wascategorised as weak overall. Another authority still was leading the way on (and wasattracting visiting parties from other authorities for) its work on business planningprocesses, but was rated fair overall. CPA ratings do therefore appear not to capture howprogressive an authority is or to what extent it is a learning organisation, characteristicsdeemed crucial for many policy-makers.

Other concerns raised during the interviews referred to the fact that control oversome service blocks, such as education, are taken increasingly out of authorities hands butattain increasing weight in the CPA exercise. The assessment exercise, so the criticismgoes, should also be based more on self-regulation and be less bureaucratic. Thesuggestion was that the Audit Commission and other inspectorates should check theextent to which the authority regulates itself effectively, rather than regulating it directlyand that it should accept the fact that party politics may have a bearing on the prioritiesassigned to the various services blocks.

Scotland

Scotland is subject to the administration of both the UK Government in Westminsterand the Scottish Executive in Edinburgh. The UK Government has responsibility for issues



10/24

w

W

m

M

such as constitutional matters, foreign policy, and defence, whereas the remit of theScottish Executive includes matters such as health, education, law and local government.As a result, the programmes of legislation enacted by the Scottish Parliament on this latterset of policy domains have seen a divergence compared with the rest of the UnitedKingdom, with the costs of university education and care services for the elderly free at

point of use being the most frequently cited examples.The 1994 Local Government (Scotland) Act led to the abolition of the previous

structure of nine regions and 53 districts, although the three island councils remained.Since then, Scotland has been divided into 32 units that are unitary administrations withresponsibility for all areas of local government (see Figure 1). Akin to England, each localauthority has a Chief Executive and is governed by a council consisting of electedcouncillors, who are elected every four years with the City of Glasgow being the largestwith more than 600,000 inhabitants, and Orkney being the smallest with less than 20,000.Scottish councils cooperate through and are represented collectively by the COSLA. Today,local government in Scotland employs some 270,000 full-time-equivalent staff.

Scotlands framework for assessing local authorities performance called the BestValue Audit is distinct from England in several respects. To start with the institutionalset-up, assessments are, unlike England, the responsibility of three different audit bodies:Audit Scotland, the Auditor General, and the Accounts Commission. The duty of, first, theAccounts Commission is to check that public money used by local authorities is spentproperly, efficiently, and effectively. The Accounts Commission is an executive Non-Departmental Public Body, independent of the bodies audited and the government of theday. The Commission was established in September 1974 and, since the introduction of the Public Accountability (Scotland) Act 2000 and the transfer of its staff to Audit Scotland(see below), is now just a Board that is serviced by Audit Scotland. The independence of the Commission is emphasised by the fact that it does not receive governmental grants.Instead, its activities are funded by contributions made by audited bodies in terms of regulations made by the Secretary of State for Scotland. The Commission is not a Crownbody and its employees are not civil servants. It works within a statutory framework, forwhich the Secretary of State is responsible. Members of the Commission serve in theirpersonal capacity, and not as nominees or representatives of any interest group. Bystatute, its six to 12 lay members are appointed by Scottish Executive Ministers.Collectively, the Commission has powers to report and make recommendations to theorganisations it scrutinises, to hold hearings, to report and make recommendations toScottish Executive Ministers, and to take action against councillors and council officialsshould their negligence or misconduct lead to money being lost or breaks the law.

With such tribunal powers, which in extremis could even lead to the abolishment of a council, the Accounts Commission in Scotland has wider duties than the AuditCommission in England. Yet, various parties during our interviews in Scotland expressedthe view that these intervention powers are not clearly defined and thus constitute nodecisive disincentive for auditees to perform badly and receive bad audit reports as aresult. One interviewee recalled from the parliamentary debates that preceded andprepared the Local Government in Scotland Act 2003 that Parliament was clearly notinterested in granting the Executive or Ministers too far-reaching intervention powers, a



11/24

w

W

m

M

claim supported by the observation that, unlike England, most Members of the ScottishParliament have prior experience in local government. As a result, ministers have nocontrol over the audits, and heavy-handed (albeit vaguely specified) intervention is anoption only once all other steps are exhausted.

The Auditor General (short for Comptroller and Auditor General), in turn, is

appointed by the Crown on the joint recommendation of the Prime Minister and theChairman of the Committee of Public Accounts who is, by convention, an Opposition MP.He is independent; accountable to the Scottish Parliament; an Officer of the House of Commons; can be removed from office only by the Queen on an address from bothHouses of Parliament; and his salary is paid directly from the Consolidated Fund withoutrequiring the annual approval of the Executive or Parliament. He reports directly toParliament and his reports are considered by the Committee of Public Accounts, whichuses them as a basis for examining the performance of senior civil servants. The maincomponents and duties of audit and budgetary control are similar between the AccountsCommission and the Auditor General, although the former has responsibility for the 32local authorities, while the latter has the additional remit for local NHS trusts and HealthBoards, the departments of the Scottish Executive, and the colleges of further education(Audit Scotland, 2004). The essential difference is the lines of accountability and the extentto which the bodies are associated with, and report to, either the Parliament (in the case of the Auditor General) or the Executive (the Accounts Commission) (Audit Scotland, 2006a).

Finally, the role of Audit Scotland is to provide the two aforementioned bodies withthe services they need to carry out their respective duties. It is an independent entityresponsible for holding to account nearly 200 public bodies in Scotland. Two-thirds of theaudits are carried out by Audit Scotland staff, with the remainder assigned to privateauditors. 1 Audit Scotland has emerged from a predecessor with UK-wide jurisdiction, theNational Audit Office.

The 10 local authority functions that are assessed by these three bodies deviateslightly from the seven service blocks audited in England. They cover social work, benefitsadministration, cultural and community services, education and childrens services,corporate management, development services, housing, protective services, roads andlighting, and waste management. Together, they consume an annual local governmentbudget worth 14.5 billion, which is financed primarily by central government grants of 10.1 billion (in the form of Revenue Support Grants of 5.3 billion, redistributed non-domestic rate income of 2 billion, and dedicated grants of 2.8 billion), council taxrevenues worth 2 billion, housing rents of 0.9 billion, and income from sales, fees andcharges of 1.5 billion (Audit Scotland, 2006b, p. 11). Unlike England, central grants inScotland are not ring-fenced, nor does the grant amount vary depending on the auditresults (as is the case with the policy agreements in Wales, see below).

The greatest departure from England is found in the assessment regime that thethree audit bodies have devised since the Local Government in Scotland Act 2003 cameinto effect. The framework is much less prescriptive than the English approach and isbased predominantly on flexible self-assessments carried out by the authoritiesthemselves. Called Best Value Audit, the assessment approach is based on criteria setout by the Scottish Executive (Scottish Executive, 2005). The choice of indicators which



12/24


13/24

w

W

m

M

Wales, where it seems to have led to more successful policy changes. In Scotland, up to adozen inspectorates and auditors knock on authorities doors with the aim to take stock of councils public service delivery. One of the authorities interviewed reported that itshousing department had been audited four times within the preceding nine months byCommunity Scotland, the Social Care Inspectorate, the Benefit Fraud Inspectorate, and theBest Value Inspection Team leading to considerable exhaustion within that department.More often than not, these bodies require identical sets of data, yet hardly ever in thesame format or on the same medium.

In an attempt to save time and resources, the authority in question had developed ageneric template for an assessment model that records performance data in one place,once a year, and in a way suitable for several inspectorates/audits. This model has sincethen been offered to all visiting auditors and inspectorates, and has generated greatinterest among neighbouring authorities. Although it may take this particular authority along time before the various inspectorates, auditors, councils, and members of parliamentare convinced of the merits of the template, the bottom-up drive to standardise a centrallydriven audit and inspection process stands in marked contrast to the prescriptive top-down approach imposed on English authorities in the past four years.

Some further drawbacks of the Scottish approach emerged during our interviews,some of which are unique to Scotland, others akin to problems found in England andWales. To start with, similar to England and Wales, some of the statutory performanceindicators are regarded as superfluous or invalid, because they fail to measure the valueprovided to the public. An example given by various interview partners is the indicatorthat measures the number of swimmers per square metre of pool water in a localauthoritys leisure service function. In order to produce a positive outcome on this

indicator, the local authority would only need to close all but one of its pools. This wouldincrease the ratio but would hardly be regarded as an indication of good public service.Alternatively, and less drastically, a department manager may want to decide to improvethe indicator, by filling up available pool slots with swimming clubs (the members of which will be happy to swim up and down the pool lanes in great numbers) rather thanproviding swimming lessons to the public (which would result in a relatively small numberof participants occupying a large area of the pool for their swimming exercises). Clearly,the public would be better served with the latter policy, but the indicator would requirefrom an authority to pursue the former. Indicators such as these have survived from thepre-devolution era, were (and still are) defined by the Scottish Executive and therespective governing bodies (in the case of swimming, the pool managers), but needurgent revising.

Secondly, the great flexibility granted to local authorities means that they are nottreated equally and consistently, a problem also reported by auditees in Wales. Coupledwith the uncertainties mentioned above regarding the exact intervention powers availableto the Accounts Commission, this has led to a situation where the audit reports producedby the Auditor General are regarded as too bland and too uniform and not hard-hittingenough. One interviewee highlighted the case of Inverclyde, which was known to be abadly performing authority and which has been the only council that saw its Chief



14/24

w

W

m

M

Executive replaced on the grounds of intervention powers applied by the AccountsCommission.

While it was commonly acknowledged that this was the right step to pursue, theaudit report for Inverclyde that had been produced just a year before did not differsignificantly from other authorities audit reports: the appraisal and criticism of Inverclydes

strengths and weaknesses read, by and large, similar to the audit reports produced forother authorities that year. Only the subsequent commentary produced by the AccountsCommission was specific and critical of individuals and circumstances and led to theeventual displacement of the Chief Executive. Inverclyde proved the point that, if required,the intervention powers are anything but mere potential and can be significant and real.Yet, the case also proved that the audit reports themselves are not explicit enough inhighlighting problems and depend on contextualisation and inside knowledge (in thiscase, on the part of the Accounts Commission) that adds meaning to the text. That beingso, the valid question to ask is what is the point in producing an Audit Report in the firstplace?

Thirdly, despite the small size of the country and local government community,some stakeholders regretted the few networking opportunities that remained after COSLAhad withdrawn from this area a few years before. Whereas Wales prides itself in formingbottom-up benchmarking clubs and groups of similarly structured authorities in order toshare best practices and compare performances, these processes appear to be much lessdeveloped in Scotland. The Scottish Improvement Service was set up in 2004 to fill thisvoid, by helping to improve the efficiency, quality, and accountability of public servicesthrough learning and sharing knowledge. The organisation is a partnership between theScottish Executive, the COSLA and the Society of Local Authority Chief Executives, and isfinanced by the Scottish Executive.

Finally, but only indirectly related to the assessment approach as such, authorityboundaries are deemed contentious and are often devised for political reasons. A need hasbeen expressed during the interviews to merge some authorities as a means to improvehuman resources and systems capacities in the smaller localities.

Wales

The Welsh local government system was closely integrated with the English systemuntil devolution in 1999 and thus contrasts with the Scottish which never has been, itsdistinctness being asserted in the Treaty and Acts of Union of 1707. Since 1999, Wales hasbeen subject to the administration of both the UK Government in Westminster and theNational Assembly for Wales in Cardiff. The UK Government retains responsibility for allprimary legislation, but the National Assembly has powers to make secondary legislationin a range of policy areas such as health, education, industry, agriculture, localgovernment, environment, and culture. The Assembly not only legislates but also funds,develops and implements policy for much of the public sector in Wales. Devolutiontherefore is creating an increasingly diverse and diverging public policy agenda across theUnited Kingdom.



15/24

w

W

m

M

Following the 1994 Local Government (Wales) Act, the eight counties and 37districts of Wales were replaced by 22 unitary authorities with responsibilities for allaspects of local government. They are responsible for 4 billion of annual publicexpenditure, one-third of the total Welsh budget. Eighty per cent of this budget isfinanced through central governmental grants, with the remainder coming from council

tax revenues. Local government is also one of the largest employers in Wales, with some164,000 employees.

Akin to Scotland, then, Wales contains much fewer local authorities than England, sothe relationship between auditor, local government department, and authority can bemore intimate than in England. League tables are avoided because the local authoritycommunity is small enough for all senior managers to know one another. Putting localauthorities in a position of direct competition with one another is therefore not desired.Instead, the audit reports for each authority contain a section (somewhat confusinglydenominated Risk Assessment) that provide a contextualised account of the service areasin which the respective authority has to improve. In these sections, councils have tocategorise themselves for each service area as low, medium or high risk, an assessmentthat appears to be a combination of performance quality and importance weighting.Although the Welsh Local Government Association has developed a Risk AssessmentTemplate of how to go about this task, its uniform application is not guaranteed and risk assessments are said to be done in 22 different ways across the nation.

Local authorities in Wales are assessed by the WAO, an organisation that was createdon 1 April 2005 following the passing of the Public Audit (Wales) Act 2004. The Actbrought together the former (pre-devolution) offices of the Audit Commission in Wales,which was responsible for auditing and inspecting local Welsh public services, and theNational Audit Office in Wales, whose role was to audit the National Assembly and itssponsored and related public bodies, into one body headed by the Auditor General forWales. Wales now has a single audit and inspection body, responsible annually for theaudit of over 19 billion of public expenditure at all levels of administration, including localgovernment.

The assessment regime dubbed Wales Programme for Improvement, WPI (WAO,2006) that is overseen by the WAO is, as in Scotland, much less prescriptive and muchless elaborate than its English counterpart. It is not only based on flexible self-assessmentscarried out by authorities, but authorities have also a say with regard to the type of inspection they would like to see for specific areas (brief reviews, in-depth audits, etc.). Noleague tables or composite rankings are published. Instead, comparisons on the basis of selected performance indicators can be made, through the website maintained by theLocal Government Data Unit, the custodian for all matters related to performance data.

However, the detailed results (including the risk assessment categories mentionedabove) in an authoritys audit report remain confidential; no one other than the authorityin question and the auditors will see them, and nothing is communicated through themedia either. The absence of rankings may be advantageous in the sense that is does notput individuals who know one another into a position of direct competition, but it iscounterproductive in another: the fact that such rankings are available for Englishauthorities across the border induces some media in Wales to produce their own league



16/24


17/24

w

W

m

M

readily acknowledged by both these bodies. Also, some complained that in thecommunication triangle between WAO, Welsh Assembly and authorities there is nofeedback loop built into the third side of the triangle, that between the AssemblyGovernment, as the body that devises the WPI, and the authorities, as the entities havingto live with it.

Some limited extra funds are made available to local authorities if they meet 16specified indicators measuring national strategic objectives. These are called policyagreements and can amount to additional funds of between 0.5 and 1.5 per cent of anauthoritys budget. The 16 indicators are the only ones that are audited, predominantlywith regard to the methods through which data are used. The majority of indicators, someof which can be chosen by authorities themselves, are not checked for accuracy orstandardised interpretation. Given that central grants in Wales are generally not ring-fenced, some interviewees criticised that these additional funds are simply added to theoverall central grant that the authority receives, rather than earmarking them exclusivelyfor the service areas that are assessed against those 16 indicators.

Unlike England, staff are not incentivised by good performance categories or lesserring-fencing attached to central grants. The main motivational drivers to do well appear tobe the work ethos of members of staff, political pressure by members of the council, andthe threat of more audits if a bad assessment report is produced. Conversely, however,good-performing authorities criticise that they are not rewarded by lighter-touchinspections and audits as they were promised. As the most important reasons forimprovements achieved in an authoritys performance, the interviewees listed better work processes and work plans, better Information Technology systems, as well as staff motivation and leadership.

The coordination between the various auditors and inspectorates appears to beworking particularly well: Relationship Managers have been institutionalised, who reportto the WAO and have the task to coordinate auditors and inspectors. Unlike England orScotland, these positions had some lasting effect on the assessment process: authoritiesobtain a regulatory plan that charts for each of them all of the visits scheduled for thefollowing year. Not all of the inspectorates have yet signed up to this scheme. The issuethat remains is the flurry of presently 34 local plans that councils have to submit and thatare in need of rationalisation.

With regard to intervention powers, the WAO (or its appointed auditors) can make arecommendation to the Welsh Assembly Government for (a) and intervention, or (b) aformal inspection that results in a peer advisory council to be set up to oversee furtherimprovements in the authority. Unlike Scotland, no senior officers in Welsh localauthorities have had to be replaced as of yet, although in 2005 three of the 22 authoritieswere subjected to one of the two types of intervention. However, given the uncleardefinition of these powers and the lack of specification as to the body on which thesepowers should fall, they appear to be as ineffective a deterrent as those in Scotland.During our interviews, statements to that effect were made by auditors and auditees alike.

Similar unanimity in feedback emerged on the question of comparability (or lack thereof) of the assessment reports produced for each authority. Most individuals we spoketo yearned for elements of the more prescriptive English system to be implemented, as it



18/24


19/24

w

W

m

M

The focus in Wales is on the identification of external and internal risks ratherthan composite categorisations; and (internal) organisational performance such asmanagement capacity, leadership, processes, systems, culture, etc. is regarded as onlyone of many risk factors, with (external) strategic, environmental, national, and Europeanrisks enjoying equal importance. That way, the Welsh regime allows for regional priorities

and circumstances to be reflected in the assessment, which the uniform CPA system inEngland does not.

From the differences outlined above, then, the reliance on authorities self-assessment is the most striking feature in the Welsh and Scottish regimes. Interestingly,it is also the one that has spawned the greatest interest among stakeholders in England.For one, the Audit Commission for England whose task it is to produce a successorregime for CPA for the assessment rounds from 2008 onwards is actively engaged inidentifying the features that are worth adapting and incorporating into the Englishscheme. Also, the authors visit to Wales to conduct interviews with Welsh auditors andauditees happened to coincide with members of the Welsh Assembly Governmentfollowing an invitation to the ODPM in London to present their approach. Two monthslater, members of the Welsh Assembly repeated the exercise and presented at aconference on local government in England, which was sponsored by the English LocalGovernment Association. Both host organisations were very receptive to the ideaspresented and sought to use them as a basis to launch a new alternative approach toperformance management that is owned and driven forward by local government staff,not by external agencies (Improvement and Development Agency, 2006).

Yet, despite the analytically interesting cross-country fertilisation made possiblethrough devolution, the question to be asked is how desirable such a convergence of theassessment approaches actually is? How beneficial would it be to introduce additionalelements of self-assessment into the audit process in England and, to use a terminologyintroduced by Hood et al. (1998), move the regulatory system from an oversight to amutuality approach? The lack of bite, the limited transparency, and inadequateopportunities for comparative analyses in Wales and Scotland were mentioned above asthe downsides of self-assessment. The greater buy-in from local authorities, the flexibilityto tailor the assessment to local circumstances, the better suitability for small geographicalareas, and the greater cost-effectiveness were stated as its advantages. But furtherdrawbacks come to light once the analytical focus moves from the devolved nations toEngland, where elements of self-assessment are already part of the CPA exercise and,hence, available for analytical appraisal.

The reader will recall from the discussion in the first section that the final CPA scoreof English authorities during the CPA rounds 2002 2004 were the result of a combinationof two assessment elements: the performance score/rating based on inspections andaudits of services, and the ability-to-improve score based on a corporate and self-assessment. While the first element was the focus of our comparative analysis in thesecond section and (to a lesser extent) the third section, it is the second element thatcomes to the forefront now.

The two graphs in Figure 2 depict a simple but very useful linear regression, byplotting authorities ability to improve scores against their actual performance at a later



20/24

w

W

m

M

FIGURE 2Ability to improve scores versus CPA performance scores



21/24

w

W

m

M

stage. To account for the time required for policy measures to have an impact onperformance, a time lag of one year ( t ' /1) and two years ( t ' /2) is built into the model.Hence, the charts plot the self-assessed 2002 ability-to-improve scores against theabsolute changes observed in CPA scores between 2002 and 2003 and between 2002 and2004, respectively.

The visual distribution patterns reveal that there is no relationship between self-assessment and improvement in performance scores, an observation borne out by the factthe regression lines actually tilt slightly downwards, indicating that the lower anauthoritys ability to improve score, the better its improvement in subsequent years.The coefficient of determination R2 , which expresses the extent to which the regressionfunction fits the set of data observations, is 0.0008 for the one-year model and 0.002 forthe two-year model.

Our analysis may be criticised on three grounds. First, it may be disapproved of forits disregard of ceiling issues in the model: authorities that attained the top performancecategory of 4 in the 2002 CPA round may be said to have been unable to improve theirratings any further in subsequent years, which may be skewing the resultant correlationcoefficients.

Yet, the model was careful to avoid this problem, by not using the four performancecategories (one to four) as the dependent variable, but the performance scores . To recall,Table 2 showed that authorities performance scores can range between a minimum of 15and a maximum of 60 points (or 12 and 48 points for county councils, which do notprovide, and are therefore not assessed for, services in housing and benefits), which arethen converted into category scores of between one and four. In order to establish anequal playing field between the two types of authorities, we then converted the

performance scores into percentage scores.The top performance category of 4 was awarded to an authority if it achieved

between 45 and 60 points (36 and 48 points for county councils). From the 148 authoritiesassessed in the 2002 CPA exercise, 21 achieved the top rating. Of those, 14 achievedfurther score improvements in 2003 or 2004, despite their achieving the top categoryrating in 2002 already, while a further seven either stagnated or dropped a category. Givensuch improvements, ceiling issues can therefore have been only a limited constraint.Rather, it appears that the authorities that did improve were not identical to those thatprevious CPA rounds evaluated as being able to improve.

In order to eradicate any remaining doubts, we re-ran the regression model, thistime without any of the 21 authorities that achieved the top 4 rating in 2002. Aspredicted, the resultant scatter plot (not shown) made hardly any difference to theregression line, although, barely visible to the eye, it tilted very slightly upwards. Thecorrelation coefficients R2 continued to produce a low value of 0.003 for both scatter plots.

The second objection that may be raised against our analysis is the claim that thelack of improvement is due to the moving of the goal posts by the Audit Commission inthe years following the initial 2002 round. This is the argument we heard from some of ourinterview partners who commented that local authorities may have found it harder toobtain good ratings in 2003 and 2004, because the Commission hardened the



22/24


23/24

w

W

m

M

between authorities performance, a feature that remains unattainable with the flexibleWelsh approach. By contrast, we found no stakeholders in Scotland who voiced a similardesire, an observation attributable to the fact that the Welsh local government system wasclosely integrated with the English one until devolution in 1999, whereas the Scottish localgovernment system never has been its distinctness being asserted in the Treaty and

Acts of Union of 1707.The cross-fertilisation between Wales and England proceeds in the opposite

direction as well. In an effort to gauge the advantages (with regard to assessment costsand buy-in from local authorities) of an assessment that is based on self-assessment, theODPM, the Audit Commission, as well as local authorities appear to be leaning towards thesubstitution of CPA from 2008 onwards with a regime that relies much more heavily onself-assessment. The research findings also showed that with the current methodology,the self-assessed ratings and scores lose their validity as indicators of the underlyingauthority attributes of performance and/or ability to improve. Policy-makers need to beaware of such assets when they are trading them in for the scheme that will replace thecurrent CPA.

NOTES1. For a detailed list of appointed auditors, see http://www.audit-scotland.gov.uk/audit/

appointments.htm2. See Audit Scotland (2006b), although the number of indicators can increase to more than

100 once non-statutory indicators (i.e. indicators determined by authorities) are included.

REFERENCES

AUDIT COMMISSION (2001)Strong Local Leadership Quality Public Services, London: AuditCommission.

AUDIT COMMISSION (2004)Holding to Account and Helping to Improve A Strategic Statement for Public Audit in Scotland 2004 06 , Edinburgh: Audit Scotland.

AUDIT COMMISSION (2005)The Framework for Comprehensive Performance Assessment for District Councils A Consultation Document , London: Audit Commission.

AUDIT SCOTLAND (2006a)Evidence Given to Finance Committee on Governance and Account-ability , Edinburgh: Audit Scotland.

AUDIT SCOTLAND (2006b)Overview of the Local Authority Audits 2005 , Edinburgh: AuditScotland.

BOYNE, G.A., MIDWINTER, A.F. & WALKER, R.M. (2003)Devolution and Regulation: Political Control of Public Agencies in Scotland and Wales , Full Report of Award No. L327253035,Economics and Social Research Council, Swindon.

ECONOMIST (2006) A funny thing happened on the way to the council, 25 February,Economist , p. 33.

HAKES, C. & REED, D. (1997)Organisational Self Assessment For Public Sector Excellence, Bristol:Bristol Quality Centre.



24/24

w

W

m

M

HAUBRICH, D. & MCLEAN, I. (2006) Assessing public service performance in local authoritiesthrough CPA a research note on deprivation, National Institute Economic Review , vol.197, no. 1, pp. 93 105.

HMSO (1999) Local Government Act 1999 , London: Her Majestys Stationary Ofce.HOOD, C., SCOTT, C., JAMES, O., JONES, G. & TRAVERS, T. (1998)Regulation Inside Government ,

Oxford: Oxford University Press.IMPROVEMENT AND DEVELOPMENT AGENCY (2006)Driving Improvement A New Performance

Framework for Localities , London: IDeA.NATIONAL ASSEMBLY FOR WALES (2006)Local Government and Public Services Committee

Meeting Minutes 9 Feb 2006 , Cardiff: National Assembly for Wales.OFFICE OF THE DEPUTY PRIME MINISTER (2005)Local Government Finance Statistics England No.

16 , London: ODPM.POWER, M. (1997)The Audit Society , Oxford: Oxford University Press.SCOTTISH EXECUTIVE (2005)The Local Government in Scotland Act 2003 * Best Value Guidance ,

Scottish Executive, Edinburgh, [online] Available at: www.scotland.gov.uk/Publications/

2005/01/20531/50061 (accessed April 2006).STOKER, G. (2005)Transforming Local Governance , Houndsmill: Palgrave MacMillan.WALES AUDIT OFFICE (2006)Wales Programme for Improvement Annual Report 2004/5 ,

Cardiff: Auditor General for Wales.

Dr Dirk Haubrich (corresponding author), Department of Politics and InternationalRelations, Research Ofcer, Department of Politics and International Relations, ManorRoad, University of Oxford, Oxford OX1 3UQ, UK. Tel: ' 44 1865 285977; E-mail:[email protected]

Professor Iain McLean , Professor of Politics, Department of Politics and InternationalRelations, Nufeld College, University of Oxford, New Road, Oxford OX1 1NF, UK. Tel:' 44 1865 278646; E-mail: [email protected]


Devolution Scotland Wales

Documents

Transcript of Devolution Scotland Wales