Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE...

7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…

http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 1/24

Journal of Comparative Policy Analysis: Research and Practice 2: 211–234 (2000)

c 2000 Kluwer Academic Publishers. Printed in The Netherlands.

Capacity Building as an Instrument ofInstitutional Reform: Improving the Quality ofHigher Education through Academic Audits inthe UK, New Zealand, Sweden, and Hong Kong

DAVID D. DILL

Professor of Public Policy Analysis and Education, University of North Carolina at Chapel Hill,

Abernethy Hall, Chapel Hill, NC 27599-3435

Key words: capacity building, institutional reform, academic audits, academic quality, imple-

mentation, higher education

Abstract

As countries convert from state to market-centered public policies, there is increasing interest

in new forms of public accountability. Capacity building initiatives that reform institutional frame-

works are useful policy instruments during this period of transition. What are the impacts and

implementation problems characteristic of this approach? This article reviews the experience with

“Academic Audit,” a capacity building accountability instrument for universities adopted in the UK,

Sweden, New Zealand, and Hong Kong. Academic audits altered the incentives for cooperative

behavior among faculty members to improve student learning. Identified implementation problemsincluded: training for the new process, the uncertainty of capacity building benefits, and the central

role of information.

Introduction

As many countries make the transition from state-centered to market-centered

public policies, there is increasing debate about the most effective forms of

public accountability for assuring the public interest (Kettl, 1997). Some advo-

cate the rapid adoption of performance-based funding, contracting, or orga-

nizational report cards (Horn, 1995; Gormley and Weimer, 1999), while others

suggest that there is the need for some intermediate form of accountability to

help publicly-funded organizations adjust to the new competitive realities. Re-cent research on developing countries for example has suggested that enhanc-

ing the performance of public sector organizations through capacity building

interventions designed to alter the “rules of the game” by which they function

can be a crucial means of managing the transition from state monopolies to

market competition (Grindle, 1997a). What are the characteristics of such “ca-

pacity building” approaches to the reform of institutional frameworks and how

can they best be implemented?



212 DILL

Higher education is an example of a public sector that has been experiment-

ing extensively with new forms of accountability. The rapid expansion of access

to higher education in Europe and Asia (termed “massification” outside the US)

stimulated public concern as to whether universities had in place sufficiently ef-

fectivemeans for assuring thequality of their programs and degrees. As notedin

a recent guide produced by the Quality Assurance Agency for Higher Education

(QAA, 1998) in the UK:

Universities and colleges have an obligation to protect the standing and good

name of higher education. When the higher education system was small and

largely uniform, and made a relatively small claim on public funds, reliance

upon implicit, shared assumptions and informal networks and procedures

may have been possible, and sufficient. But with the rapid expansion of thenumbers of students and institutions, the associated broadening of the pur-

poses of higher education, and the considerable increase in the amount of

public money, more methodical approaches have had to be employed to

provide the same guarantees. Institutions’ own internal mechanisms are im-

portant elements in providing these guarantees, but external scrutiny is also

needed to confirm that institutions’ responsibilities are being properly dis-

charged. The process of external scrutiny also makes an important contribu-

tion to the improvement of quality (pp. 6–7).

Lacking the tradition of voluntary accreditation characteristic of North America,

a number of countries have been experimenting with some novel forms of aca-

demic accountability designed to increase the quality of teaching and learningof undergraduate or first degree level university students (Brennan, De Vries,

and Williams, 1997; Scheele, Maassen, and Westerheijden, 1998).

One of these new forms of academic accountability, termed “Academic Audit,”

adopts a capacity-building approach to reform. Academic audits, first imple-

mented in the UK, attempt to assess the processes that universities have in

place for assuring the quality of student learning and the standards of their

degrees. Interest in this new policy instrument has become worldwide. Con-

cerns about the effectiveness of the existing quality assurance system of state

regulation and voluntary accreditation in the US have led to a number of new

experiments with academic audit (Dill, 2000).

These audit processes are not directly tied to funding, but evaluate and pro-

vide public reports on the quality assurance processes by which colleges anduniversities exercise their responsibility to ensure academic standards and im-

prove the quality of their teaching and learning. As observed in Sweden, the

focus of academic audits is not on the quality of measured outcomes, but on

“quality work” ( ¨ Ostling, 1997)—how a university satisfies itself that its chosen

academic standards are being achieved. The principal goal of the academic

audits is organizational improvement through the stimulus of external account-

ability for academic quality assurance processes.



CAPACITY BUILDING AS INSTITUTIONAL REFORM 213

Academic audit, from the perspective of current analyses of public sector re-

form, represents a working model of external accountability designed to achieve

institutional reform, i.e., to alter the underlying rules or incentive structures that

in turn influence the quality of teaching and student learning in higher educa-

tion. Academic work is carried out within an institutional framework that pro-

vides incentives and sanctions for individual and group behavior. These insti-

tutional arrangements include the rules and regulations of governments and

educational ministries affecting higher education, professional accountability

systems such as external examining and accreditation, the norms and beliefs

of the academic disciplines, the nature of competition in academic markets, and

the policies and practices of colleges and universities. Over time the relative

influence of the components of this institutional framework have evolved alter-

ing the incentives for academic work (Dill, 1997). Governments have thereforesought means of reforming the existing institutional framework in a manner that

provides greater incentives for the public interest. In order to clarify this focus

on institutional reform in higher education, the arguments that follow will use

the term “institutions” to refer to the institutional arrangements defined above

and the term “academic organizations” to refer to colleges and universities.

The sections to follow review the experience with academic audits in the

UK, New Zealand, Sweden and Hong Kong, assessing the impacts and im-

plementation problems characteristic of this capacity-building approach.1 The

first section suggests how capacity building can serve as a perspective for

exploring academic audits. The second section suggests why academic audit

can be conceived as a capacity building initiative for institutional reform and

summarizes the impacts of the initiative upon academic behavior. The final sec-

tion reviews the particular problems encountered in implementing this capacity

building approach to academic accountability. A more extensive discussion

of the organization and focus of the academic audit process is provided in

Appendix 1.

Capacity building and academic audits

Over the last decade as policymakers have turned to more market-oriented

means of achieving the public interest there has been a growing interest in

capacity building as an instrument of public policy (Honadle and Howitt, 1986;

McDonnell and Elmore, 1987; Grindle, 1997b). While the concept of “capacity

building” is ubiquitous in the policy literature, the meaning of the term is rather liquid. The Dictionary of Public Administration offers the following definition:

Capacity building. . . includes among its major objectives the strengthening of

the capability of chief administrative officers, departments and agency heads,

and program managers in general purpose government, to plan, implement,

manage, or evaluate policies, strategies or programs designed to impact on

social conditions in the community (quoted in Cohen, 1995, p. 409).



214 DILL

Following the first half of this definition Cohen (1995) argues that capacity build-

ing in the public sector must focus on developing human resource capacity and

therefore related interventions should be aimed at strengthening the training,

recruitment, effective utilization and retention of skilled public sector personnel.

Following the second half of this definition Honadle (1986) argues that capacity

building in the public sector must focus on developing the management capac-

ity of public organizations and agencies in a systemic sense. That is, the ability

of an organization to anticipate change, to formulate relevant policy, to devise

programs to implement policies, to attract as well as manage resources, and to

evaluate performance to guide future actions.

In contrast to these earlier perspectives Grindle (1997b) suggests that the

concept of capacity building must be reconciled with the new context of market-

oriented policies characteristic of contemporary changes in the public sector.Therefore capacity building initiatives in her view involve not only development

of human resources, and organizational strengthening, but also institutional re-

form (Table 1). Following North’s (1990) logic, Grindle suggests that capacity

building initiatives that reform institutions alter the underlying rules of the game

in which organizations and individuals make decisions and carry out activities.

Such initiatives involve development of new mechanisms of public accountabil-

ity, regulatory frameworks, and monitoring systems that help transmit informa-

tion about the structure and performance of markets, governments, and public

officials and thereby affect the incentive structure for economic and political

interaction.

This expanded framework of capacity building initiatives provides a use-

ful perspective for understanding the emergence and implementation of aca-

demic audits as a new mechanism of accountability in higher education policy.

For example, capacity-building initiatives are most likely to be pursued when

there is a perception that failures of organizational performance will lead to the

Table 1. Dimensions and focus of capacity building initiatives (Adapted from Grindle,

1997b, p. 9).

Dimension Focus Types of activities

Human resource Supply of professional and Training, salaries, conditions of

development technical personnel work, recruitment

Organizational Management systems to Incentive systems, utilization of

strengthening improve performance of personnel, leadership, organizationalspecific task and functions; culture, communications,

microstructures managerial structures

Institutional Institutions and systems; macro Rules of the game for economic

reform structures and political regimes, policy

and legal change, constitutional

reform




loss of future material, intellectual, or human benefits (McDonald and Elmore,

1987).

As previously noted the rapid expansion of higher education over the last

decade in Europe and Asia raised serious public concerns about the adequacy

of existing institutions for sustaining academic quality and standards and led to

the initiation of academic audit processes in a number of countries. For exam-

ple, the first academic audit process emerged in the UK following persistently

voiced concerns in the 1980s by the Thatcher government about the quality

of teaching in the rapidly expanding university sector.2 As a result the Com-

mittee of Vice Chancellors and Principals (CVCP), an independent association

of the heads of the UK universities of that time, established a series of study

committees to examine academic issues confronting the university sector. One

of the first committees was an Academic Standards group. By virtue of theRoyal Charters or private acts of parliament by which they were established,

British universities are responsible for the degrees they award and for their

own academic standards. Nonetheless, the CVCP was concerned that if the

universities were perceived as not monitoring their own standards, some other

agency might be assigned the job by the government. The Academic Stan-

dards Group issued a report in 1986 that recommended codes of practice for

critical academic processes such as subject examinations as well as means of

monitoring academic standards, which provided the universities yardsticks for

self-comparisons. But a later CVCP study of the extent to which the universities

had implemented the committee’s recommendations suggested that superficial

compliance was masking less than strong commitment. Under further pressure

from the government the CVCP’s Academic Standards Group recommended

the creation of an Academic Audit Unit (AAU) to provide external and indepen-

dent assurance that UK universities have adequate and effective mechanisms

and structures for monitoring, maintaining, and improving the quality of their

teaching. The unit would be “owned” by the universities themselves through

the CVCP.3

The creation of the New Zealand, Swedish, and Hong Kong academic audit

processes was similarly a response to pressure from government for greater

accountability on academic quality in the university system. In New Zealand

the 1990 Education Amendment Act established the New Zealand Vice Chan-

cellor’s Committee as a statutory body with explicit responsibility for standards

and qualifications in the university sector. The Act also strongly implied that

there should be active monitoring of these standards and qualifications. The

Committee therefore established an Academic Audit Unit in 1994 to audit thequality processes within the universities. In Sweden, the 1993 Higher Educa-

tion Reform decentralized the higher education system and as part of the re-

form directed that all universities should have quality enhancement programs.

A National Agency for Higher Education was created in 1995 that had a number

of evaluative responsibilities including the auditing of quality in the universities

and university colleges.



216 DILL

In Hong Kong the University Grants Committee was established in 1965 to

advise the government on the academic development and funding of the higher

education sector. As the system expanded in size and cost in the 1990s it

became the government’s and public’s expectation that the UGC would take an

active role in assuring the quality of higher education provision in the academic

organizations for which it had responsibility. Initially the UGC implemented a

Research Assessment Exercise similar to one developed in the UK in the mid-

1980s. This process tied the amount of UGC research allocations to universities

to an assessment of the quality of published research at the subject level in

each university. Recognizing the clear signal that this assessment sent about

the importance of research, the UGC then instituted in 1995 a process to audit

the quality of teaching and learning—Teaching Learning Quality Process Review

(TLQPR).The evolution of academic audit as a new means of accountability in higher

education therefore seems consistent with the emergence of capacity building

initiatives in international development work (Grindle, 1997b) and in K-12 edu-

cation in the US (Goertz, Floden, and O’Day, 1995). Of greater interest to those

interested in policy analysis and design, however, is the way that academic

audits illustrate what Grindle (1997b) terms institutional reform and the partic-

ular implementation problems associated with this type of capacity building

initiative.

Academic audits as institutional reform

In what way can academic audits be classified as a capacity building initia-

tive designed to reform the institutional framework for academic quality? To

understand this approach it is necessary to develop three arguments. First, to

conceive of academic quality assurance in teaching and student learning as a

problem of collective action. Second, to suggest how the changing institutional

framework of academic quality assurance is affecting academic behavior. And

third, to examine the impacts of academic audits on the cooperative efforts of

faculty members to improve academic quality.

Academic quality as a problem of collective action

From North’s (1990) perspective institutions may be understood as mechanismsfor addressing dilemmas of collective action through the structuring of coop-

erative behavior. “Dilemmas of collective action” are defined as those circum-

stances in which individuals may fail to coordinate their activity to produce a

collective benefit, because it is not in their private interests to do so. That is,

in the absence of credible commitments to one another, individuals may have

incentives to defect from or shirk cooperative behavior in favor of individually

beneficial activity. As North (1990) illustrates in his comparison of developed




and underdeveloped countries, existing institutional structures do not always

lead to productive human interaction. Unless institutional frameworks provide

incentives for verifiable, enforceable, mutual commitments human interaction

may be structured in a manner that is inefficient for the larger society. North’s

perspective is thereby valuable for assessing the extent to which mediating

institutions encourage individuals to cooperate for the common good. A first

issue, therefore, is how can academic quality assurance be understood as a

problem of collective action?

An important and controversial issue in the development of quality assurance

policies on teaching and learning is how quality assurance is conceptualized

(Dill, 1992). Is academic quality primarily the product of the independent ac-

tions of individual professors, or is it significantly influenced by the cooperative

efforts of program faculties and departments? Emerging but not yet definitiveevidence suggests that academic quality assurance is best conceptualized in

a holistic manner that is greater than the sum of the activities of individual

teachers in separate classrooms (Ewell, 1988; Pascarella and Terenzini, 1991).

As Tony Becher (1992) noted in developing a quality assurance system at the

University of Sussex in the UK:

. . . the most important consideration in quality assurance must be a holis-

tic rather than an atomistic one, namely the benefits students derive from

the totality of their degree programmes, rather than the satisfactoriness or

otherwise of their interactions with individual members of staff (p. 58).

Similarly, research in the United States on teaching and learning (Pascarella andTerenzini, 1991) consistently reveals that, while student content learning and

cognitive development is related to the quality of individual teaching students

receive, it is also affected by the nature and sequence of students’ curricular

experiences, and by the extent to which the curriculum faculty are collectively

involved with and communicating with each other about the substance of teach-

ing and learning.

This research suggests that more systematic efforts to improve the quality

of student learning will likely require cooperative actions by faculty members

to assess learning outcomes, to “restructure” the curriculum, and to better

coordinate their individual efforts at instruction.

The changing institutional framework of academic quality

Cooperative behavior in academic quality assurance within universities has tra-

ditionally been influenced by a number of interacting institutions. These include

the rules and sanctions of academic organizations, the norms and incentives

of the academic disciplines, and the influence of the competitive market for

academic products and labor. There is increasing evidence that over time the

traditional institutional framework is changing and has become less effective



218 DILL

for motivating faculty collaboration in improving teaching and student learning

within colleges and universities. The evolving institutional framework for higher

education appears to provide strong incentives for faculty disengagement from

the cooperative activities that have been linked to the improvement of academic

quality particularly in undergraduate or first degree-level education.

One example of this shifting framework is the growing “marketization” of

higher education (Williams, 1995). Global economic forces are leading to in-

creasing international competition for students, faculty members, and pro-

grams. Within countries and states this competition is being further fostered

by the adoption of quasi-market and market-based policy instruments by gov-

ernments as a means of achieving greater efficiency and innovation in their

expanding higher education systems (Dill, 1997). Because of the structure of

academic organizations, this shift in the competitive context can affect facultymembers’ incentives for teaching versus research. In a series of influential pa-

pers James (1978, 1986, 1990; James and Neuberger, 1981) argued that univer-

sities are multi-product firms, producing both teaching and research, in which

the faculty have significant control over the factors and consequently the costs

of production. As Clotfelter (1996) recently observed:

The university’s central and most distinctive activities—teaching, research,

and public service—are carried out largely by its most distinctive sector of

employees: the faculty. As a consequence, the decisions about how to allo-

cate faculty effort are basic to the functioning of colleges and universities,

and to their cost. . . . most day-to-day decisions concerning these activities

are entirely in the hands of departments and faculty members themselves. . . .(p. 179)

James (1986) noted that faculty members, particularly in universities, tend to

value research over teaching, because of its intrinsic interest, because of its

clear contribution to unit reputation (which is a major proxy for academic qual-

ity), and because in increasingly competitive research and labor markets time

spent on research can lead to enhanced grant revenue and future earnings

for the individual faculty member. In this context, and in the absence of close

monitoring and valid measures of student learning, faculty members both indi-

vidually and collectively will choose to “satisfice” teaching quality, to limit their

time investment in teaching and to maximize their time investment in graduate

teaching and research.James (1986) tested this model by examining the trends in the percentage

of self-reported faculty time devoted to teaching and in the average teaching

loads at universities during the period 1953–1965, when the US higher educa-

tion system evolved from an elite to a mass system. Her analysis confirms in

major universities a significant decline in reported time spent teaching and an

increase in reported time spent on research. More recent studies (Clotfelter,




1996; Fairweather, 1996; Getz and Siegfried, 1991) suggest a continuation of

this trend toward faculty disengagement in teaching and greater investment of

time in research.4 Similar concerns about a shift in faculty time toward research

have been noted in the UK and in Hong Kong following the implementation

of policies featuring more competitive systems for the allocation of university

research funds (Dill, 1997; Massy, 1997a).

There is also some evidence that the influence of the norms and values of the

academic disciplines on faculty collective behavior is eroding in the new con-

text of “massification” and “disciplinary fragmentation.” The explosive growth

of knowledge in the traditional disciplines as well as the evolution of new inter-

disciplinary fields such as molecular biology, public policy, and ethnic studies,

have led to the rapid expansion of academic programs in universities through-

out the world (Clark, 1996).5 There is emerging evidence in the US system thatthis expansion of academic knowledge as well as related specialization is af-

fecting the ability of faculties to reach consensus on what should be taught

within academic curricula, or what constitutes an academic curriculum at the

university level. Surveys on the views of faculty members in different disci-

plines (Lattuca and Stark, 1994) suggest declining normative influence from the

various academic professions on the content of academic programs. In many

disciplines faculty members did not easily agree on definitions of curricula con-

tent, nor were they in agreement that specified sequences of learning content

were appropriate for students. Even in professional fields such as business

and engineering, where faculty members reported the highest perceived con-

sensus on the nature of academic knowledge, they also reported that defining

the content of the professional courses was one of their most serious curricu-

lum tensions. In several disciplines, faculty members expressed the belief that

the field’s diversity precluded achieving a consensus on what students should

know.

Finally, the university itself may be conceived as an institutional structure

that can influence academic quality. Potentially universities can promulgate

rules and specify incentives or sanctions to improve the design of academic

programs, increase the time commitment of faculty members to teaching, and

stimulate collegial responsibility for academic quality. There is some evidence

from the US that the influence of university-based rules and sanctions on faculty

collaborative behavior is declining.6

Field research at the departmental level in US universities (Massy, Wilger,

and Colbeck, 1994) has revealed a pattern of “hollowed collegiality” in which

departments nominally appear to act collectively, but avoid those specific col-laborative activities that might lead to real improvements in curricula cohesion.

For example, faculty members readily reported informal meetings to share re-

search findings, collective procedures for determining faculty promotion and

tenure, and consensus decision making on what particular courses should be

offered each term and who should teach them. But:



220 DILL

Despite these trappings of collegiality, respondents told us they seldom

led to the more substantial discussions necessary to improve undergraduate

education, or to the sense of collective responsibility needed to make de-

partmental efforts more effective. These vestiges of collegiality serve faculty

convenience but dodge fundamental questions of task. This is especially the

case, and is regrettable, with respect to student learning: collegiality remains

thwarted with regard to faculty engagement with issues of curricular struc-

ture, pedagogical alternatives, and student assessment (Massy, Wilger, and

Colbeck, 1994, p. 19).

A major contributor to this hollowed collegiality was an observed pattern of

fragmented communication within departments in which traditions of individual

autonomy and academic specialization have led to atomization and isolationamong faculty members. Not only do faculty members do much of theirteaching

alone, but also because disciplinary sub-fields are defined quite narrowly, many

faculty members find it almost impossible to discuss their teaching with other

faculty members.

The above analysis suggests that maintaining or improving academic quality

represents a problem of collective action. The evolving institutional framework

of market competition, professional norms, and university rules and sanctions

appears to provide insufficient or negative incentives for cooperative academic

activity leading to improved student learning. In this new context there appears

to be an increasing rationale for government accountability initiatives such as

academic audits designed to alter the existing incentive structure for academic

quality. What then has been the impact of academic audits on faculty cooper-ation for improving teaching and student learning?

The impacts of academic audits on academic behavior

The academic audit processesin the UK, NewZealand, Sweden, and Hong Kong

have each been evaluated (Coopers and Lybrand, 1993; Brennan et al., 1999;

Stensaker, 1999; Meade and Woodhouse, 2000; Nilsson and Wahlen, 2000).

These evaluations have identified the types of impacts of the first cycle of aca-

demic audits on academic behavior within universities. These initial academic

audits have accomplished the following:

r

Placed responsibility for improving teaching and student learning on uni-versity agendas. The public nature of the academic audit process and the

publication of the audit reports placed responsibility for improvement of the

processes designed to assure and improve the quality of teaching and stu-

dent learning on the agenda of each audited college and university, in many

instances for the first time (Coopers and Lybrand, 1993; Brennan et al., 1999;

Meade and Woodhouse, 2000; Nilsson and Wahlen, 2000). Because there




were in every academic organization some faculty members who recognized

the need for such improvement, this public attention provided an opportunity

for changes in university policies and structures that were often described

after the fact as “already underway.” r Facilitated active discussion and cooperation within academic units on means

for improving teaching and student learning. All the evaluations noted that

faculty members at the departmental level reported an increased amount of

collegial discussion and cooperative activity on academic quality assurance.

This was particularly the case in the Swedish and Hong Kong versions of aca-

demic audit, which encouraged both university and departmental responses

to the audit reviews (Brennan et al., 1999; Stensaker, 1999). r Helped to clarify responsibility for improving teaching and student learning at

the academic unit, faculty, and organizational level. The academic audit pro-cess often revealed that a pervasive norm in many academic organizations is

the avoidance of responsibility for the quality of teaching and student learn-

ing. University administrators often adopted an “invisible hand” approach to

academic quality assurance, either because they equated academic qual-

ity with recruiting the best faculty members and students and leaving them

alone, or because they misinterpreted academic freedom to mean that indi-

vidual faculty members have an unchallenged right to determine the content

of their courses.7 In either case, the academic audit process was frequently

reported as clarifying that the departments, faculties, and university leader-

ship each have a responsibility for the active development of quality assur-

ance and improvement processes for teaching and student learning (Massy

and French, 1999; Stensaker, 1999; Nilsson and Wahlen, 2000). r Provided information on best practices both within academic organizations

and across systems. The process of academic audit often revealed to admin-

istrators, sometimes for the first time given the norm of the “invisible hand,”

that there were “best practices” of academic quality assurance within their

universities that could be disseminated as a means of academic improvement

(Brennan et al., 1999). Similarly, the UK developed a “quality enhancement”

function designed to disseminate best practices discovered through the pro-

cess of academic audit (Coopers and Lybrand, 1993).

There is some evidence from these countries that the academic audit initia-

tives have altered the institutional framework for academic quality assurance,

at least in the short run, and provided some incentives and support for collec-

tive actions designed to improve the quality of teaching and student learning.In reaction to these reviews, the universities audited are changing their internal

norms, policies, and structures with regard to “quality work.”8 In the process

they are more closely approximating “learning organizations” (Dill, 1999a), build-

ing a capacity for developing and transferring knowledge for the improvement

of their core academic processes.



222 DILL

Policy implementation

The introduction of academic audit into the higher education sector also illus-

trated a number of special implementation problems characteristic of capacity

building initiatives designed for institutional reform. These included whether

the initiating agency has developed the knowledge and skills necessary to con-

duct process-oriented audits, the concern of policy makers and others about

the intangible and uncertain results of capacity building activities, and the role

of information in capacity building efforts designed to alter the structure of

incentives.

Training for process review

A distinctive characteristic of academic audit is its focus on process. The Hong

Kong TLQPR for example evaluated quality assurance processes in undergrad-

uate or first degree teaching such as the means of designing curricula, evaluat-

ing teaching, assessing learning outcomes, and providing resources to quality

assurance work. In each country, however, it quickly became apparent that even

experienced academics—whether auditors or audited—may not have a clear

conception of what constitutes academic quality assurance process. In each

of the systems there were reports of confusion both among those audited and

those conducting the audits as to the differences between academic substance

and process, outcomes and process, objectives and process, etc. Auditors and

faculties reviewed both reported that a great deal of learning took place about

the natureof academic quality assurance processes and best practices in higher education. This discovery led in every country to a major emphasis on auditor

training and to preparatory visits designed to orient universities to the purposes

and practices of academic audits.

With the exception of Hong Kong, which utilized an intact team for all reviews

that was composed primarily of members of the University Grants Committee

and local university representatives, the other audit processes had to recruit

auditors. Recruitment was often carried out through the professional contacts

and experience of the individual directing the audit process, although university

nominations and individual applications for auditor were invited in the UK, New

Zealand, and Sweden. In the UK the Academic Audit Unit evolved a process of

selection in which potential auditors were invited to participate in a day of role-

play with audit materials. In this manner their knowledge of quality assuranceprocesses, understanding of the goals of audit, and skills in working as a mem-

ber of a team could be assessed prior to their actual selection as an auditor.

Of particular concern to the AAU was weeding out “quality ideologues,” those

who felt that there was “one best way” to assure academic quality assurance.

New Zealand, Sweden and the UK all used unique teams of auditors, con-

structed for each audit visit. All but Sweden also created “banks” of trained

auditors, individuals appointed for a period of time, who were then used for




a number of different audits as a means of improving audit reliability. Hong

Kong, by contrast, utilized an intact team over the course of a year for all its

university audits as a means of enhancing auditor learning and the reliability

of the process. The relatively small size of the Hong Kong system (seven col-

leges and universities at the time of the audits) made this a feasible approach.

The employment of an intact team enabled the audit process to begin more

quickly, with both the auditors and the universities learning about the process

as it evolved.

The UK, New Zealand, and Sweden also developed auditor training in which

members of audit teams were given an opportunity to read and critique pilot

audit reports, interview those involved in previous audits, and develop skills in

questioning. Sweden, New Zealand and the UK developed audit protocols to

help guide auditors in their visits. Audit guides including these protocols weremade publicly available so as better to inform universities about how audits

would be conducted. The openness of this process underscores the emphasis

on development and improvement as a goal for the audit process.

The uncertainty of capacity building benefits

As McDonald and Elmore (1987) suggest, one limitation of capacity building

initiatives is their future orientation and the concern this raises in the minds

of policy makers about the benefits of the process. Building the capacity of

universities to assure academic quality offers the potential of future returns,

but because the benefits are distant, uncertain, and difficult to measure policy-

makers may have problems justifying such initiatives and supporting them over time. This issue was raised in a number of the external evaluations of academic

audits (Coopers and Lybrand, 1993; University Grants Committee, 1997; Nilsson

and Wahlen, 2000). That is, the logic of academic audits relies upon an underly-

ing but difficult to prove assumption—that an improvement in quality assurance

processes will eventually lead an improvement in academic outcomes. Under-

standably, academics and policy makers sought evidence confirming that such

a relationship actually existed, evidence that academic audits with their focus

on process could not easily provide.

As academic audit matures, however, the reviews are responding to this crit-

icism by focusing more sharply on the measures of learning outcomes em-

ployed by academic programs and universities themselves as well as on the

role such measures play in university quality assurance and improvement pro-cesses. In the second cycle of academic audit in the UK, “Continuation Audit”

teams are pursuing the following question as a primary means of evaluating

how effectively universities are discharging their responsibilities for the aca-

demic standards and quality of their programs and awards (QAA, 1998): “Can

you convince us that the evidence that you are relying on for this purpose is

sufficient, valid and reliable?” In both New Zealand (Meade and Woodhouse,

2000) and Hong Kong (Massy and French, 1999) it has been proposed that the



224 DILL

second cycle of academic audit focus more on the outcomes of the quality

assurance processes being audited by asking how the universities themselves

measure such outcomes. In the US the recently designed program accreditation

process being implemented by the Teacher Education Accreditation Council

(TEAC, 1998), which is based upon the precepts of academic audit, will require

that each program to be accredited submit an “Inquiry Brief,” which demon-

strates and documents evidence: 1) about students’ learning and understand-

ing of the curriculum, 2) that assessments of student learning are valid, and 3)

that the program’s quality assurance system yields information that leads to

program improvement.

The role of information

In contrast to traditional initiatives for capacity building, which have emphasized

improvement in personnel or training in new management systems (Cohen,

1995), capacity building initiatives designed to reform institutions appear most

effective when they emphasize the communication of information (Jacobs and

Weimer, 1986).

The success of academic audits must be attributed in part to their clear com-

munication of public expectations that academics are both individually and

collectively responsible for assuring academic quality. By conducting and pub-

lishing the academic audits, the auditing agencies clarified both the faculty’s

professional authority for assuring academic standards and their accountability

to the public for carrying out this responsibility. The academic audits directly

challenged the prevailing norm of the “invisible hand” approach to academicquality assurance. The resulting debates within the audited universities about

the appropriate balance between individual and collective responsibility for aca-

demic quality and student learning were generally seen to be valuable. The aca-

demic audit reports also helped to make public for the first time variations in

quality assurance processes across units and universities and thereby provided

an opportunity for improvement through transfer of best practices both within

and across universities.

Academic audits also helped to foster communication among academics on

issues related to improving student learning (Dill, 1999a). First, the academic

audits introduced a common language for discussing academic quality assur-

ance, which facilitated collective discussion within and across academic de-

partments. Second, academic audits themselves helped to create or strengthenchannels of communication. For example, academic audits frequently stimu-

lated the adoption of systematic feedback mechanisms from the clients of aca-

demic programs—whether stakeholders external to the universities or currently

enrolled students. If the audit processes were designed to facilitate the devel-

opment of academic quality assurance processes at both the university and

the departmental level, as they were to differing extents in the UK, Sweden, and

Hong Kong, they also helped to stimulate vertical communication on academic




quality assurance between departments and central administrations as well as

horizontal communication among faculty members within a department. In a

number of cases, once developed, these communication channels have been

further strengthened as part of an ongoing improvement of quality assurance

processes and procedures within the audited universities (Dill, 1999a).

As noted above, perhaps the greatest contribution of academic audit may

be in encouraging faculties to develop public measures of the value-added by

academic programs. The lack of this information appears to be the most critical

cause of the dilemma of collective action in academic quality assurance (Dill,

1999b). As game theorists have noted (North, 1990), even in a context charac-

terized by small numbers and repeated dealings among individuals, such as an

academic department, unless individuals have sufficient information to calcu-

late relevant costs and benefits cooperative behavior for joint gain is unlikelyto occur. But summative measures of student learning, such as comprehensive

examinations or common projects, are becoming less common in all systems

of higher education (Dill, 1999b). With the increasing privatization of academic

work we lose the information that once made teaching and student learning

“community property” (Shulman, 1993).

Thus, if a measure of the value-added to students by an academic program

is not publicly available, then individual faculty members will base their deci-

sion on the amount of time to commit to cooperative activity in teaching and

curriculum improvement on the individual costs and benefits to themselves.

The benefit of cooperating with other faculty members in the design and imple-

mentation of higher quality educational academic programs is therefore likely

to receive little or no value. By the same logic, faculty members also have little

incentive to invest time and effort in developing or maintaining measures of the

value-added by academic programs, therefore the decline or rise in the quality

of student learning becomes increasingly invisible and cannot act as a check

on opportunistic faculty teaching behavior. By creating an incentive for univer-

sities as well as academic departments and programs to collectively assess

and discuss student learning, academic audit helps to create the information

necessary for cooperative behavior in professional work.

Conclusion

The shift from policies of state-supervision to market-based steering in the pro-

vision of public services such as higher education is an obvious trend in manycountries of the world. How best to assure public accountability in these new

systems is a point of continuing discussion and debate. Accountability instru-

ments that adopt a capacity building approach, such as academic audit, appear

to offer a number of advantages. First, if focused on information and commu-

nication, such capacity building initiatives have the potential for institutional

reform by altering well-entrenched professional norms and incentive structures

in public organizations that affect the ability of these organizations to respond to



226 DILL

market signals. Second, capacity building initiatives for institutional reform can

serve as effective transitional policy instruments that can be “faded” as markets

become more perfectly competitive and thereby more effective in steering pub-

lic sector organizations in the public interest. Finally, capacity building initiatives

such as academic audit are a relatively flexible and generalizable mechanism of

public accountability. The rapid diffusion of this new accountability mechanism

across cultures and countries offers some insights into the characteristics of

policy instruments that are more easily transferable.

Capacity building approaches that focus on information and communication

such as academic audit appear to be particularly effective in reforming the in-

stitutional framework of professionally based public organizations such as uni-

versities. In these settings the value of professional specialization may come

to dominate the needs of clients, customers, and the public interest. Prevailingprofessional norms and practices such as academic freedom may then pro-

vide cover for the pursuit of private interests. In the instance of organizations

like universities that offer joint products this problem is further confounded by

the evolution of differentially competitive markets for research and educational

programs. This imbalance can create incentives for faculty members to cross-

subsidize time for research with time for teaching. As will be noted below, this

potentially inefficient institutional framework may be corrected in the long term

by freeing and facilitating market competition for educational programs through

policy instruments such as performance-based funding, organizational report

cards, and student-based funding. But in a sector dominated by publicly funded

organizations, the failure of universities to adapt to these market signals may

have significant consequences for equal access to quality education. Further-

more, professional values and norms are deep-seated and, given the neces-

sary tradition of professional autonomy, difficult to alter through market signals

alone. Information-based accountability initiatives such as academic audit can

confront professional values and practices at the level of the basic production

unit, utilizing peer-based discussions and reviews that are more likely to lead

to behavioral change over time. If well designed, capacity building initiatives

can encourage the development of new communication channels between the

larger society and the focal organization, as well as restore communication

links among the professionals themselves, helping to rebuild collegial bonds

supportive of collective action on academic programs.

Capacity-building accountability approaches such as academic audit also of-

fer a valuable policy instrument for making the transition from systems of state

supervision to more market-based policies. Over time changes in the marketstructure for academic programs as well as in the underlying technology of

teaching and learning will likely be essential for reforming the institutional struc-

ture influencing academic work. Academic audits are of necessity occasional

and as such can sometimes be successfully resisted or compromised. The rapid

development of international distance learning and the adoption of information

technology in campus-based teaching already offer some evidence of the ways



228 DILL

Kingdom the Academic Audit Unit (AAU) of the Committee of Vice Chancellors

and Principals (CVCP) was influenced by the concept of “quality audits” as

defined by the International Standards Organization:

Quality audit is a systematic and independent examination to determine whe-

ther quality activities and related results comply with planned arrangements

and whether these arrangements are implemented effectively and are suitable

to achieve objectives (HEQC, 1993).

In adapting this concept of a quality audit to higher education, the AAU defined

the “planned arrangements” referred to in the ISO definition as “those processes

by which academic institutions exercise their responsibility to assure academic

standards and improve the quality of their teaching and learning.” Academic audits of universities are now being conducted on a comprehen-

sive basis in the UK, New Zealand, Sweden, and Hong Kong. Each of the audit

approaches has in common an organizational focus, an orientation to quality

process, and a similar procedure for evaluation featuring peer review, a univer-

sity submission, a site visit, and a published report. However, other aspects of

the audit designs vary (Table 2). For example, the audit processes differ in the

composition of the audit teams, the continuity of the teams in the reviewing

process, the length of the audit visit, the means of distributing reports, and the

follow-up activities. Some of these differences are likely due to the substantial

variation in system size among the countries, ranging from the UK’s 500+ col-

leges and universities to New Zealand and Hong Kong with only 7 colleges or

universities each. Other variations (e.g., the composition of audit teams) appear to be due to political or cultural considerations.

Audit teams were composed primarily of experienced academics in every

country (the majority of the team members representing the University Grants

Committee in Hong Kong were also local academics). In Sweden there was a

student member of every team, while in both Sweden and New Zealand non-

academics were also appointed. The UK, New Zealand, and Sweden all used a

unique team for each audit, although the UK and New Zealand appointed for a

period of years audit panels from which they could draw an audit team. In Hong

Kong, a single large team was used to review all colleges and universities in

the first cycle of academic audits. Audit teams generally visited each university

twice: a first, briefer visit in which representatives of the audit team introduce

the audit process to the academic community and answer questions, and a

second visit during which the actual audit was conducted. Audit reports were

sent to the relevant university for factual correction in each country and then

were made public by the auditing agency by direct distribution, press releases,

and/or by publication on the Internet. All the countries instituted feedback visits

or progress reports to follow-up on the recommendations of the audit team. The

UK, as well as Sweden and Hong Kong to a lesser degree, conducted quality




T a b l e 2 .

B a s i c o r g a n i z a t i o n o f a c a d e m i c a u d i t s .

U K

N e w Z e a l a n d

S w e d

e n

H o n g K o n g

D a t e B e g u n

1 9 9 0

1 9 9 4

1 9 9 5

1 9 9 5

C o l l e g e s a n d

1 8 0 u n i v e r s i t i e s ,

7 u n

i v e r s i t i e s

3 6 c o l l e g e s a n d

u n i v e r s i t i e s

7 o r g a n i z a t i o n s

u n i v e r s i t i e s

c o l l e g e s o f h i g h e r

( p o l y t e c h n i c s ,

a u d i t e d

e d u c a t i o n , a n d

u n i v e r s i t i e s , a n d a

s p e c i a l i s t a c a d e m i c

c o l l e g e )

o r g a n i z a t i o n s

M a k e u p o f a u d i t

t e a m

( 4 )

( 7 )

( 7 )

( 1 8 )

3 U K a c a d e m i c s

S e c r e t a r y

2 N Z

a c a d e m i c s

3 S w e d i s h a c a d

e m i c s

8 U G C

2 I n d u s t r y / C o m m e r c e

1 I n d u s t r y / G o v e

r n m e n t

8 H K a c a d e m i c s

1 O v

e r s e a s a c a d e m i c

2 O v e r s e a s a c a d e m i c s

D i r e c t o r A A U

1 S t u d e n t

S e c r e t a r y

S e c r e t a r y

P r o j e c t o f fi c e r

T e a m

U n i q u e t e a m

U n i q

u e t e a m

U n i q u e t e a m

I n t a c t t e a m

C o n t i n u i t y

A u d i t p a n e l

A u d i t p a n e l

A u d i t V i s i t

T w o v i s i t s

T w o

v i s i t s

T w o v i s i t s

T w o v i s i t s

S e c o n d v i s i t 2 - 4 d a y s , v a r i e s

S e c o n d v i s i t 3 d a y s

S e c o n d v i s i t 2 - 3

d a y s , v a r i e s

S e c o n d v i s i t 1 . 5 d a y s

b y u n i v e r s i t y s i z e

b y u n i v e r s i t y s i z e

R e p o r t

S e n t t o u n i v e r s i t y




D i s t r i b u t i o n

A v a i l a b l e t o p u b l i c

A v a i l a b l e t o p u b l i c

A v a i l a b l e t o p u b

l i c

I n t e r n e t

D i s t r i b u t e d



I n t e r n e t

F o l l o w - u p

P r o g r e s s r e p o r t

V o l u

n t a r y p r o g r e s s r e p o r t

F e e d b a c k v i s i t

P r o g r e s s r e p o r t

Q u a l i t y e n h a n c e m e n t

P u b l i s h e d s t u d i e s

T e a c h i n g w o r k s h o p

P u b l i s h e d s t u d i e s

E v a l u a t i o n

E v a l u a t i o n

C o d e s o f c o n d u c t

E v a l u a t i o n



230 DILL

enhancement efforts designed to disseminate examples of good practice in

academic quality assurance identified through the audit process.

Another source of variation among the audit approaches is the scope of the

processes they assess. The widest scope is to be found in the audits carried

out in Sweden, which focused on evaluating university “quality enhancement”

programs and their implementation. Therefore, Swedish quality audits exam-

ined: university strategies for quality implementation; how and to what extent

teachers, researchers, administrative personnel and students were committed

and involved in quality enhancement programs (“universal participation in qual-

ity enhancement”); the ways external stakeholders had been identified and their

needs and demands determined (“cooperation among interested parties”); and

methods for evaluating activities with particular attention to staff and student

recruitment and development (“evaluatory and follow-up systems”). Given thegovernment’s desire to encourage university self-regulation, the Swedish au-

dits placed particular emphasis on the exercise of organizational leadership in

quality enhancement programs and on the strategies utilized to develop na-

tional and international contacts for improving professional activities (“external

professional relations”).

The New Zealand approach, modeled on the UK, focused attention on means

of quality assurance in the appointment and development of staff, as well as

quality assurance mechanisms in courses and programs. But the New Zealand

approach added the additional topic of quality assurance in research with par-

ticular regard to its connection with teaching (Woodhouse, 1998).

The UK academic audit in its earliest manifestation focused on university

mechanisms for quality assurance in relation to academic staff, in the provision

and design of courses and degree programs, in teaching and communication

methods, and in taking account of the views of stakeholders.

The Hong Kong TLQPR has the most carefully defined scope of the four

processes reviewed, evaluating quality assurance processes in undergraduate

or first degree teaching including the means of designing curricula, evaluat-

ing teaching, assessing learning outcomes, and providing resources to quality

assurance.

Acknowledgments

This article is a revised version of a paper presented at the annual meeting of the

Association for Public Policy Analysis and Management (APPAM), WashingtonDC, November 1999. The paper was developed with the support of a grant from

the Pew Charitable Trusts. I wish to give my sincerest thanks for useful criticism

to David Breneman, Iris Geva-May, an anonymous reviewer, and Brian Kropp as

well as to a large number of people involved with academic audit in the listed

countries and agencies who provided valuable information and advice, most

particularly to Nigel French, William Massy, Lars Niklasson, Malin ¨ Ostling, Bjorn

Stensaker, Staffan Wahlen, Peter Williams, and David Woodhouse.




Notes

1. The article focuses on academic audit activities in the UK, New Zealand, Sweden and Hong

Kong because each of these countries has completed a full cycle of academic audits of their

university sector and an evaluation of the process has been conducted. The analysis is based

upon interviews and correspondence with administrators in the various audit agencies as well

as participation in an external evaluation of the Hong Kong TLQPR program. In addition the

following sources wereutilized:Hong Kong(Massy, 1997a; Massy and French, 1997;UGC, 1997);

New Zealand (Woodhouse, 1995a, 1995b); Sweden(National Agency for Higher Education,1996; ¨ Ostling, 1997); United Kingdom (Williams, 1992).

2. An early advocate of the term “educational auditing” in the US was Fred Harcleroad, who

suggested a model for reforming US academic accreditation that was drawn from the financial

auditing system developed over time by the Securities and Exchange Commission (Harcleroad,

1976).

3. The Academic Audit Unit (AAU) was established by the Committee of Vice Chancellors andPrincipals (CVCP) in 1990. In 1992, with the passage of the Higher and Further Education Act,

UK higher education was substantially reorganized and Academic Audit was assigned to a

new intermediary agency, The Higher Education Quality Council. In addition, Higher Education

Funding Councils were created for England, Scotland, and Wales, part of whose responsibility

was to initiate a separate system of assessments of teaching quality at the subject level in all UK

universities. Following the publication of the Dearing Report on Higher Education in 1998, the

recently elected Labor Government endorsed the creation of a new UK-wide Quality Assurance

Agency (QAA), which now has responsibility for completing the cycles of the two preexisting

quality processesas well as designing a new overall process. Theanalysisin this article focuses

on the first cycle of Academic Audit as it was originally implemented under the CVCP.

4. Many would argue that teaching and research, particularly in the university sector, are joint

products—faculty time spent on research can improve the quality of academic content all stu-

dents learn (Clark, 1997). However as Massy (1996) notes, a faculty members’ commitment to

the quality of teaching and the quality of research logically requires a tradeoff at some point.

If a faculty member’s time commitment to research, unconstrained by institutional structures,increasesbeyond this point, it becomes a substitute for time committed to teaching. The quality

of student learning inevitably will be affected negatively by this decline in the time faculty mem-

bers can devote to assessing students, to the design of new curricula and teaching methods,

and to teaching itself.

5. Commenting on the contribution that disciplinary fragmentation makes to the complexity of

higher education systems, Clark (1996) observed:

in mathematics, 200,000 new theorems are published each year, periodicals exceed 1,000,

and review journals have developed classification schemes that include over 4,500 subtopics

arranged under 62 major topic areas. In history, the output of literature in the two decades of

1960–1980 was apparently equal in magnitude to all that was published from the time of the

Greek historian Thucydides in the fourth century B.C. to the year 1960. In psychology, 45 ma-

jor specialties appear in the structure of the American Psychological Association, and one of

these specialties, social psychology, reports that it is now comprised of 17 subfields . . . . In the

mid-1990s, those who track the field of chemistry were reporting that more articles on chem-istry have been published in the past 2 years than throughout history before 1900. Chemical

Abstracts took 31 years to publish its first million abstracts, 18 years for its second million, and

less than 2 years for its most recent million. An exponential growth of about 4 to 8 percent an-

nually, with a doubling period of 10 to 15 years, is now seen as characteristic of most branches

of science (pp. 421–422).

6. Similarly, the first cycle of academic audit in the UK discovered serious erosions in the well-

established university process of external examining. The audits revealed substantial variations



232 DILL

in examiner appointment and procedure across subject fields, ineffective monitoring by the

universities, and a frequent failure to utilize examiners’ reports for educational improvement

(HEQC, 1994).

7. Therelationshipbetween academic freedom and the collectivefacultyresponsibility for teaching

and learning is thoughtfully explored in the US context by Rosovsky and Ameer (1998).

8. For examples of the types of policy changes and structural adaptations made by universities

participating in audit processes, see Dill (1999).

References

Becher, T. (1992). “Making Audit Acceptable: A Collegial Approach to Quality Assurance.” Higher

Education Quarterly 46, 47–66.

Brennan, J., P. De Vries, and R. Williams (eds.) (1997). Standards and Quality in Higher Education.London: Jessica Kingsley.

Brennan, J., D. Dill, T. Shah, A. Verkleij, and D. Westerheijden (1999). A Campaign for Quality: Hong

Kong Teaching and Learning Quality Process Review . Hong Kong: University Grants Committee.

Clark, B.R. (1996). “Substantive Growth and Innovative Organization: New Categories for Higher

Education Research.” Higher Education 32, 417–430.

Clark, B.R. (1997). “The Modern Integration of Research Activities with Teaching and Learning.”

Journal of Higher Education 68, 241–255.

Clotfelter, C.T. (1996). Buying the Best: Cost Escalation in Elite Higher Education. Princeton: Prince-

ton University Press.

Cohen, J.M. (1995). “Capacity Building in the Public Sector: A Focused Framework for Analysis and

Action.” International Review of Administrative Sciences 61, 407–422.

Coopers and Lybrand (1993). Review of Quality Audit: Higher Education Quality Council. London:

HEQC.

Denzau, A.T. and D.C. North. (1994). “Shared Mental Models: Ideologies and Institutions.” Kylos 47,

3–31.Dill, D.D. (1992). “Quality by Design: Toward a Framework for Academic Quality Management.” In

J. Smart (ed.), Higher Education: Handbook of Theory and Research, vol. VII. New York: Agathon

Press.

Dill, D.D. (1997). “Higher Education Markets and Public Policy.” Higher Education Policy 10, 167–

186.

Dill, D.D. (1999a). “Academic Accountability and University Adaptation: The Architecture of an Aca-

demic Learning Organization.” Higher Education 38, 127–154.

Dill, D.D. (1999b). “Student Learning and Academic Choice: The Rule of Coherence.” In J. Brennan,

J. Fedrowitz, M. Huber, and T. Shah (eds.), What Kind of University? International Perspectives

on Knowledge, Participation and Governance. Buckingham: Open University Press and Society

for Research into Higher Education.

Dill, D.D. (2000) “Is There an Academic Audit in Your Future?: Reforming Quality Assurance in US

Higher Education.” Change Magazine 32(4), 34–41.

Ewell, P. (1988). “Outcomes, Assessment, and Academic Improvement: In Search of Usable Knowl-

edge.” In J. Smart (ed.), Higher Education Handbook of Theory and Research, vol. IV . New York: Agathon Press.

Fairweather, J. (1996). Faculty Work and Public Trust: Restoring the Value of Teaching and Public

Service in American Academic Life. Boston: Allyn & Bacon.

Getz, M. and J.J. Siegfried (1991). “Costs and Productivity in American Colleges and Universities.”

In C.T. Clotfelter, R.G. Ehrenberg, M. Getz, and J.J. Siegfried, Economic Challenges in Higher

Education. Chicago: University of Chicago Press.

Goertz, M.E., R.E. Floden, and J. O’Day (1995). Studies of Education Reform: Systemic Reform.

New Brunswick, NJ: Consortium for Policy research in Education, Rutgers University.




Gormley, W.T. Jr. and D.L. Weimer (1999). Organizational Report Cards. Cambridge. MA: Harvard

University Press.

Grindle, M.S. (ed.) (1997a). Getting Good Government: Capacity Building in the Public Sector of

Developing Countries. Cambridge, Mass.: Harvard University Press.

Grindle, M.S. (1997b). “The Good Government Imperative: Human Resources, Organizations, and

Institutions.” In M.S. Grindle (ed.), Getting Good Government: Capacity Building in the Public

Sector of Developing Countries. Cambridge, Mass.: Harvard University Press.

Harcleroad, F. (1976). Educational Auditing and Accountability . Washington DC: The Council on

Postsecondary Accreditation.

Higher Education Quality Council (HEQC) (1993). Notes forthe Guidance of Auditors. London: HEQC.

Higher Education Quality Council (HEQC) (1994). Learning from Audit . London: HEQC.

Honadle, B.W. (1986). “Defining and Doing Capacity Building: Perspectives and Experiences.” In

B.W. Honadle and A.M. Howitt (eds.), Perspectives on Management Capacity Building. Albany,

NY: State University of New York Press, 1986.

Honadle, B.W. and A.M. Howitt (eds.) (1986). Perspectives on Management Capacity Building. Al-bany, NY: State University of New York Press.

Horn, M.J. (1995). The Political Economy of Public Administration: Institutional Choice in the Public

Sector . Cambridge: Cambridge University Press.

Jacobs, B. and D.L. Weimer (1986). “Inducing Capacity Building: The Role of the External Change

Agent.” In B.W. Honadle and A.M. Howitt (eds.), Perspectives on Management Capacity Building.

Albany, NY: State University of New York Press.

James, E. (1978). “Product Mix and Cost Disaggregation: A Reinterpretation of the Economics of

Higher Education.” Journal of Human Resources 13, 157–186.

James, E. (1986). “Cross Subsidization in Higher Education: Does it Pervert Private Choice and

Public Policy?” In D. Levy (ed.), Private Education: Studies in Choice and Public Policy . New

York: Oxford University Press.

James, E. (1990). “Decision Processes and Priorities in Higher Education.” In S.A. Hoenack and

E.L. Collins (eds.), The Economics of American Universities: Management, Operations, and Fiscal

Environment . Albany, NY: State University of New York Press.

James, E. and Neuberger, E. (1981). “The University Department as a Nonprofit Labor Cooperative.”Public Choice 36, 585–612.

Kettl, D. (1997). “The Global Revolution in Public Management: Driving Themes, Missing Links.”

Journal of Public Policy Analysis and Management 16, 446–462.

Lattuca, L.R. and J.S. Stark (1994). “Will Disciplinary Perspectives Impede Curricular Reform?”

Journal of Higher Education 65, 401–426.

Massy, W.F. (1996). “Productivity Issues in Higher Education.” In W.F. Massy (ed.) Resource Alloca-

tion in Higher Education. Ann Arbor: University of Michigan Press.

Massy, W.F. (1997a). “Teaching and Learning Quality-Process Review: The Hong Kong Programme.”

Quality in Higher Education 3, 249–262.

Massy, W.F. (1997b). “Life on the Wired Campus: How Information Technology Will Shape Institu-

tionalFutures.” In D.C. Oblinger and S.C. Rush (eds.), The Learning Revolution. Bolton, MA: Anker

Publishing Company.

Massy, W.F. and N. French (1997). “Teaching and Learning Quality Process Review: A Review of

the Hong Kong Programme.” Paper presented at the INQAAHE Conference, Berg-en-Dl, South

Africa, May.Massy, W.F. and N. French (1999). “Teaching and Learning Quality Process Review: What Has the

Programme Achieved in Hong Kong.” Paper Presented at the International Network for Quality

Assurance Agencies in Higher Education (INQAAHE) Meeting, Santiago, Chile, 2–5 May.

Massy, W.F., N.G. French, and Q. Thompson (1999). “Management Reviews: An Outline of the Hong

Kong Program.” Paper Presented at the International Network for Quality Assurance Agencies in

Higher Education (INQAAHE) Meeting, Santiago, Chile, 2–5 May.

Massy, W.F., A.K. Wilger, and C. Colbeck (1994). “Overcoming Hollowed Collegiality.” Change 26,

10–20.



234 DILL

McDonald, L.M. and R.F. Elmore (1987). “Getting the Job Done: Alternative policy Instruments.”

Educational Evaluation and Policy Analysis 9, 133–152.

Meade, P. and Woodhouse, D. (2000) “Evaluating the Effectiveness of the New Zealand Academic

audit Unit: Review and Outcomes.” Quality in Higher Education 6, 19–30.

National Agency for Higher Education (1996). The National Quality Audit of Higher Education in

Sweden. Stockholm: National Agency for Higher Education.

Nilsson, K.-A. and S. Wahlen (2000). “Institutional Response to the Swedish Model of Quality As-

surance”. Quality in Higher Education 6, 7–18.

North, D. (1990). Institutions, Institutional Change, and Economic Performance. Cambridge: Cam-

bridge University Press. ¨ Ostling, M. (1997). “Self-Evaluation for Audit—A Tool for Institutional Improvement?” Paper pre-

sented at the INQAAHE Conference, Berg-en-Dl, South Africa, May.

Pascarella, E.T. and P.T. Terenzini (1991). How College Affects Students: Findings and Insights From

Twenty Years of Research. San Francisco: Jossey-Bass.

Quality Assurance Agency for Higher Education (1998). Quality Assurance in UK Higher Education: A Brief Guide. Gloucester, UK: Quality Assurance Agency for Higher Education.

Rosovsky, H. and I.-L. Ameer (1998). “A Neglected Topic: Professional Conduct of College and

University Teachers.” In W.G. Bowen and H.T. Shapiro (eds.), Universities and Their Leadership.

Princeton: Princeton University Press.

Scheele, J.P., P.A.M. Maassen, and D.F. Westerheijden (eds.) (1998). To Be Continued Follow-up of

Quality Assurance in Higher Education. Maarssen, The Netherlands: Elsevier/de Tijdstoom.

Shulman, L. S. (1993). “Teaching as Community Property: Putting an End to Pedagogical Solitude.”

Change 25, 6–7.

Stensaker, B. (1999). “External Quality Auditing in Sweden: Are Departments Affected?” Higher

Education Quarterly 53, 353–368.

Teacher Education Accreditation Council (TEAC) (1998). Prospectus. Washington DC: TEAC.

University Grants Committee (1997). “Report on TLQPR Seminar Held on 8 April 1997.” Hong Kong:

University Grants Committee.

Williams, G.L. (1995). “The “Marketization” of Higher Education: Reforms and Potential Reforms in

Higher Education Finance.” In D.D. Dill and B. Sporn (eds.), Emerging Patterns of Social Demand and University Reform: Through a Glass Darkly . Oxford: Pergamon Press.

Williams, P. (1992). Annual Report of the Director 1990/91: Academic Audit Unit . London: CVCP.

Woodhouse, D. (1995a). Audit Manual: Handbook for Institutions and Members of Audit Panels .

Wellington, New Zealand: Universities Academic Audit Unit.

Woodhouse, D. (1995b). Institutional Responses to Quality Assessment: Qualifications and Quality

in New Zealand . Paris: OECD.

Woodhouse, D. (1998). “Auditing Research and the Research/Teaching Nexus.” New Zealand Jour-

nal of Educational Studies 33, 39–53.

David D. Dill is Professor of Public Policy Analysis and Education at the University of North Car-

olina at Chapel Hill. He has published widely on policy issues in higher education. His recent books

include: with Marvin Peterson and Lisa Mets, Planning and Management for a Changing Environ-

ment: A Handbook on Redesigning Postsecondary Institutions (Jossey-Bass, 1997); with Barbara

Sporn, Emerging SocialDemands and University Reform: Through a Glass Darkly (Pergamon Press,

1995).

Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE...

Documents

Transcript of Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE...