CC/CSP/CP/12

APPLICATIONS AND LIMITATIONS OF COST-BENEFIT ANALYSIS

AS APPLIED TO CULTURAL DEVELOPMENT

by

Xavier DUPUIS

1985

This study has been prepared by Mr Xavier DUPUIS at the request of Unesco. The opinions expressed therein are those of its author and do not necessarily reflect the viewpoint of the Organization.

(i)

CONTENTS

Page

FOREWORD iii

PART I: THE BASIC THEORY AND METHODOLOGY OF COST-BENEFIT ANALYSIS 1

k

PART II: APPLICATIONS OF COST-BENEFIT ANALYSIS IN THE FIELD OF

CULTURAL DEVELOPMENT 7

PART III: IMPACT STUDIES, A NEW EVALUATION TECHNIQUE FOR CULTURE 19

OTHER POSSIBILITIES 27

ANNEXES 35

BIBLIOGRAPHY 45

INDEX 49

Í

(iü)

FOREWORD

It is only recently that economists have begun to interest themselves in the cultural sector, the first serious studies not having appeared until the mid-sixties.-'- For nearly 15 years research focused on specific problems: the chronic shortage of live shows because of the productivity gap and alternative sources of finance (subsidies/patronage). It was only towards the end of the last decade that new developments were envisaged using the tools provided by micro-economics.2 The desire to determine the economic weight of cultural activities derives from that line of thinking and the explanation is simple.

In 1978 budget austerity in the United States generated the gloomiest forebodings about the future. In September a conference attended by the mayors of several American cities found that all too often in times of budgetary restraint money allocated to the arts was considered an unnecessary luxury whereas in reality it was an investment and not pointless spending that could easily be dispensed with.3 Thus it was that the idea of a strategy of self-defence vis-à-vis the authorities that would express the impact of spending in favour of culture in economic terms took stronger and stronger root.

Within that strategy the application of cost-benefit analysis seemed a natural consequence: it is a favoured tool of public sector decision-makers and is based on strict economic mathematics. It is a sophisticated method presenting, for all to see, the criteria of scientific truth, coherence and objectivity that lend conviction to a pragmatic approach carrying the weight that is necessary in negotiations with authorities operating a policy of budgetary restraint.

Yet, as we shall see, using cost-benefit analysis to evaluate a cultural project poses many questions, all - or practically all - of which are serious obstacles so that it even has to be questioned whether the very principle on which it is based (that of monetary value) is not an insuperable obstacle.

This explains why economists have now moved on from cost-benefit analysis to another form of evaluation - the impact study. In spite of its qualities, however, this technique does not solve all the problems either and so other, longer-term, approaches have to be considered: cost-effectiveness analysis, tables of indicators, models, etc.

1. Authors regularly refer to William BAUMÖL and William BOWEN, Performing Arts: The Economic Dilemma. Twentieth Century Fund, M.I.T. Press, New York, 1966.

2. Mainly the following: Gary S. BECKER, Human Capitalf N.B.E.R. 2nd ed., 1975 Gary S. BECKER, A Theory of Allocation of Time, The Economic Journal, Vol. 75, 1965, pp. 494-517.

Kelvin LANCASTER, Consumer Demand: A New Approach. New York: Columbia University Press, 1971.

3. The Taxpayer's Revolt and the Arts, a United States Conference of Mayors Position Paper, Washington, September 1978.

- 1 -

PART I

THE BASIC THEORY AND METHODOLOGY OF COST-BENEFIT ANALYSIS

Cost-benefit analysis is the application of neo-classical economic calculus to the fields of public decision-making. It is therefore based on micro-economic analysis theory and on the general principle of expressing costs and benefits in monetary terms. It therefore implies the inclusion not only of all the financial factors but also of monetary equivalents (by monetary simulation) of all the other features of the project being evaluated.

Its purpose is to assist public decision-making, not in terms of producing the ideal project but simply by proposing the optimum solution for the community out of the spectrum of possibilities. The objective, therefore, is to determine optimum quantities as a contribution to decision-making or to evaluate the effectiveness of decisions already taken. Because of its paradigms and systematic use of monetary units, cost-benefit analysis holds a priviledged position in the 'rationalization of budgetary choices' technique where it represents the end-point of orthodox public sector economics in direct line of descent from the work of SAMUELSON1 and MUSGRAVE2.

The validity of cost-benefit analysis is therefore grounded on bold and restrictive assumptions making for a normative approach to public decision-making. The government is assumed to do what the theory says it should and to comply with the rules of economic behaviour so defined, acting as a rational agent maximizing utility functions under various constraints -which is why it is important to begin with an explanation of the basic theory of cost-benefit analysis.

I. THE PURSUIT OF OPTIMIZATION: THE SURPLUS THEORY

The large majority of economists have always been against government intervention. From Adam SMITH who used the word irresponsibility to John Maynard KEYNES who took cover behind the 'general interest' concept, they all had misgivings about the State and it is no exaggeration that there is a veritable tradition on the subject, expressed in most cases by censure and condemnation in the name of market optimality.

Economic thinking generally, therefore, only allows the government the right to intervene in cases where the market fails to secure the optimum as given by PARETO,3 i.e. the best theoretical definition of the maximization of welfare. There are four such cases:

monopoly situations;

those where marginal costs tend to fall;

those where there are external effects; and

the production of public goods.

Even then, the action of the State must continue to obey the criteria of market efficiency, it has to lead to the optimum.

1. SAMUELSON, The Theory of Public Expenditurer The Review of Economics and Statistics, 1954.

2. MUSGRAVE, A Theory of Public Finance. MacGraw Hill, New York, 1967. 3. Vilfredo PARETO, Manuale di Economia Politica, 1906.

- 2 -

To achieve this result, economists first attempted to work out a general rule for decision-making that would enable this Paretian optimum to be achieved regardless of the problem at issue. This approach however, known as the 'theory of constitutions', came up against an impasse with the work of Kenneth ARROW1 and today the only object is to propose the optimum solution to public decision-makers on a case by case basis2. From this concern arose 'rationalization of budgetary choices' and first and foremost cost-benefit analysis which is based, at the theoretical level, on the concept of surplus.

The surplus theory is a way of defining positive and/or negative variations in welfare in each individual case as a result of putting a decision into effect. The surpluses of all the agents concerned are then aggregated in order to find out whether the net benefit from the decision is positive or negative. If the decision-maker has to choose among several projects, the one recommended will be that which maximizes the net bottom line at the collective level.

In a free market situation the collective surplus (equal to the sum of the producer's surplus and the consumer's surplus) is maximized by the equilibrium price. In graph No. 1:

0 = supply curve

D = demand curve

Q = quantity supplied to the market

P = price of product

triangle ADC = producer's surplus

triangle BDC = consumer's surplus

triangle ABC = collective surplus

P *

B

A

optimum Q

Graph No. 1

The existence of the consumer's surplus is explained by the general law of diminishing marginal utility. Up to the Pareto-optimal level, the utility

1. Kenneth ARROW, The Theory of competitive equilibrium. Lectures at Northwestern University, 1962.

2. MUSGRAVE, Cost-Benefit Analysis and the Theory of Public Finance, Journal of Economic Literature. Vol. 7, No. 3, 1969, pp. 797-806.

- 3 -

of one unit of the product is higher than the price actually paid since the market price only represents the utility of the last unit put on the market. The surplus for each individual is therefore defined as the difference between what that individual is prepared to pay (i.e. utility expressed in money terms) to acquire a given quantity and what he actually does pay. Expressing utility in terms of money is therefore the logical consequence of applying the surplus concept.

So cost-benefit analysis means working out the surpluses for all the agents involved, making an exhaustive list of these surpluses and then calculating the net collective surplus - representing the evaluation in money terms of the project on which a decision has to be made. This is the strict application of the theory. But the latter is based on extremely restrictive assumptions which constitute so many limitations. There are three of these assumptions:

The first is that all the factors to be taken into account have to be quantifiable.

The second is that costs and benefits have to be evaluated by the same system of quantification.

The third and last is that it must be possible to aggregate, In other words the quantification system has to be the same for all individuals.

As can be seen, these assumptions are highly restrictive and adopting them implies various kinds of bias that need to be pointed out from the start.

The principle of aggregation, for instance, implies highly significant indigenous limitations because it will apply to:

values: a real or fictitious monetary value is attributed regardless of the nature of the effect (quantitative or qualitative, negative or positive);

groups: regardless of the social group concerned the hierarchy of values is assumed to be the same for all;

time: the discounting process assumes that the time horizon can be known, in other words that it is possible to determine the duration of the effects and to place a value on that duration. So, by applying a rate of discount for that purpose the preferences of future generations are assumed to be the same as those of today's.

This explains why, in spite of its formal coherence, the basic theory of cost-benefit analysis prompts certain reservations and these, which we have looked at here in the general context, become even more serious as soon as cost-benefit analysis is applied to specific fields; as we shall see, the cultural sector is no doubt one of those where the problems that arise prove to be the most difficult to get round.

II. METHODOLOGY OF COST-BENEFIT ANALYSIS

Monetary valuation techniques vary but they all take the market as their yardstick, i.e. referring constantly and directly to market price, or else determine a fictitious price using various indirect methods.

\

- 4 -

There are three methods for the direct valuation of a cost or benefit:

the value may be based on that of an alternative good or service available on the market;

it may be deduced from the value of the complementary private good or service, which enables the intensity of demand for the public good to be measured in terms of the amount of the cash costs relating to its consumption;

it may be inferred from that of an associated product that exists on the market.

In some cases, no direct valuation method works. Then recourse is had to expedients such as the values established by the courts for certain types of damages. It should be noted here that the courts also refer to the market because judges always implicitly use market yardsticks to determine the amount of compensation to award. In most cases where reference to the market is not possible or unsuitable the 'willingness to pay' method is used.

Willingness to pay is a particularly useful method for measuring effects with no market value. Benefits are valued in terms of the sum that users would be prepared to pay in order to continue to have the use of the good or service concerned. Negative effects are valued in terms of the sum that individuals would be prepared to pay to do away with the good or service causing them.

Lastly economists use another device - 'price equivalent measure' - which may be regarded as the opposite of the willingness to pay. This consists in valuing a benefit as the sum that the users of a good or service feel would compensate for its disappearance and a 'cost* by the amount that would compensate for its continuance.

These two methods are based on statistical techniques for sounding the opinion of target populations. The problems they raise are considerable: over-evaluation with the 'price equivalent measure' method, under-estimation with the 'willingness to pay' method, and so on. What is more the results clearly depend very much on the distribution of income. All other things being equal, it is certain that individuals do not all react in the same way faced with a possible compensation or payment. In the present case, the supposition of the same marginal utility of money for all is completely unrealistic.

By way of illustration and to provide a summarized picture of the various methods of valuation that have been used, the following table lists a number of fields of government intervention where cost-benefit analysis has been applied.1 The headings to the columns are various market references and the figures in the columns relate to the notes that follow giving a brief description of the ways most frequently used by economists to arrive at money values.

1. For further details the reader is referred to Xavier DUPUIS & Xavier GREFFE, Rapport technique sur les méthodes de quantification du non-marchand avec une application au cas des associations d'environnement, Centre 'Travail et Société', Université de Paris IX Dauphine, 1979,

- 5 -

^~v. Footnote ^~\Number

Field ^ > .

Education

Health

Housing

Natural Resources

Transport

Recreation

Substitute products

I-I

3

6

9

12

15

Complementary products

7

10

13

16

Associated products

2

4

8

11

14

17

Other

5

Education

1. Value of the equivalent service on the private education market.

2. Value of the capitalization of the average annual variations in benefit due to the variation in the level of knowledge.

Value of the reduction in crime expressed in terms of the reduction in policing costs.

Value of the reduction in the amount of subsidy because of the reduced inequality in wages.

Health

3. Value of the equivalent service on the health market.

4. Value of the capitalization of the benefits due to higher productivity or to the possibility of performing an activity. Reduction in later health spending that would have been necessary if the possibility of certain diseases had not been anticipated.

Value of the increase in consumption due to improved physical condition.

Housing

5. Value of tax surplus due to the improvement in housing and higher rents.

6. Value of the corresponding service on the private housing market.

7. Value of the related costs people are prepared to pay to be able to benefit from the improvement in housing conditions (transport costs, for example).

8. Value of the increase in income from rent or land in the neighbourhood of the improved area.

Value of the reduction in staff costs because of the reduced fire risk.

Value of the reduction in police costs because of the reduced need for policing associated with the reduced risk or rate of crime.

- 6 -

Natural resources

9. Price of energy, if there is a private market, or marginal cost pricing.

10. Costs people are prepared to pay in order to settle in areas improved by the building of the dam.

11. Value of the capitalization of the variation in harvest yield due to the control or elimination of flooding.

Value of the variation in land prices.

Value of the variation in economic activity in the area concerned.

Value of the saving made through not having periodically to destroy public works structures.

Value of savings made because of the lower risk of epidemics.

Transport

12. Value of the equivalent service on the private market.

13. Value of what people are prepared to pay to get to or to live near a new transport system.

14. Value of the savings in time, safety and comfort due to the introduction of a new transport system in a given area.

Recreation

15. Cost of the equivalent recreation facility on the private market (theatre, park, zoo, etc.).

16. Value of the costs that people would be prepared to meet to use a given recreation facility (cost of transport, time, etc.).

17. Value of the cost involved in reconstituting the human resource from the viewpoints of health, education, reduction of crime.

Examples of the application of cost-benefit analysis to transport and public health are given in an annex.

Cost-benefit analysis is, without any doubt, an extremely powerful weapon. Because of its indiscriminate application to all the groups concerned and the standard nature of its mathematical method, the money indicator is an extremely simple synthetic criterion for decision-making. In addition, the surplus theory gives the method the attributes of objectivity and scientific truth, surplus in this case acting as an appropriate measure of the satisfaction of the collectivity and of variations in that satisfaction.

It has to be admitted, however, that cost-benefit analysis does not rule out value judgements because of the arbitrary features of its valuation methods. Clearly, too, it is based on highly restrictive assumptions. This being so, its application to the cultural sector must inevitably be difficult. A major problem is the complex and varied nature of the field of analysis.

- 7 -

PART II

APPLICATIONS OF COST-BENEFIT ANALYSIS IN THE FIELD OF CULTURAL DEVELOPMENT

Applying cost-benefit analysis to the field of cultural development proves to be very difficult which is why no in-depth applications are to be found. Before examining the practical aspects of the technique to see what its real use in the cultural sector might be let us first take a look at the various types of cost and benefit.

I. TYPES OF COST AND BENEFIT

The different kinds of cost and benefit can be defined by almost as many different criteria as there are authors. The list we give below is based on MUSGRAVE's work! with some departures at the conceptual level. The main typse of cost and benefit are as follows:

Cost and benefits may be real or monetary.

Real costs and benefits may be:

tangible or intangible;

internal or external;

final or intermediate.

1. Monetary and real costs and benefits

This is the first and most important distinction. Monetary costs and benefits are easy to pinpoint and measure. Expressed directly in money terms they pose no quantification problems and are the basis of all economic calculations. They cover all the money invested in the cultural project being evaluated and all the monetary benefits generated by the implementation of the project. However, they give no idea of the net costs and benefits to the community; they are simply financial quantities and do not therefore represent real costs and benefits.

Real costs and benefits, for their part, are not expressed in money terms. They are real in the sense that it is they that affect the general level of satisfaction of society. Since they relate to the development of individual and social values their nature is complex. Sometimes impossible to quantify they are, in most cases, difficult at the very least even to identify clearly. In the environmental field, for example, real costs and benefits are those related to ecological potential (destruction, reconstitution, development, and so on), whilst in the field of cultural development they are related to the intellectual development of the individual and so on. This explains why a second distinction has to be drawn - that between tangible and intangible real costs and benefits.

2. Tangible and intangible real costs and benefits

Tangible costs and benefits refer to those to which a market value can be assigned, either directly or indirectly, by the various methods we have

1. Here we refer to the nomenclature defined in R.A. MUSGRAVE & P. MUSGRAVE, Public Finance in Theory and Practicef MacGraw Hill, 3rd edition, New York, 1980.

- 8 -

described. Intangible costs and benefits are those for which a market valuation proves impossible or is completely meaningless. Unfortunately most real costs and benefits fall into this category. Whereas intangible costs are frequent in environmental matters for example (the most important being cases of irreversibility, e.g. the destruction of part of the ecological heritage with no reconstitution), intangible benefits are most frequent in the cultural field (intellectual enrichment, aesthetic value, the historic heritage aspect, etc.).

Even though these real intangible costs and benefits are fundamentally important, no cost-benefit analysis will include them because there is no way of giving them a money value. Bound by the money value rule, the economist is powerless in the face of this problem.

3. Internal and external real costs and benefits

The concept of externality (one of the most important in public sector economics) enables another distinction to be drawn: that between internal and external real costs and benefits. It is a concept that reigns over all economic theory concerned with welfare and it is central to the argument about how to handle the fields where the qualitative aspect is paramount (culture, health, environment, etc.).

There is no precise definition of the concept of externality but normal practice is to use William BAUMÖL's approach1 according to which an externality is considered to exist where there is interdependence but no compensation.

A (positive or negative) external effect may then be expressed as a (positive or negative) variation in the level of utility enjoyed by an agent as a result of an action by another agent without this effect being assigned a market value.

Internal costs and benefits are those directly related to the implementation of the project whereas external costs and benefits are all those inferred by the project. Their exhaustive identification obviously poses vast problems. In actual fact a kind of boundary has to be set; every economic system consists of a set of interrelationships and no action at whatever level is neutral upstream or downstream.

Identifying external effects is extremely difficult. It is not always easy to make due allowance within the economic structures of production but the analysis has to be contained within specific space and time boundaries. The fact is that the implications of projects in practically every case concern far larger areas and populations than those under the direct responsibility of the authorities concerned. This ripple effect raises another boundary problem.

The same applies to the time factor. The various implications of a project may vary in the time they take to appear or disappear. A number of graphs showing how, in space research, the effects do not all develop at the same pace are given in an annex.2 For cost-benefit analysis, therefore, it is important to define a time boundary and to know how to allow for differing rates of propagation for the effects of the project. This will be increasingly difficult the more one is dealing with final costs and/or benefits.

1. W.J. BAUMÖL, Welfare Economics and the Theory of the State. 2nd edition, Bell and Sons, London, 1965.

2. Pages 65-70.

- 9 -

4. Final and intermediate real costs and benefits

At the methodological level this distinction is important. A cost or benefit is 'final' if it is borne by or directly benefits the end consumer. An intermediate cost or benefit arises at the level of the production of other goods or services and will therefore affect the welfare of consumers (measured in terms of surplus) only in an indirect manner. The difficulties of identification and quantification are not so great in this latter case because there are no doubt fewer intangible elements but they are nevertheless considerable. For final costs and benefits they are in most cases insurmountable.

This catalogue of the various types of cost and benefit shows that cost-benefit analysis must inevitably be incomplete. The existence of intangible factors and boundary problems irremediably route cost-benefit analysis towards a truncated and biassed conclusion. At each stage it is a question of making the best of things, i.e. isolating what is quantifiable from what is not. Then monetary values have to be worked out for each cost and benefit. Aggregating these will make it possible to calculate the surplus for the agents affected by the project first at the individual level and finally, after further aggregation, at the collective level.

The following chart gives the essential steps in this process in summary form. The limitations, for each type of cost and benefit, appear clearly at each stage and answer the question why examples of the direct application of cost-benefit analysis are so rare in the field of cultural development.

II. EXAMPLES OF THE APPLICATION OF COST-BENEFIT ANALYSIS IN THE FIELD OF CULTURAL DEVELOPMENT

It is not surprising therefore that so few attempts at using this sophisticated method of evaluation are to be found. As we have seen, it is very difficult to use cost-benefit analysis in fields in which intangible costs and benefits figure so largely. The complexity and reductionism that it involves are, in many respects, irremediable defects. The only examples we have been able to find of any value and credibility are those for:

recreation facilities (natural and amusement parks, etc.),-'-

educational projects, and

exhibitions of the plastic arts.2

New applications, more particularly concerned with art facilities^ and historic monuments^ have been reported in recent research.

1. Cf. M. CLAWSON & J.L. KNETSCH, Economics of Outdoor Recreation, John Hopkins Press for Resources for the Future, 1966 and: R. KALTER, The Economics of Water-based Outdoor Recreation: A Survey and Critiquef

United States Army Engineer Institute for Water Resources, Report No. 71-78, National Technical Information Service of the United States Department of Commerce, Washington, D.C., 1971.

2. E.W. HANTON & W.S. HENDON, The Akron Art Institute Study, Akron, Center for Urban Studies, The University of Akron, 1973, Volume III.

3. CD. THROSBY, Social and Economic Benefits from Regional Investment in Art Facilities: Theory and Application, Journal of Cultural Economics. Vol. 6, No. 1, June 1982, pp. 1-14.

4. William S. HENDON, Benefits and Costs of Historic Preservation, The University of Akron, 1983.

- 10 -

COSTS AND BENEFITS

MONETARY

*-l AGGREGATION

CALCULATION OF SURPLUS FOR EVERY AGENT

AGGREGATION

CALCULATION OF NET COLLECTIVE SURPLUS

REAL

j TANGIBLE INTANGIBLE

INTERNAL OR EXTERNAL

FINAL OR INTERMEDIATE

>\ MONETARY VALUATION |<-

- 11 -

The educational sector provides a very good illustration of what can be done. With no claim to exhaustivity, the main costs and benefits of a project may be listed in the form of a simple table:

Benefits Costs

Tangible Future increase in income

Loss of present income

Reduction in crime

Cost of facilities

Reduced prevention costs

Operating costs (wages, etc.)

Intangible Intellectual enrichment

Opportunity cost of leisure

Various studies have used the method for surplus calculation but many aspects have still not got beyond the research stage (the 'rate of return1

question in particular).

The methodology developed for outdoor recreation facilities and youth centres^ also provides pointers for the application of cost-benefit analysis to cultural development projects. Their results, however, seem greatly dependent on the assumptions made with regard to duration and rate of discount.2

To take one of these examples, Maragret 0'SHEA's analysis of the Mildura art centre in Australia suggests that, at an annual rate of return of 8.5 per cent, the running of the centre is generating a net surplus of 150,000 Australian dollars a year. However, this study, though unique from many points of view, should be read with great caution. The analysis is incomplete since the nature of the costs and benefits used is very limited at the level of their definition and valuation. Though without doubt a stimulating and highly interesting stylistic exercise this cost-benefit analysis is not in conformity with the underlying micro-economic theory.

In the field of historic heritage, cost-benefit analysis is still in the research stage rather than that of direct application. Consumer surplus can be calculated using a demand curve derived from the CLAWSON and KNETSCH model (so the methodology would stem from the work down on the environment and local development). A distinction is made between primary and secondary benefits, a large number of the latter being intangible. The following list is based on that proposed by W.S. HENDON:3

Primary benefits:

induced personal income;

consumer surplus;

2.

3.

J.M. KANANAGH, Programmatic Benefit-Cost Analysis of Local Youth Recreation Programmesr Institute of Government and Public Affairs, U.C.L.A., 1968. W.S. HENDON, Evaluation of Urban Parks and Recreation. Praeger, New York, 1981. Taken from the article already mentioned.

- 12 -

value of free admissions;

donations and subsidies .

Secondary benefits:

social value of the public good;

induced economic effects (tourism);

increase in tax revenue;

stimulation of private investment;

reduction in crime costs;

reduction in opportunity costs.

As for costs, these obviously include all the expenditures involved (purchase, catering, development, maintenance, administration, etc.) but they have to be considered at the level of their adjustment and valuation in the light of the risk factors (for which games theory could be used), opportunity costs and long-term social costs. Here again we come up against the question of knowing what time horizon and, consequently, what rates of discount to apply.

The obstacles are therefore manifold and the applications not very convincing - good reasons for questionning the soundness of cost-benefit analysis as applied to cultural development and to look at it again in the framework of 'rationalization of budgetary choices*.

III. THE VALIDITY OF THE APPLICATION OF COST-BENEFIT ANALYSIS TO CULTURAL DEVELOPMENT

The first question that needs addressing is the definition of cultural development itself. This is an old issue to which there is no satisfactory answer. Cultural development is a collective function whose links with the education and recreation functions are evident but difficult to define. As the French Commissariat General du Plan recognized in 1970i1

'Those who try to define the cultural development function oscillate between a very restrictive conception of culture as a heritage to be preserved and a very wide conception according to which everything which is not strictly economic and utilitarian is called cultural. In both cases the search for indicators of results cannot fail to be affected because it becomes either extremely easy but not very meaningful or else discouragingly complex.

The search for a middle way could lead to the adoption of four purposes for the cultural development function:

the defence and portrayal of the existing cultural heritage,

additions to this heritage by new aesthetic creations,

the practice of cultural activities by individuals, and

the consumption of culture by individuals,

1. Note sur l'utilisation du mode d'approche R.C/B. dans l'analyse des fonctions collectives oummissariat Général du Plan, Paris, January 1970.

- 13 -

it being understood that there may be some overlapping: achievements under the second heading, for example, will extend the field of action of the first (...)•

It should be noted that cultural is here defined by reference to the notion of 'beautiful', thus leaving aside everything involved in the socialization process of the members of the community which is more the task of the education function. But one could perfectly well conceive of a single function embracing both cognitive and aesthetic aspects of the development of the personality.'

Though written 15 years ago these lucid judgements, pessimistic though they are with regard to practical applications, are still relevant. At the conceptual level, no noteworthy progress has been made. This being so, the use of cost-benefit analysis appears more doubtful than at first it seemed reasonable to hope.

1. The limitations of cost-benefit analysis as applied to cultural development

The main limitation resides in the very principle of the method, i.e. monetary evaluation. As we have seen, the economist has to convert everything to money values which, directly or indirectly, means using market prices. But, in fact, not all costs and benefits can be estimated in terms of market prices and in such cases indirect simulation methods are used in which arbitrary treatment and therefore varyingly explicit value judgements are inevitable.

Monetary valuation also implies complete homogenization. All costs and benefits, once expressed in money terms, are comparable and at the conclusion of the analysis the surplus arrived at conceals its real components: there is no going back. This means that specificity of any kind is erased. No trade-off is possible except in favour of the project showing the maximum collective net surplus. No qualitative dividing line can be drawn between the surplus produced by a cultural project and that produced by an industrial project.

Apart from this chronic eclipse of the qualitative dimension to the sole advantage of the quantitative dimension many 'intangible' costs and benefits cannot be taken into account even though they are probably the most important. Thus cost-benefit analysis gives no more than a truncated view of the real implementation of a project

In addition to these intangible elements, other essential factors are either neglected or even omitted in the economic evaluation. It is clear for example that - aside from any quantification or money value problems - no cost-benefit analysis can claim to identify all costs and benefits. Then there are 'boundary' problems: the economic boundary to determining positive and negative external effects in all their complexity, the geographical boundary and lastly the time boundary. The assumption of stability of preference over time is very rigid in this respect and there can be no scientific criterion for the choice of rate of discount (5, 7, 10 per cent, etc.) or period covered (15, 20, 30 years, etc.).

Cost-benefit analysis, derived directly from micro-economic theory, is therefore dependent on its postulates and assumptions. Having to use a strict economic calculus it has to be restrictive and standardizing - features not always offset by perfect objectivity. The theoretical and methodological difficulties soon become serious and even insoluble when attempting to apply the technique to fields as specific, complex and heterogeneous as cultural development. This explains the relative paucity of research in this area.

- 14 -

Whilst there is no intention of promoting the idea that cost-benefit analysis is not applicable in this case, it has to be recognized that the method needs many refinements before it can be operational. This is a question of research but, over and above that requirement, cost-benefit analysis needs to be returned to its proper place in 'rationalization of budgetary choices'. It is only in this general context that the question of its validity can be addressed.

2. 'Rationalization of budgetary choices' and cultural development

The French 'rationalization of budgetary choices' concept may be defined as scientific and extensive research into all procedures and methods designed to improve the suitability, coherence and effectiveness of public decisions for which budgetary resources are provided or whose effects have an impact on the budget. Similar systems elsewhere are PPBS (programme, planning and budgeting system), output budgeting, program budgeting, PAR (program analysis and review), etc. Thus rationalization of budgetary choices is an all-embracing process in which cost-benefit analysis is just one tool among others.

In theory, analysis in the rationalization of resource allocation implies several successive phases of research:

analysis of the structure of the objectives of cultural development,

description of the various actions in the cultural development field,

choice of priorities among the various actions in the light of the objectives and the cost and effectiveness of each,

choice of ways and means of implementing the selected actions,

definition of the level of déconcentration in the setting of objectives, and

verification of the effectiveness of the actions undertaken at the central level and at the level of the various establishments and bodies concerned.

The process involved in rationalization of budgetary choices is not, therefore, simply integrative but also iterative as the two block charts that follow show. Alongside the sequence at the level of concepts there is a corresponding sequence in terms of the techniques that can be used. That being so, the importance of cost-benefit analysis is automatically relativised and a far broader field of possibilities emerges.

But between theory and practice there is a wide gap. What is more, returning cost-benefit analysis to its more general context of rationalization of budgetary choices makes clear its real scope. Rationalization of budgetary choices is a process with its own logic and its own purpose and cost-benefit analysis stems from that logic and that purpose. Trying to use it to measure the impact of cultural projects puts the economist in a difficult position.

There are two fundamental differences between rationalization practice and evaluation studies. Firstly rationalization of budgetary choices is an ex ante process whereas most evaluations are ex post. Secondly rationalization methodology gives precedence to the normative approach which is based, as we have seen, on the principles of econometrics whereas evaluation uses a deliberately empirical approach aimed at measuring and explaining the effects of decisions that have been taken.

- 15 -

THE SEQUENCE IN TERMS OF CONCEPTS

End-purpose

Objectives

Structure of

objectives

Indicators of

objectives

Means, Action

Product

Structure of

programmes

Indicators of

objectives

Benefit

Cost-effects

Target groups or

groups concerned

Structure of programme

objectives

Effectiveness

- 16 -

THE SEQUENCE IN TERMS OF TECHNIQUES

Formulation

Modelling techniques:

- Cost-benefit

- Simulation

Description techniques:

- Morphological analysis

- Delphi method

- Graph method

Management techniques:

- Authorization

- Analytical

accounting

- Management review

Comparision of objectives

and programmes

Evaluation techniques:

- Cost-effectiveness analysis

- Cost-benefit analysis

- Multi-criteria analysis

- Discounting

Choice of programmes

and budgetary impact

Management

- 17 -

Today it is an unquestionable fact that rationalization of budgetary choices is used very little if at all (like the American PPBS). The reasons are simple. For one thing, it has become cruelly clear that there is little understanding of the real processes by which the authorities take their decisions and act upon them. For another, trying to impose the normative economic model on government has led to many deadlocks in which bureaucratic logic has won the day over alleged rationality. Events show that rationalization of budgetary choices is not a winner.

This explains why there can be little surprise that cost-benefit analysis should not be the method that economists prefer to apply to the cultural sector. Taking a more pragmatic, or at least less ambitious, tack they have turned for some years now to a method of assessment known as the 'impact study'. Although this technique takes the opposite logic to that of cost-benefit analysis the two are obviously related. So here we again find many of the limitations we found with cost-benefit analysis. Nevertheless, developments with impact studies deserve consideration even though, as might be imagined, other methods of evaluation also need to be envisaged.

- 19 -

PART III

IMPACT STUDIES, A NEW EVALUATION TECHNIQUE FOR CULTURE

Whilst cost-benefit analysis is a micro-economic approach, impact studies are based on the multiplier concept and address cultural development from the macro-economic angle. It is no longer a matter of calculating the various economic implications at the level of each individual but of bringing out the main factors and flows generated at the level of a local economy.

By comparison with cost-benefit analysis, impact studies are a simplification: the objective is more limited because here the aim is to assess the impact of one or more cultural activities over a given geographical area, generally for a period of one year and leaving costs out of account. An impact study may therefore be likened to a pseudo cost-benefit analysis only concerned with the benefits. But the analogy should not be taken too far; that would be a serious theoretical and methodological mistake. What we have here is a genuine method of evaluation which is far less costly to carry out than a cost-benefit analysis but has the advantage of producing results that are no less relevant.

I. THE IMPACT STUDY PROBLEM

What is meant by impact? Here, of course, we find the same distinctions as we found in cost-benefit analysis and our classification divides impacts into two types, each of these subdivided into two subtypes.

The first type of impact corresponds to a restricted perception of the idea and relates to the monetary contribution (the 'dollars and cents' to use the American expression) that culture makes to economic development. This type of impact consists of:

the direct impact: i.e. all the economic flows generated by the cultural activity concerned, and

the indirect impact: i.e. all the induced effects. This is where a number of multipliers are introduced whose choice is, obviously, a matter of strategy (we shall return to this question when discussing methodology).

The second type of impact is defined by the extension of the concept and covers all the repercussions on society of putting a cultural project into effect. The subdivision here is between the impact on the economic system in the strict sense (improvement in knowledge, capacity to innovate, pro­ductivity, etc.) and that on the social system (intellectual enrichment, conservation and development of the historic and cultural heritage, the stimulus to creativity, improvement in the quality of life, less crime, etc.).

It is clear that, whereas the former type of impact - with varying degrees of difficulty - is quantifiable, the latter is not for it comprises the intangible components already referred to in the discussion of cost-benefit analysis.^ Impact studies therefore are confined to measuring direct and indirect impacts, i.e. the monetary contribution of culture to the economy.

II. IMPACT STUDY METHODOLOGY

Quantifying the amounts directly invested in the cultural field presents no special problem apart from that of access to sources of information.

1. On this point see: S. GLOBERMAN, Cultural Regulation in Canada. The Institute for Research on Public Policy, Montreal, 1984.

- 20 -

Quantifying induced monetary flows, however, does pose a problem and for this economists fall back on the multiplier principle.

The idea of the multiplier is simple: every franc or dollar put into any kind of economic activity produces a multiplier effect, i.e. one induced by that economic activity on the 'snowball' principle. There remains the problem of getting the multipliers right. This is done by surveys and monographs on which general factors or 'multipliers' are based. Simplifying, these may be reduced to two:

The employment multiplier - i.e. the number of jobs induced.

The income multiplier - i.e. the cumulative effect on the income in circulation, not only personal earnings but also tax revenue accruing to the local authority. This multiplier depends largely on local income distribution patterns, consumption standards and tax systems but it also depends on market structures in terms of local versus imported products.

Multipliers can be calculated at local, regional and national level but their validity is by no means beyond question. Unfortunately the impact studies in our possession show that they are difficult to calculate and that many approximations are required, the problem being at order of magnitude level rather than in the figures as such this explains the current tendency to use 'short-hand' methodologies.1

Also called 'abridged* methods, these were developed because the full impact study, using statistical surveys and monographs, is very costly and yet, for all that, not very reliable. There is nevertheless a real need for the economic assessment of cultural development and this type of method is fairly easy to apply, even for strangers to economics and, above all, makes minimal demands as regards data input requirements.

For the moment the method has only been applied to one-off cases of non-profit live performance activities and its restrictive nature is clear from the outset. It uses simple data and is only concerned with the costs associated with the 'consumption' of live performances as a way of quantifying the induced economic activity.

The following data have to be collected:

the resources of the establishments themselves;

the costs of these same establishments; and

the population of the community in the geographical area concerned.

Three 'income' multipliers are then used:

The 'audience's related costs' multiplier, i.e. the costs that spectators have to meet to go to the performance (eating out, transport, baby-sitting, etc.).

The local multiplier, i.e. the cumulative effect on income in circulation (spending by the establishments producing the show plus spectators' related costs). This is assumed to increase with the size of the population.

1. Cf. W. BAUMÖL, The impact of the Broadway Theater on the Economy of New York CitvT Mathtech, Princeton, New Jersey, 1977.

- 21 -

The national multiplier (the same as the local multiplier but at the national level).

If

R = resources of the establishments in proceeds from ticket sales

C = audience's related costs obtained by multiplying spectators' primary costs R (tickets) by Mc

Ml = local multiplier

Mn = national multiplier

we can define the following sequence:

Step 1: ticket sales - R

Step 2: related costs, C = Mc x R

Step 3: total costs involved, R + C = R(l + Mc)

Step 4: economic impact at local level, Ml (R + C)

Step 5: economic impact at national level, Mn (R + C).

Although this abridged method is obviously somewhat rough and ready and highly simplified it has its advantages - foremost among which is its very simplicity and therefore low cost, which explains the relatively large number of applications, in the United States and in Canada.1

III. EXAMPLES OF IMPACT STUDIES IN THE CULTURAL SECTOR

Some impact studies have already achieved considerable distinction, examples being the work done by David CWI and Kathryn LYALL2 and by D.R. VAUGHAN,-* the studies already referred to and the research done recently for the Port Authority of New York and New Jersey (see below). All of these concern the narrow definition of cultural development since they all focus solely on artistic activities.

On this subject see: Methode abrégée d'évaluation des répercussions économiques des arts du spectacler Conseil des Arts du Canada, Note de Recherche, March 1982, and: H. CHARTRAND, An Economie Impact Assessment of the Canadian Fine Arts, Third International Conference on Cultural Economics and Planning, Akron, April 1984. D. CWI and K. LYALL, Economic Impact of the Arts and Cultural Institutions: A Model for Assessment and a Case-Study in Baltimore, Research Division Report, National Endowment for the Arts, Washington D.C., 1977, and D. CWI, The Economic Impact of the Arts: A Study of 49 Institutions in Six U.S. SMSA's, Research Division Report, National Endowment for the Arts, Washington D.C., 1981. D.R. VAUGHAN, Does a Festival Pav? in W.S. HENDON, J. SHANAHAN and A.J. MacDONALD (eds), Economic Policy fo the Arts. Cambridge, Mass., Abt Books, 1980.

- 22 -

In a study of 49 cultural institutions in six United States towns, David CWI finds that the total direct impact in one year is nearly 69 million dollars. The indirect economic impact work out at:

129 million dollars of industrial and commercial income;

49 million dollars of personal income;

20 million dollars deposited with the banks; and

6,700 jobs.

Other, more detailed findings in the study are even more informative. One is that the impact is highly dependent on local economic structures and that there is no straightforward linear relationship between numbers of cultural institutions and scale of impact. Over 3,000 jobs were generated in Minneapolis Saint Paul compared with only 160 in Springfield though the number of institutions - 10 - is the same (the other cities studied were Columbus, San Antonio, Salt Lake City and Saint Louis). The scale and type of the activity are decisive factors.

The input to the local economy from outside is also of considerable importance. In Saint Louis, for example, the audiences at the eighth institutions total two million local residents and 500,000 from elsewhere. Whereas the former spend 3.31 dollars each, the latter spend 32 dollars when the object of the visit is a cultural activity and 157 dollars each when they come for some other reason.

A similar study on a small town (Aspen in Colorado) concludes that spectators and visitors drawn there by the music festival spend 4.7 million dollars a year.-'- if applied to such famous festivals as those at Salzburg, Bayreuth, Verona, Aix en Provence, etc., the same kind of arithmetic would presumably yield even more impressive figures as the research by D.P. VAUGHAN on the Edinburgh festival (already referred to) suggests. The same is true of all the big exhibitions (everyone remembers the extraordinary commercial success of the Tutankhamen exhibition in the United States to quote just one exammple).

The most interesting impact study that has been made so far is the recent research commissioned by the Port Authority of New York and New Jersey.2 Some of its findings are as follows:

The direct economic impact of the arts in the region comes to 5.6 billion dollars.

The arts generate over 2 billion dollars, of personal earnings and over 117,000 jobs.

The arts attract an input of some 1.6 billion dollars into the regional economy from outside.

The arts bring in nearly 150 million dollars in duty and taxes.

Industry benefits to the tune of 5.6 billion dollars of additional activity.

1. Peggy CUCITI, Economics of the Arts in the State of Colorado. Denver, University of Colorado, Center for Public-Private Co-operation, 1983.

2. The Arts as an Industry: Their Economic Importance to the N.Y. and N.J. Metropolitan Area, New York, The Port Authority, 1983.

- 23 -

Other findings could also be quoted, because this is a very full study identifying the impact for each activity, and in every case they prove that the weight of the contribution made by the artistic sector is considerable.

Since these impact studies are confined solely to the arts it is easy to imagine how much greater the contribution from the cultural sector in its broader definition would be. In that connection, the Canadian studies - though highly specific in their focus - show the importance in quantitative terms of the impact of spending on cultural activities.

Using the abridged method we have already described, three applications arrive at some particularly striking figures.

Rough estimate of the impact of the performing arts in Toronto on the local and national economies

Step 1

Tickets sold by 35 companies in 1979: 8.8 million dollars

Step 2

Audiences' related costs: 8.8 million (step 1) x 1.06 = 9.3 million dollars

Step 3

Total spending in connection with performing arts: costs of 35 companies, i.e. 21.6 million dollars plus 9.3 million dollars (step 2) = 30.9 million dollars

Step 4

Impact on Toronto economy: 30.9 million dollars (step 3) x 1.6 = 49.4 million dollars

Step 5

Impact on Canadian economy: 30.9 million dollars (step 3) x 2.1 = 64.9 million dollars

Rough estimate of the impact of the performing arts in Ontario on the national economy

Step 1

Tickets sold by 70 companies in 1979: 18.5 million dollars

Step 2

Audiences' related costs: 18.5 million (step 1) x 1.06 = 19.6 million dollars

Step 3

Total spending in connection with performing arts: costs of 70 companies, i.e. 39.6 million dollars plus 19.6 million dollars (step 2) = 59.2 million dollars

- 24 -

Step 4

Impact on National economy: 59.2 million dollars (step 3) x 1.6 = 124.3 million dollars

Rough estimate of the impact of the performing arts in Canada on the national economy

Step 1

Tickets sold by 184 companies in 1979: 37.4 million dollars

Step 2

Audiences' related costs:

37.4 million (step 1) x 1.06 = 39.6 million dollars

Step 3

Total spending in connection with performing arts: costs of 184 companies, i.e. 90.3 million dollars plus 39.6 million

dollars (step 2) = 129.9 million dollars

Step 4

Impact on national economy: 129.9 million dollars (step 3) x 1.6 = 272.8 million dollars On the basis of the above figures^ it can be argued that the arts

constitute the 11th largest sector in the Canadian economy in terms of money, the fourth in terms of employment (146,000 jobs) and the sixth in terms of wage-bill (2.3 billion dollars). In 1980 it is claimed that the arts' contribution to the Canadian economy amounted to 2.4 per cent of Canada's gross domestic product.

Whilst these impact studies with their glowing results are clearly tempting and would appear suitable for general application it would be unwise to grow too euphoric about the method. Impact studies are no miracle formula. It must not be forgotten that they are no more than a second-best solution in the absence of cost-benefit analysis. They also suffer from the same distortions and shortcomings as cost-benefit analysis. They are equally reductionist (if not more so) and in any case incapable of measuring the true economic significance of cultural development. There are problems in carrying them out and the result should not be allowed to hide the underlying assumptions and the approximations that they are based upon (particularly in the calculation of the multipliers).

Care must be taken therefore not to overestimate them, although it is even more important not to underestimate their value. Though highly imperfect tools and quite unsuitable for measuring cultural development in its full dimensions, specific characteristics and complexity, impact studies - like cost-benefit analysis - are none the less interesting initiatives which need to be developed and enriched.

On the other hand it would be wrong to reduce the question of measuring the cultural phenomenon to impact studies alone. There are other ways of

1. Taken from the paper already referred to: Methode abrégée d'évaluation des répercussions économiques...

- 25 -

organizing a strategy of defence against the budgetary restraint that the cultural sector is faced with and today it is in their direction that research and investment need to be steered.

- 27 -

OTHER POSSIBILITIES

There are four other avenues to be explored:

cost-effectiveness analysis; modelling; indicators; and policy assessment practice.

All have their strong and weak points of course but they all represent many additional possibilities.

COST-EFFECTIVENESS ANALYSIS

This is another technique used in rationalization of budgetary choices but it would appear better suited than cost-benefit analysis for application to cultural development. It has been employed with some success in the health sector, for example, which has many aspects in common with the cultural sector, but its relevance has also been recognized in other sectors including the rationalization of defence spending.

For one thing cost-effectieness analysis, in principle, avoids the problems of aggregation and valuation on which cost-benefit analysis founders. By leaving factors to be expressed in their own units and not using any system of single yardstick comparison, cost-effectiveness offers the advantage of avoiding any form of denaturization or standardization.

Its purpose is to examine the positive and negative aspects of each programme and on that basis to help decide which is the best in terms of either:

maximum positive effects for a given negative effect, or

minimum negative effects for a given positive effect.

Positive and negative effects are expressed in their own value system and not converted to money values. To that extent cost-effectiveness analysis is not simply a variant of cost-benefit analysis.

There are three essential steps in the method:

definition of the missions and of what effectiveness means in the particular case;

operational definition of the system and of the effectiveness model;

the cost model.

The missions and the definition of effectiveness used in the analysis are based on contextual variables, the object being to respect the specific features of the field of study.

The cost assigned to each system has to meet four conditions:

it has to be a forecast;

it has to be financial;

- 28 -

it has to be complete, in other words it has to include all the financial implications in both space and time;

it has to be marginal in the sense that the cost has to be confined to the financial implications of the particular programme to the exclusion of all those arising out of programmes implemented earlier.

The problem associated with the cost-effectiveness analysis method is, of course, the difficulty of comparing factors that are not directly comparable. Not departing from the real significance of costs and benefits means, by implication, that some kind of hierarchy of values must exist according to which preference goes to the project that generates those categories of benefits or minimizes those types of cost that are felt to be more important than others. So the method is not wholly immune to criticism or difficulty in application. It is nevertheless rich in potential. It is a fine-tuned analysis that harmonizes its quantitative and qualitative components. It can also be used for partial optimization and to apply it to cultural development would therefore seem a legitimate option. All it needs is for the researchers to try it out.

MODELLING

It would be wrong to expect too much of modelling, of course. Clearly cultural development can never be expressed in terms of an equation - which is reassuring, in a way. In certain specific fields, however, modelling based on the vast progress that has been made in macro-economics can throw light on the socio-economic prospects of cultural development in some highly interesting ways.

Unfortunately this is as yet an extremely tentative, not to say non-existent, research avenue. Experiments in this direction are extremely rare, the most interesting being that made in 1978 to assess the demand for Broadway Theater.1 The model tested on that occasion - NYREM (New York regional econometric model) - took many different variables into account: tourism, per capita income, price of tickets, crime rate, etc. The authors themselves declared the results to be disappointing. Nevertheless this should not deter economists from trying again with this method. At the methodology level, modelling (like cost-effectiveness analysis) poses the problem of indicators - possibly the most interesting of the various techniques.

INDICATORS

The lack of and disparity in the data available at the non-aggregated level are constant difficulties for practitioners in the cultural field. Much remains to be done before it will be possible to draw up tables of representa­tive indicators whose periodical updating could be used to measure the impact of cultural policies. A set of such indicators like an array of monitoring instruments could also help in arriving at significant ratios and in drawing up satellite accounts, and thus extending the coverage of national accounts.

The purpose of satellite accounts is to identify and quantify the reciprocal effects between developments at the macro-economic level (and

1. Harry H. KELEJIAN and William J. LAWRENCE, Estimating the Demand for Broadway Theater, in Economic Policy for the Arts, Abt Books, Cambridge, Mass., 1980.

- 29 -

therefore in production structures) and the organization of certain social activities. Their best illustration is given by the public health account drawn up by the Commission des Comptes Santé (public health audit board) in France.I Along the same lines, the classification of data by sector carried out at the request of the French Commissariat General du Plan^ also represents a useful broadening of the national accounts, its purpose being to bring together all the information available about a particular field irrespective of its source or nature - monetary, physical or demographic -whereas a satellite account is only concerned with monetary values. The method has been applied to education^ but its relevance justifies similar work in the cultural field.

These developments underline the importance of the definition of a wide range of indicators - a familiar subject of debate in the social policy field, the 'social indicators' movement having been the focus of considerable activity throughout the last decade.^ In spite of the considerable research effort it prompted however, it produced no tangible results - education apart - in the cultural field, though this was less a question of failure than the consequence, in effect, of not talcing cultural activities into account.5

A generally accepted definition of social indicators is that they are numerical yardsticks of the State - and in particular the welfare - of society. They are of five kinds:

situation indicators - which reflect the present situation in a given social field;

indicators of ways and means which reflect action taken in the social field;

product indicators which express the quantitative effect of action taken;

internal effectiveness indicators expressing the relation between product indicators and ways and means indicators; and

external effectiveness indicators expressing the relation between situation indicators and product indicators.

Social indicators bring out the basic distinction that exists between functional and substantial rationality. While functional rationality is concerned with decisions taken in the light of purely economic calculation, substantial rationality takes the view that the value of the result achieved depends to some extent on the way in which it is achieved; in other words it implies that a qualitative dimension be introduced into decision-making - as is essential when the decisions concern cultural development. The importance of social indicators is therefore too obvious to need any proof. Defining them is a difficult process but the possibilities they open up are such as to make their exploration indispensable.

1. See Annex on page 43. 2. Cf. Catherine GIRARDEAU, Vers un système de statistiques sociales. INSEE,

Paris C.14, 1970. 3. See annex on page 43. 4. Cf. Jacques DELORS, Les indicateurs sociauxf SEDEIS, Paris, 1971 and

0. MANSON, Rapport Social, indicateurs sociaux, comptes sociaux. Analyse et Prévision, April 1969.

5. See Annex on page 44.

- 30 -

POLICY EVALUATION

It would be wrong to end without reference to the potential presented by policy evaluation methods. In the cultural field, evaluation studies are in most cases confined to the formulation of a descriptive balance sheet setting out the results of programmes or activities: attendance at a community facility for example, broken down by age group, socio-economic category, educational level, place of residence, etc. This 'body-counting' is, of course, the most elementary form of evaluation and in no case could this method be held to offer any measurement of the effectiveness of a policy. Today policy evaluation is increasingly taking the path of management review. The French study on the Maisons de la Culture and Centres d'Animation Culturelle is a very revealing case in point. 1 Its purpose was to bring about more rational decision-making in the use of the resources provided for these facilities and the five recommendations it made proposed that:

The unit costs of the different services should be worked out so that high-cost activities could be reduced and preference given to the others.

The cost per unit of service provided should be worked out and a classification drawn up.

The Maisons de la Culture should be set the minimum target of recovering the variable costs of live performance activities out of their own takings.

Own resources should be developed, in particular by renting out unoccupied premises for outside activities.

An activity selection procedure should be introduced designed to meet two objectives:

to optimize the number of performances within the limits of the grant made to the establishment;

to maximize audiences, which should be measures by applying socio-economic weightings.

Management review, though not to be confused with policy evaluation, is none the less its central core. Management review can be envisaged at its true value only by reference to the objectives of the policy concerned.

The prior statement of objectives in as operational a formulation as possible is essential for any evaluation. The problem is that cultural policies always refer to quantitative objectives. It is a matter of providing facilities, creating jobs, developing practices, setting up institutions and so on. But these are means rather than ends. The real objectives are at the very general levels of cultural democracy or cultural development. So policy evaluation requires clarification - ex ante (definition of objectives), during (management review) and ex post (effectiveness and efficiency).

The evaluation process is complex because all levels of decision and implementation have to be included in the analysis. This means that several different theoretical approaches - normative and explanatory - are necessary,

1. J. DE CHALENDAR, Quelques remarques sur la question des Maisons de la Culture et des Centres d'Animation Culturelle. Rapport de l'Inspection des Finances, Paris, 1973.

- 31 -

linked together to give an overall view of the policy being studied as the two following flow charts show.l

The component parts of policy analysis

Examples of normative approaches 'Policy Science' 'Public Choice' school

Explanatory approaches

POLICY ANALYSIS

EX ANTE EVALUATION Cost-benefit Cost-effectiveness

ECONOMIC THEORY OF BUREAUCRACY

AXIOMATIC POLICY EVALUATION

PUBLIC

1 '

POLICY-MAKERS

1 f

ADMINISTRATIVE SYSTEM

^ '

TARGET GROUPS

OTHER GROUPS

POLICY-MAKING (Sociology of

decision making)

POLICY IMPLEMENTATION

(process analysis)

POLICY EVALUATION ('Formative' evaluation)

1. From Jean-Pierre NIOCHE, De l'évaluation à l'analyse des politiques publiques, Revue Française de Science Politique, Vol. 32. No. 1, February 1982.

- 32 -

Assessment of public organizations' policies and instruments

Public action in terms of organization

Public action in terms of process of action

Normative approaches

Explanatory approaches

PUBLIC MARKETING (Needs analysis phase)

l l

PUBLIC

PROGRAMME BUDGETS

MANAGEMENT REVIEW MONITORING SYSTEMS

PROGRAMME

RATIONALIZATION OF BUDGETARY CHOICE STUDIES

(Ex ante econometric evaluation)

DECISION­MAKERS

ADMINISTRATIVE SYSTEM

POLICY EVALUATION

PUBLIC MARKETING (Communication and distribution phase)

TARGET GROUPS

(Government-citizen relations)

(Specific campaigns)

- 33 -

What is at issue in policy evaluation is therefore far more than just the evaluation of a specific project. All the analysis techniques we have reviewed (cost-benefit analysis, impact studies, cost-effectiveness analysis, modelling and indicators) contribute to policy research and it is with that purpose in mind that they have to be considered. Cultural development is too abundant a field for any one of these instruments to suffice to measure it.

In concrete terms, while researchers need to continue their efforts (virtually nothing has yet been achieved), it is possible to put forward three recommendations aimed at improving our knowledge of the economics of cultural development.

A complete inventory should be made of the various actions that are under way, their medium-term consequences and their effectiveness, assigning all current costs to one or other of these actions.

All the information necessary for the preparation of new actions needs to be gathered so that a real dialogue develops between all the economic partners of those actually involved in cultural development at every level.

A system of management review should be instituted based on strict comparison between programme forecasts and implementation and not merely on inspection of the nature of the way money is spent.

These conditions would ensure a constructive approach to cultural policy and allow the development of a real evaluation capability.

The key questions that we have discussed, using the cultural field as an example, are those at the centre of the discussion on evaluation for all projects initiated by the public authorities and they come up again and again in the history of rationalization of budgetary choices. The economist has no miracle solution to offer the decision-maker. His tools are imperfect and very difficult to apply to the field of cultural development. They should not for all that be rejected but they do need to be used with discernment. The avenues of research are fairly numerous and appear promising enough to hope that soon it will be possible to analyse the economic dimension of culture with the help of higher performance instruments than those that are at present available.

These instruments will then make it possible to put up an effective defence against budget cuts that would be highly damaging not only for cultural development but also for the economy as a whole. Now more than ever, cultural development constitutes an investment and it is time to discard those assumptions that cause it to be regarded as luxury spending and an ideal target in the general trade-offs that all too often sacrifice qualitative aspects for the purely quantitative aspects.

- 35 -

ANNEXES

TYPICAL COST-BENEFIT TABLE FOR A SECTION OF THE R.E.R. (Rapid transit system in the Paris region)

(Rate of discount: 10 per cent)

Date of entry into service of RER 1970 1975 1980 1985

INVESTMENT (F million)

Investment cost, RER Cost at 1.1.1970 prices

Cost of alternative investment (for reference) Cost at 1.1.1970 prices

Extra investment cost at 1.1.1970 prices

180 180

40 20

200 124

40 20

210 81

40 20

220 53

40 20

160 104 61 33

II. RUNNING COSTS (F million)

Annual running costs for RER

Annual running costs for alternative investment (train, bus, car)

Annual saving in running costs

Annual saving at 1.1.70 prices

3.5 4.2 5.0

7

3.5

40

7.9

3.7

27

9

4

18

5.8

10.8

5

12

III SAVING IN TIME

Traffic (million passengers)

Saving in time (million hours/year)

Value of annual saving in time (F million)

Value of saving at 1.1.1970 prices (F million)

10.5

1.3

13.5

1.7

16

2.1

18.5

2.4

6.5

136

11

111

17

85

25

60

IV TOTAL (II + III - I) (F million) 16 34 42* 39

Optimum date.

Source: Forecasting Directorate, Ministère de l'Economie et des Finance.

- 36 -

Results of cost-benefit analysis carried out by the United States Department of Healthf

Education and Welfare

Programme

Use of seatbelts

Restricted programme

Cost from 1968 to 1972 in

million dollars

2.2

0.7

Saving or benefit in

million dollars

2,728

681

Cost-benefit ratio

1,351.4

1,117.1

Number of

deaths avoided

22,930

5,811

Cost per death avoided

in million dollars

87

103

Reducing the number of pedestrian casualties 1.1

Campaign on the wearing of motorcycle helmets

Anti-arthritis campaign

Drunk driving campaign

Anti-syphilis campaign

Uterine cancer campaign

Lung cancer campaign

Breast cancer campaign

Tuberculosis campaign

Campaign for driving licence revision 6.6

Head and neck cancer campaign 8.1

Cancer of the spine campaign 7.7

153 144.3 1,650

8.0

37.6

31.1

55.0

73.7

47.0

17.0

130.0

413

1,489

613

2,993

1,071

268

101

573

23 3.8

1.1

0.5

442

666

55.6

42.5

21.5

16.7

9.0

5.7

4.5

4.4

2,398

5,340

11,590

34,200

7,000

2,396

5,700

3,336

5,824

22,252

3,470

6,400

7,663

22,807

13,801

268 29,100

170 42,944

Source: Department of Health, Education and Welfare

Source: Marie-Christine L0DE0N, La R.C.B. dans le secteur sanitaire. Analyse et Prévision, IX, 1970.

- 37 -

Examples of technology transfer from the space sector to other sectors

Source: Futuribles, November 1980 after a study by the Bureau d'Economie Théorique et Appliquée,

University of Strasbourg

Recipient sector

Space sector origin

Nature of innovation

Automobile industry

satellite stabilization

welding of space materials

space electronics

satellite and launcher materials

inertia wheel to reduce vehicle's fuel consumption

ion bombardment welding of automobile parts

data configuration table for tracing faults

(future) manufacture of brake pads from carbon fibre

Robotics Opto-electronics Automated production control systems in industry

Sea transport

propulsion

space electronics

propulsion

submarine propulsion systems

visual display systems

ship-to-ship communications

submarine ballast expulsion systems

Air transport

space electronics

Ariane

electronic flight recorders aircraft ('Black box')

in

future use - several years from now — of liquid rocket fuel (used in Ariane) as aviation fuel

space materials use of carbon fibres in aircraft structures

New sources of energy

conversion of solar energy

space materials

photovoltaic solar cells and power generators

man-made fibre vanes for wind-powered generators

project control of nuclear power station management operation

- 38 -

Computer space computer technology science

Meteosat

Banking space opto­electronics

Fishing space opto­electronics

Traffic satellite control altitude monitoring

Storage of energy storage electric power batteries

Construction space opto-industry electronics

Medicine space materials

space computer science

opto-electronics

Road satellite thermal engineering control

Telecom- space data munications transmission

Oil industry space materials

space electronics

Public satellite control transport

space materials

new components, computer cooling systems, computer coupling, bilingual computers

data archiving system

automatic cash dispensers

Fish shoal finding

urban traffic monitoring

standby storage by kinetic wheels

miniature batteries (in calculators)

detection of thermal leakage in walls and roofs

prostheses

use of computers in examination of heart, lung, etc.

low-risk X-ray radiography

ice prevention on roads and bridges by heat conductors

development of data transmission networks

steel-aluminium alloy cylindrical refinery structures

communications between offshore platforms

simulation systems for automatic underground railway operation

materials for rolling stock

- 39 -

urban space project management of large towns management organization

Pollution space opto- monitoring of polluted areas of the control electronics North Sea

Security space computer computer system for monitoring science technological and chemical risks of

offshore fires

space opto- burglar alarm sensors electronics

Sports space materials use of man-made fibres in ski and industry tennis racket production

Instrument- space instruments ultrasonic flowmeters for metering ation (oil) level in storage tanks

data display system

vibration dampers

Food satellite and stainless steel beer barrels launcher materials

Television space electronics electronic components, cathode ray tubes

Telephone satellite standby power systems for telephone stabilization exchanges

- 40

Productivity gains attributable to space research

Productivity gains

IÎ «•--

10 •*--

Curves showing total economic impact and total value of contracts awarded to industry

MUA

»<K

l<«

Tota

Total economic impact

1 value of contracts

awarded to industry

i«j" ivs: Year

MUA = million European units of a

- 41 -

Curves showing trends in various types of induced effect due to spending on space research

1 MUA - Mí) /

Technological effect /

-im /

1964 1982 | MUA -Tiii /

Effect on trade ,-——

-K«I /

__y **

1964 1982 MUA

-;i«>

Effect on organization and methods /^^^

-n».i s^

1964 1982 f MUA - :<<'

Effect on labour force

""" / ^ ^ ^ - ^

1964 1982*

- 42 -

Ratio of benefits to cost of participation in the METEOSAT satellite

(weather forecasting satellite)

from Futuribles November 1980

Benefit/cost ratios for European countries

Country Assumed share of funding per cent

4.29

2.29

4.73

21.07

12.19

5.60

25.57

15.35

4.41

3.96

in Benefit/cost

ratio

2.3

3.3

7.8

3.4

4.3

1.1

2.6

1.6

1.7

1.7

Belgium

Denmark

Spain

France

Italy

Netherlands

Federal Republic of Germany

united Kingdom

Sweden

Switzerland

Overall benefit/cost ratio 2.9

- 43 -

Public health satellite accounts

Products

Production Units

Type of household

Production of health services

Consumption of health services

Ways of meeting cost

Medical insurance institutions

Financing of consumption of health services

Social transfers

Interconnections between the various poles of development of a system of social statistics as applied to education

Production system > -i

Other fields

Health

Recreation Culture

Education sector.

\

Vocational training

Famili es

State of education of the population

i Products of the ! education system

School population

Education system

Socio-demographic system

System describing physical patrimonies

System of satellite accounts (costs, funding,

products, recipients)

Education statistics Education and indicators satellite

account

From: Xavier GREFFE, Politique Sociale. Coll. Sup. Presses Universitaire de France, Paris 1975.

- 44 -

Types of social indicator and fields covered

Active population and activity

INSEE French VIth and Vllth Plan Council of Europe

Health and sickness INSEE French VIth and Vllth Plan OECD Council of Europe Economic Com­mission for Europe

Education and culture

Time and leisure milieu,

Goods and services available

Physical environ­ment and housing

INSEE French VIth and Vllth Plan OECD Council of Europe

INSEE French Vllth Plan OECD Economic Com­mission for Europe

INSEE French Vllth Plan OECD

INSEE French Vllth Plan

Demographic data Data on structure of the active population, working hours and rates, accidents, disputes, remuneration, job security, career prospects, etc.

Morbidity rates; incidence, prevention, quantity of resources for a given population, equality of access to care, invalidity rates, mortality rates by sex, SEC*, etc., stillbirths by SEC*, average annual mortality quotients by cause of death, structure of care received by SEC*, etc.

Characteristics of school system, school results by SEC*, qualifications of teachers, enrolment rates, opportunities for further training, adult education, etc.

Time budgets by SEC*, sex, etc.

Rates and duration of participation in cultural activities, etc.

Income, guarantees against loss of earnings, patterns of consumption by SEC*, degrees of accessibility to public services, valuation of consumer prices per SEC*

Housing conditions, house equipment, overpopulation, distance from town centre, participation democratic life, etc.

* SEC = Socio-economic category.

From: X. GREFFE, Politique Sociale, op cit.

- 45 -

BIBLIOGRAPHY

Though not claiming to be exhaustive, this bibliography sets out a number of references in the field of the economic assessment of the cultural sector. Thus it includes work on cost-benefit analysis, impact studies and work concerned more generally with assessment. It is not therefore a bibliography on the economics of culture.

Azarchi, Lynne. Effects Resulting from the 1980 Metropolitan Opera Season Postponement. New York: Lincoln Center for the Performing Arts, Inc., 31 December 1980. Mimeographed.

Baird, Nini. The Arts in Vancouver: A Multi-Million-Dollar Industry. Vancouver: Community Arts Council of Vancouver, March 1976.

Baumöl William J. Welfare Economics and the Theory of the State. Bell and Sons, London, 1965.

Baumöl William J., and William G. Bowen. Performing Arts - The Economic Dilemma. Cambridge, Mass: MIT Press, 1967.

Brown, Harris, Stevens, Inc. Updated Report on the Influence and Effect of the Lincoln Center Complex on Manhattan Real Property Values. New York: Lincoln Center for the Performing Arts, Inc., August 1978. Mimeographed.

Chartrand Harry, An Economic Impact Assessment of the Canadian Fine Arts. Third International Conference on Cultural Economics and Planning, Akron, April 1984.

Citizens Union of the City of New York. Economic Implications of the Arts in New York City Subcommittee Draft Report. New York: 1 October 1981.

Clawson M. and Knetsch J.L. Economics of Outdoor Recreation, John Hopkins Press for Resources for the Future, 1966.

Council on Foundations Project in the Arts. 'Special Report II: Inflation, Recession and the Arts'. Foundation News (January/February, 1976).

Cuciti Peggy. Economics of the Arts in the State of Colorado, Denver, University of Colorado, Center for Public-Private Co-operation, 1983.

Cwi, David. 'Economic Studies with Impact'. Museum News (May/June 1981).

. Issues in Developing a Model to Assess the Community-wide Economic Effects of Cultural Institutions. Baltimore: Johns Hopkins University, November 1977. Mimeographed.

. Perspectives on the Economic Role of the Arts and the Impact of Arts Impact Studies. Conference on the Economic Impact of the Arts, Cornell University Graduate School of Business and Public Administration, 27 May 1981. Mimeographed.

. The Role of the Arts in Urban Economic Development. U.S. Department of Commerce, Economic Development Administration, Urban Consortium Information Bulletin. September 1980.

- 46 -

Cwi, David, and Katherine Lyall. Economic Impacts of Arts and Cultural Institutions: A Model for Assessment and a Case Study in Baltimore. U.S. National Endowment for the Arts, Research Division Report No. 6, November 1977.

Cwi, David, and Susanne Moore. The Economic and Social Impact of the Proposed Long Island Performing Arts Center. New York: The New York State Urban Development Corporation, October 1981.

De Chalendar J. Quelques remarques sur la gestion des Maisons de la Culture et des Centres d'Animation Culturelle, Rapport de l'Inspection des Finances, Paris, 1973.

Dupuis Xavier and Greffe Xavier. Rapport technique sur les méthodes de quantification du non-marchand. Centre 'Travail et Société' de l'Université de Paris IX Dauphine, 1979.

Frost, Murray, and Garneth 0. Peterson. The Economic Impact of Non-Profit Organizations in Nebraska. 1976-1977. Omaha: Nebraska Arts Council, August 1978.

Globerman Steven. Cultural Regulation in Canada, The Institute for Research on Public Policy, Montreal, 1984.

Greater Philadelphia Cultural Alliance. An Introduction to the Economics of Philadelphia's Cultural Organizations. February 1975.

Hammer, Siler, George Associates. Economic Impact of Selected Arts Organiz­ations on the Dallas Economy. City of Dallas: November 1977.

Hanten E.W. and Hendon W.S., The Akron Art Institute Study. Akron, Center for Urban Studies, The University of Akron, 1973.

Hart, Krivatsy, Stubbee. Lincoln Square Community Action Planning Program. New York: Lincoln Square Community Council and New York City Department of City Planning, 1971.

Haworth, John. The Arts and New York City's Economy. New York: December 1979. Mimeographed.

Hendon, Willaim S., James L. Shanahan, and Alice J. MacDonald, eds. Economic Policy for the Arts. Cambridge, Mass: Abt Books, 1980.

Hendon William S. 'Benefits and Costs of Historic Preservation' in: Economics and Historic Preservation, The University of Akron, 1983.

. Evaluation of Urban Parks and Recreation, Praeger, New York, 1981.

Jensen, Robert. Project Director/Curator. Devastation/Resurrection: The South Bronx. Exhibition catalogue. Bronx, New York: Bronx Museum of the Arts, 1979.

Kansas City Arts Council. Economic Impact of the Performing Arts on Kansas City. Kansas City, Missouri: Hallmark Educational Foundation, June 1980.

Lawrence, William J. Economic Impacts of 'Leisure-Time' Activities: Modeling the Cultural. Arts, Sports, and Tourist Industries in New York City. New York: Sociometrics, Inc. Mimeographed.

- 47 -

Mathtech. The Impact of the Broadway Theatre on the Economy of New York City. New York: The League of New York Theatres and Producers, Inc., 22 February 1977.

Metropolitan Arts Council. The Economic Impact of the Arts on Indianapolis. Indianapolis.

Musgrave. 'Cost Benefit Analysis and the Theory of Public Finance', Journal of Economic Literature. Vol. 7, No. 3, 1969.

Musgrave R.A. and Musgrave P. Public Finance in Theory and Practicer MacGraw Hill, New York, 1980.

Netzer, Dick. The Subsidized Muse: Public Support for the Arts in the United States. New York: Cambridge University Press, 1978.

New England Foundation for the Arts. The Arts and the New England Economy. 2nd ed. Cambridge, Mass: 1981.

New York Zoological Society. New York City Needs the Bronx Zoo: An Economic Impact Statement from the New York Zoological Society. New York: 26 April 1977. Mimeographed.

Nioche Jean Pierre. 'De l'évaluation à l'analyse des politiques publiques', Revue Française de Science Politique, Vol. 32, No. 1, February 1982.

Parham, Sidney F. 'States and the Arts: More than Meets the Eye'. State Legislatures Magazine (February, 1979).

The Port Authority of New York and New Jersey, The Arts as an Industry; Their Economic Importance to the N.Y. and N.J. Metropolitan Arear New York 1983.

Public Information Office, Lincoln Center for the Performing Arts, Inc. 'The Economic Impact of Lincoln Center' (slide show). Revised, 18 February 1982.

Sullivan, John J., and Gregory H. Wassail. The Impact of the Arts on Connecticut's Economy. Hartford: Connecticut Commission on the Arts, March, 1977.

Throsby CD. 'Social and Economic Benefits from Regional Investment in Art Facilities: Theory and Application', Journal of Cultural Economics, Vol. 6, No. 1, June 1982, pp. 1-14.

U.S. National Endowment for the Arts, Research Division Report No. 15. Economic Impact of Arts and Cultural Institutions: Case-Studies in Columbus: Minneapolis/St. Paul; St. Louis: Salt Lake City; San Antonio; Springfield. January 1981.

Vaughan D.R. 'Does a Festival pay?' in: Hendon W.S., Shanahan J. and MacDonald A.J. (eds.), Economic Policy for the Arts. Cambridge, Mass, Abt Books, 1980.

Violette, Caroline, and Rachelle Taqqu, eds. Issues in Supporting the Arts: An Anthology based on the Economic Impact of the Arts Conference. Ithaca, New York: Graduate School of Business and Public Administration, Cornell University, 1982.

- 48 -

Wiener, Louise W. Perspectives on the Economic Development Potential of Cultural Resources. White House Conference on Balanced National Growth and Economic Development, January 1978.

Worcester Cultural Commission. 1981 Economic Impact: A Study of the Impact of Cultural Activity on the Economy of Worcester. Massachusetts. 1981.

- 49 -

INDEX

Page

Foreword . iii

PART I: THE BASIC THEORY AND METHODOLOGY OF COST-BENEFIT

ANALYSIS 1

I. The pursuit of optimization; the surplus theory 1

II. Methodology of cost-benefit analysis 3

PART II: APPLICATIONS OF COST-BENEFIT ANALYSIS IN THE FIELD OF

CULTURAL DEVELOPMENT 7

I. Types of cost and benefit 7

1. Monetary and real costs and benefits 7

2. Tangible and intangible real costs and

benefits 7

3. Internal and external real costs and benefits . . . 8

4. Final and intermediate real costs and benefits . . . 9

II. Examples of the application of cost-benefit analysis in the field of cultural development 9

III. The validity of the application of cost-benefit analysis to cultural development 12

1. The limitations of cost-benefit analysis as applied to cultural development 13

2. 'Rationalization of budgetary choices' and cultural developmet 14

PART III: IMPACT STUDIES, A NEW EVALUATION TECHNIQUE FOR CULTURE . . . . 19

I. The impact study problem 19

II. Impact study methodology 19

III. Examples of impact studies in the cultural sector . . . . 21

OTHER POSSIBILITIES 27

Cost-effectiveness analysis 27

Modelling 28

Indicators 28

Policy evaluation 30

- 50 -

Page

ANNEXES 35

Typical cost-benefit table for a section of the R.E.R. . 35

Results of cost-benefit analysis in the health sector . . 36

Examples of technology transfer 37

Benefit/cost ratios of a space programme 42

Structure of a public health satellite account 43

Classification by field: the example of education . . . . 43

Types of social indicator and fields covered 44

BIBLIOGRAPHY 45

INDEX 49