Kracht Koppeling in de Industrte, December 1980. 4Warmte/Kracht potentieel bij industri~le bedrijven in Nededand, Krachtwerktuigen, December 1979. sj. Quakernaat and K.A. Duijves, Chemie en Kolen, ESC-13, June 1981. 6Gebmik van Kolen in de Industrie, AER, Staatsuitgevedj, Den Haag, May 1981. ~Advanced Technologies for the Control of Air Pollution from Coal Combustion, NATO- CCMS Conference, Copenhagen, September 1980.

sJ.C. Buschmann, E.L. Rasmussen and S.M. Kaplan, Disposal of Wastes from Dry SO2 Removal Processes, Presented at: Joint Power Generating Conference, Phoenix, AR, USA, 28 September - 2 October 1980. 9Nota Energiebeleid, Dee l 2/Kolen, Tweede Kamer der Staten-Generaal, Z~ing 1979-1980, No 15802, February 1980. ~OK.A. Duijves, Steenkoolas, ESC-12, Petten, June 1981. 1 tStatistisc h zakboek 1981, CBS.

Evaluating utility residential energy conservation programmes: an overview of an EPRI workshop

The Electric Power Research Institute sponsored a two-day workshop, 'Measuring the Effects of Utility Conservation Programs', in February 1982. The workshop featured 10 presentations by electric utility analysts on several subjects related to evaluation of utility residential energy conservation programrnes. This paper summarizes the major themes and findings of the workshops. The workshop papers are published in the Proceedings, available from EPRI.

Keywords: Energy; Conservation; Residential

Residential energy conservation programmes represent an important and rapidly growing element of electric utility activities. For example, the Tennessee Valley Authority provided home energy audits to about 20% (more than 490 000) of their residential customers between August 1977 and September 1981. t California electric and gas utilities spent $157 million in 1981 on their customer conservation programmes (up from $91 million in 1980).2 As their conservation activities increase, utilities are devoting more attention to the determinations of the energy-saving effects of these pro- grammes and their cost-effectiveness (to participating and non-participating customers and to utility stockholders).

The growing importance, size and cost of these programmes prompted the Electric Power Research Institute to sponsor a two-day workshop, 'Measuring the effects of utility

conservation programs'. This work- shop focussed on two issues related to the operation and performance of these programmes. First, what are the effects of such programmes on elec- tricity sales, peak loads, and revenues; and what are the programme costs? Second, what methodologies work best (and at what cost) in conducting these evaluations?

The workshop, held at the Battelle Columbus (Ohio) Laboratories on 11- 12 February 1982, featured talks from 10 electric utility personnel (see Table 1). The 75 attendees were from electric utilities, state and federal energy agencies, and a variety of university and research organizations.

The workshop's primary emphasis was on measuring the impacts of resi- dential conservation programmes. This focus was chosen because the residen- tial sector is less heterogeneous than other sectors and because few evalu-

Communica t ions on energy

ations of non-residential programmes have been completed. Accurate measurement of the impact of resi- dential conservation programmes is an essential element of effective long-term power system planning. (See Macphee's discussion of how estimated impacts of conservation programmes are inte- grated into the TVA system planning models.)

Most workshop participants agreed that conservation can, and should, be treated as a resource (as discussed by Brennan). This, however, requires accurate estimates of the savings that will result from various levels of con- servation programme investment. Programme evaluations, if carefully done, can provide such estimates of energy saved per dollar invested. These results can also be used to provide justification for the expansion, modi- fication, or elimination of specific programmes; to forecast demand in the utility service area; and to analyse the need for additional generating capacity.

Measurement of effects

Measurement of the energy savings which are attributable to a particular programme as distinct from other forces driving conservation, is a com- plex undertaking. (See the papers by Hannigan, Weiss, Burnett, and Shutes for discussions of some of the problems likely to be encountered.) A house- hold's demand for energy has many determinants including fuel prices, weather, appliance stock holdings and usage patterns, weatherization features of the dwelling unit, and behavioural characteristics of the occupants. All the determinants of demand interact with the influence of the utility programme intervention to produce changes in energy consumption. Similarly, the effect of adopting any particular energy- saving device or practice depends upon the mix of energy-related features and behaviours present in the household.

Nearly all studies of the impact of residential conservation programmes (and all the studies discussed at this workshop) have been observational studies. This type of study differs from an experiment in that treatments are not randomly assigned to households, a Without random assignment, many of

ENERGY POLICY March 1983 77

Communicat ions on energy

Table 1. Pape~ ixesented at the February 1982 EPRI w o r l ~ on 'Mmmudng the Effects of Utility ~ Programs 'a

W.G. Bentley and L. Garrett, The Demand for Energy Conserving Devices, Florida Power and Light Company.

J.E. Brennan, Is Conservation Really a Resource?, Pacific Gas and Electric Company.

Tom Bumett, Measuring Weathedzation Effec- tiveness: Portland Genera/ Electric Company's Experience, Portland General Electric Company.

Scott R. Hannigan, Residential Conservation Programs at Pacific Power & Light Company: Models, Forecasts, and Assessments, Pacific Power & Ught Company.

Larry E. Lewis, Energy Conservation Measure- ment of Residential Nonexperimenta/ Data, Consumers Power Company.

B.A. Macphee, R.E. Seiber, and J.P. Harper, TVA Methodology for Determining the Cost/Effec- tiveness of Conservation, Load Management, and Renewable Energy Progr~ns, Tennessee Valley Authority.

Dennis C. O'Neill, Can the Effects of Residential Conservation be Estimated from a Customer Survey? GPU Service Corporation.

David Shutes, The Potential for Response Bias in Audit Evaluation, Wisconsin Power and Light.

Cadn S. Weiss and Tim M. Newcomb, Evaluation of the Home Energy Check Program, Seattle City Light.

Michael V. Williams and Robin J. Walther, Issues in the Evaluation of Conservation Programs: Non-Random Data and Participation Bias, Southem California Edison Company.

aThe papers are published in Workshop Proceed- ings; Measuring the Effects of Utility Conservation Programs, E. Hirst, ed, EPRI EA-2496, Electric Power Research Institute, Palo Alto, CA, July 1982.

the variables that affect energy con- sumption, besides the programme's influence, may be differently distri- buted across participant/non-partici- pant groups. When the two groups are not equivalent, estimates of program- me impact may be biased and therefore inadequate to support general conclu- sions about the programme's true effects.

Systematic bias The problem of systematic bias in observational studies is especially acute in the case of conservation programmes because the magnitude of expected programme effects is in the same range as the magnitude of effects due to con- founding factors. While engineering estimates suggest that savings of 50% and more 4 are technically possible s in the average home, programme effects are unlikely to exceed 20%. Con- founding factors such as changes in

appliance stock (eg wood stoves) or usage patterns (eg heating more rooms) may produce changes in con- sumption of similar magnitude.

There are two principal strategies for reducing bias in observational studies: matching or matched sampling; and statistical adjustment by means of models relating the dependent variable (energy demand) to confounding vari- ables. For either strategy, one must identify the major confounding vari- ables, obtain data on them and devise ways to reduce the bias they may cause. Much of the research reported upon at the workshop illustrates this process.

Burnet t ' s paper, in particular, illus- trates how estimates of programme impact are altered as more refined adjustments for confounding factors are introduced. His methods of weather adjustment (weather is a con- founding variable which nearly all the studies adjust for) are especially detailed. He also points out the importance of differences in vacancy rates between participants and non- participants as a source of bias in estimates of programme impact. Other common sources of bias identified by workshop speakers include response bias (see Shutes), inaccuracy of self reports , and a ' rebound' effect in which programme participants raise their thermostat settings after completing conservation retrofits.

Assumption of equivalence Most of the studies which attempt to estimate the savings due to a program- me compare the before and after consumption of programme partici- pants to that of a comparison group of non-participants. (See Hannigan, Weiss, Burnett , Shutes, and Lewis.) Even though efforts are made in these studies to adjust for some confounding factors, it is clear that some potential sources of bias are not examined. Instead the assumption is made that the part icipant and comparison groups are nearly equivalent.

Williams and Walther argue that the assumption of equivalence between participant and comparison groups is usually incorrect in conservation programmes. 6 Furthermore, if par- ticipation is a function of energy usage

(ie if, as seems likely, households with higher usage are more likely to par- t icipate), an estimated coefficient for a part icipation variable in a single- equation regression model will not reflect solely the influence of participa- tion on consumption. Some component of the coefficient for this participation variable will be a result of the process by which customers select themselves into the programme. Williams and Wal ther suggest that the impact of a programme can be estimated more accurately with the use of a two- equation model that includes partici- pat ion as an endogenous variable.

Determinants

Their discussion of the potential value of a multiple equation model with an endogenous variable for participation is essentially a proposal for future research. No efforts to estimate the probabil i ty of participation in such a model are discussed. There are how- ever, two papers which deal with issues related to the development of models that give more attention to the deter- minants of participation. Bentley and Garre t t discuss the demand for energy- conserving devices and emphasize the importance of studying the distribution of consumer discount rates and expected savings in attempting to fore- cast saturation levels.

A recurrent theme of the workshop is the need to disaggregate data in order to consider the distribution not only of residential customer discount rates, but also of their appliance holdings, expected savings, preferences and responses to programmes.

Customer response Most evaluation studies do not analyse the diversity of customer response to programmes. Differences between participants and non-participants, between early and late participants, and among participants in their levels of programme-induced response have not been adequately explored. Greater at tention to the range and distribution of customer response is an essential part of future research efforts. Under- standing the distribution of discount rates and of other customer character-

78 ENERGY POUCY March 1983

istics that predict response to pro- grammes is an important part of developing more reliable and better specified models.

Conditional demand a n a l y s i s

Ano the r line of research that should aid in the development of models which integrate equations that predict both participation and energy demand are conditional demand models such as the one O'Neil l discusses (based on work by Parti and PartiT). Conditional demand analysis is a tool for obtaining the kind of information end-use metering could provide, but more cheaply. In O'Neil l 's conditional demand model, energy use is estimated as a function of appliance stock, economic/demographic characteristics and conservation actions taken.

The regression parameters associ- ated with the appliance-ownership dummy variables estimate the average monthly kWh usage of the appliances. The coefficients associated with conservation actions estimate the average reduction in monthly usage due to specific conservation steps. The coefficients associated with the economic/demographic characteristic's est imate each characteristic's impact on demand. If knowledge of the distri- bution of characteristics among participant/non-participant groups can be combined with adjustment factors obta ined from conditional demand models, more accurate estimates of the energy savings due to programmes should result.

Consensus Although there is considerable diver- sity among the approaches used and att i tudes expressed at the workshop, there is also a surprising amount of con- sensus on some common themes. Our understanding of this consensus is summarized below.

First, there is general agreement (particularly among the Pacific North- west utilities; see papers by Hannigan, Weiss, and Burnett) that utility home energy audit programmes save energy for customers and are cost-effective for them. s For example, the Pacific Power & Light evaluation of their home energy

analysis and weatherization loan pro- gramme shows an average annual saving of almost 3 600 k w h per partici- pating household (see Hannigan); this is the saving that was attributed directly to the programme, net of the saving achieved by non-participating house- holds (870 kWh/year). The Portland Genera l Electric estimate of net saving per participating household is 2100 k w h (see Burnett) and the Seattle City Light (SCL) estimate is 1500 kWh (see Weiss).

On the other hand, evaluation of the Wisconsin Power & Light (WPL) audit programme (which is limited to gas space heating customers and does not include the attractive financial incentives that are part of the Pacific Northwest programmes) shows a very small net saving of about 2% (about 2 million Btu/household, see Shutes). Analysis of energy savings due to the Wisconsin programme is greatly complicated by the low survey response rate (about 40%) among non- participants; alternative analyses suggest energy savings up to 9% due to the WPL programmes.9

Regional benefits The Seattle City Light evaluation analyses not only cost-effectiveness for participants but also the economic benefit of their programme to both SCL and the region as a whole. Because the marginal cost of electricity is much higher for the region as a whole than for the SCL, the net present value is positive for the region regardless of pr imary home heating fuel type. How- ever, the programme yields a benefit to SCL only for participating homes that are electrically heated.

The workshop papers include several ways to approach conservation evalu- ation issues. Participants agree that this kind of ' tr iangulation' of methods is important because each of the approaches has limitations; use of several approaches which (one hopes) lead to similar conclusions lends confidence to the evaluation findings. The use of alternative lines of evidence to study the same issue is a central tenet of evaluation research.I° Clearly, workshop participants are aware of the need to use a variety of approaches;

Communications on energy

several methods to measure program- me impacts are used in the workshop papers.

Burnett discusses several levels of regression analysis, including estimation of separate equations for participant and non-participant groups, for the pre- and post-audit time periods. Williams and Wal ther suggest estimation of separate regression equations for pre- dicting programme participation and energy consumption. Weiss discusses use of later programme participants as a comparison group for earlier pro- gramme participants. In Weiss' study, post-audit electricity consumption for the two groups is compared using a simple t-test of the difference in the means. Lewis discusses use of various time-series analysis methods to evalu- ate the energy savings of conservation programmes; his example focuses on changes in rate structure for elderly families.

Bentley and Garret t , Brennan, and Macphee discuss different approaches to incorporating the outcomes of evaluations (in terms of estimated energy savings and programme costs) into other utility analyses such as estimation of future loads, penetration of different end-use technologies, and analysis of future generation require- ments and costs.

These papers and the ensuing dis- cussions lead to another common theme: evaluation is part of a larger process related to overall utility planning. Evaluators must be cognisant of the users of their evaluations (eg conservation programme managers, utility corporate planners, public service commission staff), the purposes of these evaluations (eg to improve programme performance, to plan for future capacity expansion, to establish rates and rates of return), and the dif- ferent perspectives for an evaluation (participating and non-participating customers, utility stockholders, the region as a whole; see the Weiss and Macphee papers).

Data problems Because these evaluations deal with complicated social processes, there are many difficulties associated with the conduct of a useful evaluation. The

ENERGY POUCY March 1983 79

Communicat ions on energy

most serious - and expensive- relate to collection of relevant data. Problems associated with the quality and avail- ability of data are particularly severe. Issues related to the accuracy of house- hold self-reports (on, for example, their recent conservation practices and measures) and non-response (lack of data from large fractions of programme participants and/or non-participants; see Shutes) are particularly trouble- some.

S e l f - s e l e c t i o n

Difficulties associated with correction for self-selection are discussed above and in many of the workshop papers. Briefly, there are two ways to deal with the voluntary nature of these conservation programmes. The first is through programme design, by attempting to design it so that it closely approximates the classical experi- mental design (with random assign- ment); this increases the similarities between the participant and non- participant groups. For example, the Seattle City Light evaluation uses later programme participants as a compari- son group (see Weiss); this group seems quite similar to the earlier participants.

The second approach is to correct for self-selection in the analysis stage of the evaluation (see Hannigan, Burnett, and Williams and Walther). The most common method here is to use multi- variate regression equations that include several explanatory variables that 'correct for' differences between participants and non-participants.

N e t effects A closely related issue concerns attribution Of observed effects to the particular programme under evalu- ation. Disentangling the confounding effects of weather (see Burnett), fuel prices, and differences between the participant and non-participant groups is as difficult as it is important in unambiguously estimating the energy savings for a particular programme, net of all other effects.

Underlying many of these concerns is an implied need for better under- standing of consumer decision making.

This includes the need to understand the distribution of consumer discount rates, perceived costs and benefits of conservation practices and measures, sources of information, etc and the need for better models to predict household conservation behaviours and programme participation.

Workshop participants express con- cern that little evaluation attention has yet been devoted to utility programmes in non-residential sectors. In particular, the participants think that utility pro- grammes aimed at improving energy efficiency and load factors within com- mercial buildings are particularly deserving of additional attention.

Similarly, participants think that more effort needs to be devoted to analysis of the load reduction effects of utility programmes. The importance of end-use metering to determine load shapes and to validate engineering estimates of energy savings is empha- sized repeatedly. Depending on a utility's perspective, the effects of con- servation programmes on peak loads may be even more important than the effects of these programmes on energy consumption.

C o o p e r a t i o n

Finally, there is agreement that addi- tional cooperation among utilities (and other interested parties) is needed. Evaluations are expensive (particularly data collection) and different groups have developed valuable approaches to addressing these common evaluation problems (as demonstrated during this workshop). Therefore, additional co- operative efforts are needed both to improve the cost-effectiveness of evalu- ations themselves as well as to improve their reliability. These efforts can occur through Electric Power Research Insti- tute and Gas Research Institute research projects, publications, and workshops. In addition, utilities within a state can cooperate, either directly or through their state public service com- mission. For example, the California utilities conducted a statewide con- sumer survey to develop baseline information before the start of their Residential Conservation Service programme.

Cooperation in data collection

efforts and in the routine exchange of information on evaluation results and methods also should be supplemented with systematic recta-evaluation. Meta-evaluation, or the evaluation of evaluations, is often used to provide more persuasive evidence of, or shed new light on, a programme's impacts than was present in an original study or studies. 11 Through similar and differ- ent analyses on the same data base, meta-evaluation examines the quality of evaluations by testing the validity of their original conclusions. Meta-evalu- ation also can be used to improve the planning of future evaluations and the selection of methodological tools. 12 Cooperative sponsorship of meta- evaluation research is an important means of improving the quality of con- servation programme evaluations.

B a r r i e r s

Overall, the workshop presents a prom- ising picture of the state-of-the-art in evaluation of utility residential con- servation programmes. There is a good deal of consensus on the major research issues, methods, and future directions. Data quality and avail- ability seem to be the primary barriers to improving research results. Data acquisition and manipulation require far more resources than do analytical and modelling activities. Perhaps pooling of utility resources could make it possible to develop more useful data bases and to ensure continued meth- odological progress.

Linda Berry and Eric Hirst Oak Ridge National Laboratory

TN, USA

This research was sponsored by the Electric Power Research Institute under Interagency Agreement ERD-81-142 under Union Carbide Corporation contract W-7405-eng-26 with the US DeparbT~ent of Energy.

tDivision of Energy Conservation and Rates, Program Summary, Tennessee Valley Authority, Office of Power, October 1981. ZG. Amaroli, '1980 energy savings and peak demand reductions, and 1976-1981

80 ENERGY POLICY March 1983

expenses for energy conservation pro- grams', Memorandum to members of the Califomia Public Utilities Commission, 29 October 1981. 3D. Campbell and J. Stanley, Experimental and Quasi-Experimental Designs for Research, Rand McNally, 1966. Also T. Cook and D. Campbell, Quasi-Experi- mentation, Rand MoNally, 1979. 4Solar Energy Research Institute, A New Prosperity: Building a Sustainable Energy Future, Brick House Publishing, 1981. sObviously, in practice, not all technically feasible (or even all cost-effective) con- servation actions will be taken. 6L. Berry et al, Review of Evaluations of Utility Home Energy Audit Programs, Oak Ridge National Laboratory, ORNL/CON-58, March 1981. 7M. Parti and C. Parti, 'The total and appli- ance-specific conditional demand for

electricity in the household sector', The Bell Joumal of Economics, Vol 2, No 1, Spring 1980. aWhile the cost-effectiveness of programs to participants has been demonstrated, programme cost-effectiveness to non- participants ratepayers, to utilities and to society as a whole is still debatable. More research is needed to resolve these issues. 9S. Grady and E. Hirst, Evaluation of Utility Home Energy Audit Programs: A Wisconsin Example, Oak Ridge National Laboratory, ORNL/CON-88, March 1982. loCampbell and Stanley, op cit, Ref 3. ltC.L. Gruder and T.D. Cook, 'Metaevalu- ation research', Evaluation Quarterly, Vol 2, 1978. 12E.J. Soderstrom, L. Berry and E. Hirst, 'The use of metaevaluation to plan evalu- ations of conservation programs', Evaluation and Program Planning, Vol 4, 1981.

Thermodynamics, energy price and standard of living This article takes issue with Per Thoresen's assertion, in an earlier paper, that energy price increases need not reduce living standards. Edward Saraydar maintains that Thoresen's conclusion is based on very strong assumptions, ignoring the basic economic proposition that, given demand, it is conditions of supply that determine price. Thoresen's 'thermodynamic analysis' does emphasize the role of energy input into production, but adds little insight as to how increasingly scarce non-renewable energy sources may affect future economic welfare, and is misleading in its policy response to price increases.

Keywords: Energy; Price; Thermodynamics

Per Thoresen has recently 1 taken a thermodynamic approach to the deter- mination of goods and services output to argue that ' there is no reason why increases in energy prices should reduce our standard of living' (p 145). This article aims to demonstrate that Thoresen 's conclusion is based on very strong assumptions, and in particular that it ignores the basic economic proposi t ion that, given demand, it is condit ions of supply that determine price, and not the other way around. In o ther words, a change in price reflects a

change in the conditions of supply and, as will be shown, it is essentially a non- competi t ive market for energy sources that dictates a decline in economic welfare in energy-importing countries with an increase in energy price.

Thoresen argues that 'standard of living is i ndependen t of the price of energy ' (p 145, his emphasis) but his

conclusion is dependent on his defini- tion of 's tandard of living'. The first definition (Ref 1, p 146) states 'in this paper , "standard of riving" is defined materialistically, ie as the consumption of goods and services'. Since he defines ncPc as ' the value of goods and services consumed in the country' , this would indicate that standard of living = ncP o

or, in real terms, nc. However, Thoresen then contradictorily states that ' s tandard of living may be repre- sented by Net National Product (NNP) ' (pp 145-146), defined as:

N N P = ncP c + neP e - n ip i

- E i P E = m L (1)

where c, e, and i stand for consump- tion, exports, and imports respectively; P, for prices; PE = energy prices; n = real quantities; Ei = the real quantity of energy imports including raw materials; m = the number of people

Communications on energy

having income; and L -- average income. Note that ncPc here is only one

c o m p o n e n t of NNP. Now differentiate Equation (1):

nc dPc + Pcdnc + nedPe + Pe dne - nidPi - Pidni - EidP E - PEdEi = m d L + L d m (2)

Unfortunately, after this differentia- tion, Thoresen then assumes his conclusion! That is, 'assume produc- tion of goods and services to be constant, d n = 0, dEi = 0', as well as constant employment, dm -- 0 (p 147), so that Equation (2) becomes:

ncdPc + n e d P e - nidPi - E i d P ~ = m d L (3)

If one equates 'standard of living' with NNP, and then assumes, as does Thoresen, that real output and employ- ment of all factor inputs, including energy, remain constant and inde- pendent of all changes in final product prices and factor (including energy) prices, ie demand insensitive to price, the only (trivial) thing left to demon- strate is the extent to which final product prices would have to respond to a change in the price of energy, given the assumed constancy of real output and input.

Thoresen's Equation (9) suggests that the change in product prices depends upon the size of C, which in his Equat ion (6) determines the size of the increase in 'wages' (average income) ' demanded ' by 'consumers' who 'will demand the same standard of living' with an increase in prices (p 147), ie d L / L = C ( d P / P ) , with C a constant greater than or equal to zero. There are two points to be made here. First, L is defined by Thoresen (p 146) as the average 'value of all remunerations (wages, interests, profit etc)', and not just the average wage of consumers. Second, note that the particular relationship postulated between 'average income' and prices is irrelevant to Thoresen's argument, and, as will be demonstrated, C is in fact always equal to unity.

First, however, it is important to explore Thoresen's model in some detail . The self-fulfilling assumption that only product and non-energy factor prices change with an increase in energy price should be relaxed. Then,

ENERGY POLICY March 1983 81