1

Utility Investment Valuation Strategies:

A Case for Adopting Real Options Valuation

Richard White Principal Author POLICY AND PLANNING DIVISION

Chris Villarreal Principal Author POLICY AND PLANNING DIVISION

Marzia Zafar

Director

POLICY AND PLANNING

DIVISION

October 3, 2013

POLICY AND PLANNING DIVISION

2

Executive Summary It has long been recognized that electricity and water infrastructure throughout the United States is

aging faster than it is being repaired. It has been estimated that the long-term impact due to this

underinvestment will be a significant decrease in the US GDP in the coming years.1 In addition to the

declining state of infrastructure, new external priorities, such as the adoption of renewable energy

sources, are putting new types of pressure on utilities’ resources. This confluence of events is

generating renewed pressure to find new and more robust investment strategies that can keep up with

these changes.

Traditional approaches to making capital investment decisions and prioritizing operations and

maintenance (O&M) projects have relied heavily on methods such as Net Present Value (NPV) and Cost

Benefit Analysis (CBA). The veracity of these models is contingent upon assumptions about the future

returns from the prospective investments. Increasingly, the rise of uncertainty in utilities operations2 is,

however, compromising these assumptions and the ability of utilities and regulators to predict with

confidence what these expected returns will be. When using the deterministic NPV–like models, a

consequence of high uncertainty tends to be lower valuation of projects. As a result, many projects that

could end up being beneficial to both utilities and ratepayers are not undertaken.

In this paper we argue that utilities and regulators should consider new types of investment valuation

strategies for maintenance and upgrade projects in electric and water utilities. While the NPV methods

are effective at developing a baseline assessment of a project’s value, they neglect a significant value

associated with the ability to actively manage the uncertainty in a project. We discuss one class of

methods in particular - real options - that attempts to value the managerial flexibility in a project. This

approach includes options such as delaying projects, expanding projects, and abandoning projects.

We further suggest that utilities already, either explicitly or implicitly, exploit the option value in

projects. The most obvious managerial option that utilities have is the option to delay a project.

Waiting to see which systems fail can assure that equipment is used to its maximum useful life and could

reasonably be argued as a rational way to minimize system costs (“if it’s not broke, don’t fix it”). This, of

course, has a direct consequence to the customers who may experience service outages and other

negative events when equipment does in fact fail. If utilities operated in a competitive environment,

they would be discouraged from waiting indefinitely, as competitors could provide better service levels,

but in the current regulatory environment there is not this external pressure to resolve these

uncertainties before events happen. Clearly there are issues here that regulators should heed.

1 See American Society of Civil Engineers (ASCE) “Failure to Act Report Card: 2013” (available at

http://www.infrastructurereportcard.org/grades/), which gave the electric infrastructure a rating of D+ to water a D. See also, McKinsey Global Institute “Game Changers: Five Opportunities for US Growth and Renewal” (2013), which estimates that investment in utility infrastructure needs to rise from 2.6% of GDP to 3.6 % by 2030. 2 Increasing uncertainty can come from various sources including from the exposure to the long neglected

infrastructure, the impact and effectiveness of new technological innovations, the rise of new intermittent energy sources in the electricity sector, and the loss of new sources water in the water sector.

3

Finally, the increasing availability of advanced technologies to provide additional information about

utility operations can provide additional input into a utility’s option portfolio. As a utility collects

additional and more granular data, the utility can make more informed investment decisions, including

targeted application of new technologies, maintaining existing technologies, or replacing technologies

with similar operational characteristics. This ability is new and may upset traditional approaches to

investment decisions, as well as regulatory decisions regarding utility investments in a General Rate

Case.

The implications for utilities and regulators for using this type of investment strategy are also discussed.

Some of the key characteristics of this approach are:

Development of comprehensive strategic maintenance and upgrade plans that extend beyond

the current three-year budget cycle.

Single project performance would be less of a focus; rather, the performance of a portfolio

would be more emphasized. Individual project failures would not be seen as a waste of money,

but part of the management of a broader portfolio of assets.

Information-gathering associated with resolving uncertainties becomes a more precisely valued

activity and becomes a valuable component in the investment planning process. Understanding

how new information can resolve uncertainties will help define funding objectives for new more

risky projects

This new approach requires that utilities more actively search for opportunity that may be perceived as

more risky under the traditional valuation methods. It also requires regulators to support utility

flexibility in an investment strategy that includes the potential for failures, recognizing, however, that

this new paradigm can be seen as reducing the overall risk and costs. The chicken and egg problem is in

setting risk tolerance. The current process has commissions assigning budgets, and utilities interpreting

and implementing them.

4

Contents Executive Summary ....................................................................................................................................... 2

Introduction .................................................................................................................................................. 6

Background ................................................................................................................................................... 7

Utility Maintenance, Repair and Upgrade Economics .................................................................................. 8

Deterministic Valuation ............................................................................................................................ 9

Current Practices in Maintenance Decision Making ............................................................................... 11

Stochastic Valuation ................................................................................................................................ 12

Options and the Value of information ........................................................................................................ 13

Strategic Management of Real Options Portfolios ................................................................................. 14

1) Timing Option ............................................................................................................................. 15

2) Expand Option............................................................................................................................. 15

3) Abandon Option .......................................................................................................................... 16

Barriers to Valuing Real Options ................................................................................................................. 16

Utility ....................................................................................................................................................... 17

Regulatory ............................................................................................................................................... 17

Current Metrics ....................................................................................................................................... 18

1) Electric service ............................................................................................................................ 19

2) Water Service .............................................................................................................................. 20

Conclusion ................................................................................................................................................... 21

Recommendations ...................................................................................................................................... 22

References .................................................................................................................................................. 23

5

This White Paper was prepared by Policy & Planning Division (PPD). PPD is a division within the

California Public Utilities Commission consisting of a small group of policy analysts charged with

identifying and analyzing utility industry issues, internal and external procedures, and interagency

relationships that would not ordinarily be addressed by the Commission’s industry divisions in their

course of operations. PPD provides Commissioners, the Executive Director, and the Management Team

with independent analysis and advice focusing on Commission practices, procedures, issues, and

policies. PPD’s main mission is to provide proactive leadership on emerging policy issues of broad

importance to the Commission and support sound, long-term policy development through independent

research and analysis in concert with other divisions and agencies. This paper does not necessarily

represent the views of the Commission, its Commissioners, or the State of California.

6

Introduction The responsibility for maintaining large, diverse and complex electric grids, telecommunications

networks, natural gas pipelines and distribution systems and water supply networks is a daunting task.

These expansive networks have been built out and developed over the course of decades and contain

equipment of varying ages and vintage. With each new era comes a new set of technical standards and

maintenance requirements layered on top of the already existing ones. Utilities are now faced with

what appears to be an increasing level of stress on the network from weather related events, which

places an extra risk burden on older assets. The utilities must manage this mélange of equipment and

ensure high standards of safety and reliability while balancing a constantly changing set of priorities for

ongoing maintenance and repair. Typically, utilities rely on a combination of sensors, measuring

devices, physical inspections, and other stress tests to assess the performance and the remaining

productive life of components and sub-systems in their networks. Unfortunately, none of these

processes or tests can provide perfect insight into which system will fail and when, although predictive

analytics and life cycle information technology are getting increasingly accurate and insightful.3

Furthermore, even if ideal measurement and inspection processes could be achieved, it is simply not

possible or practicable to measure and inspect every inch of pipeline, distribution line, or switch in a

network.

Utilities must also recognize the uncertainties within their networks and develop processes that account

for and manage the uncertainties. Utilities rely on the collective knowledge and wisdom of technicians,

managers, and engineers to rate components and assess the overall state of the network. Based on

these evaluations, utilities must balance what is known about the state of the network through physical

and historical records against what is uncertain or unpredictable and develop priorities, and budgets

that will assure the safe and reliable operation of their networks.

Managing these networks in the face of both technical and operational uncertainties requires the

utilities to make some assessment of the risks associated with both technical and operational

maintenance of investments. Inherent in this process is some estimation of the likelihood of certain

types of events happening (e.g., equipment failure rates) and consequences if they occur. The concept

of risk for utility investments is not new; every utility makes their own determination and risk

assessment is a key consideration in ensuring the continuity of utility service.

What is new, however, are rapid changes in monitoring and information technology. This in turn is

challenging utility managers to effectively make use of these new technologies. These technologies can

help consolidate and filter large amounts of data more efficiently. They can also provide powerful

analytic methods that allow deeper dives in the data and provide better insight into the

interconnections, dependencies, and condition of systems. These new technologies clearly disrupt the

way utilities operate on a regular basis. For example, powerful probabilistic based predictive models

can more accurately estimate which systems are about to fail and should be replaced and which ones

3 IBM and others are commercially deploying highly predictive asset maintenance systems with precision on timing

and accuracy of future pipeline and electric infrastructure incidents.

7

can continue to operate. This can impact a common utility practice of “run to failure” that is used for

some non-critical systems. In addition to the operational impacts, information can also change how

utilities and regulators think about and plan capital projects. New information can improve how utilities

predict equipment lifetimes, both for installed and new equipment. This can change the timing and size

of replacing and upgrading of capital projects.

When there is little technological innovation, traditional deterministic valuation methods, such as Net

Present Value (NPV) and Cost Benefit Analysis, may be reasonable investment models, but when new

technology arises these approaches may not fully value the new opportunity or the risks. Utilities have

substantial investments in infrastructure, technology, and supporting services. New information is now

allowing utilities and regulators to better identify the real value of these assets. This new, technology

driven, information can change the assessments of capital projects as we monitor and learn about the

real cost of assets- both installed and new. New investments can provide utilities with more and better

information about the performance of the grid and can allow a utility to enhance the delivery and

reliability of the services they provide.

This paper will address the risk inherent in both paradigms: valuing maintenance investments based on

the set deterministic value or based on a value driven by the assumption that information can be used

to raise the value of an investment. This paper also looks at the role that regulation can play in assuring

that the maintenance investment strategy of utilities is aligned with customer risk preference, overall

system performance and the value of the resource being served.

Background In preparing for this paper, the authors interviewed an electric company and a water company to

understand their investment strategies regarding new technology. The fundamental question asked of

both utilities was this: Are utilities fundamentally risk-averse in their asset investment strategy? In both

instances, the answer was “it depends.” This answer frames the bulk of this paper. It depends because

utilities provide various levels of basic service. What that means is electric and water services are basic

needs for all people; electricity is needed to run the plethora of technology devices in our homes and at

work, as well as to run water delivery pumps, waste water systems, and such, and water is needed for

human survival. Each utility described three types of conditions that determine their risk tolerance for

asset investment:

1. Providing for safe and reliable service

2. Providing for service to new customers

3. Implementing or meeting policy objectives

Each utility described these three conditions in a similar manner. The utilities we spoke with placed the

provision of safe and reliable service as their primary goal. In order to meet this goal, the utilities

expressed the strategy that technology that either enhanced the level of service OR did not degrade the

level of service would receive primary consideration. Additional benefits or services that could be

8

provided by a specific piece of technology received less consideration, unless the utility determined such

information was needed at a location to enhance safety and reliability.

Next, technology that is useful in hooking up new customers would receive the second level of priority.

Again, maintaining a level of service was the primary goal of the utility, so only if the utility determined a

need for advanced technologies would such technology be installed.

Finally, meeting policy objectives set the third level of consideration regarding the use of advanced

technologies. This can be associated with handling the influx of variable electricity resources onto the

electricity grid, or the need to better monitor water quality or other water efficiency measures. This

section provides for the greatest amount of variety for consideration of advanced technologies, and is

likely where most requests for advanced technologies occur.

Technologies that could assist with any of the three conditions received consideration and an associated

risk tolerance. That does not mean, of course, that the utility automatically invested in emerging

technology just for the sake of providing safe and reliable service; technology still needed to be proven

effective, safe, and reliable, and provide positive operational benefits, over the existing models.

However, what remains unclear is to what extent is information being valued in any of the three

circumstances? For example, advanced technologies that can communicate real-time or near real-time

performance levels of transformers could satisfy the safety and reliability needs of a utility; indeed, the

recent Metcalf substation incident showed that such information can notify the utility of a

malfunctioning piece of equipment before it fails and shuts down an entire substation.4 This is where

information can be used to not only enhance safety and reliability-- but through its monitoring and

communication capability-- provide a utility with valuable information regarding performance.

We chose a water and electric company for the foundation of this analysis because each are perceived

as “behind the times” regarding adoption of advanced technology and have a substantial amount of

“legacy” assets still in operation. This perception has been borne out in a recent American Society of

Civil Engineers (ASCE) report that found “that capital spending has not been keeping pace with the

needs for water infrastructure.”5 This shortfall is leading to an increasing gap and greater reliance on

O&M budgets rather than capital projects to assure that a desired level of service is maintained. This

report graded utilities across the country and gave an overall “almost” failing grade to infrastructure

investments made by both water and electric utilities.

Utility Maintenance, Repair and Upgrade Economics In general, the objective of a utility maintenance, repair and upgrade investment plan (MRU) is to

minimize costs while maintaining service levels. Utilities do this through a series of investments, both in

4 The Metcalf substation is located in San Jose and was the location of a security incident that resulted in damage

to several large transformers at the location. See http://sanfrancisco.cbslocal.com/2013/04/16/gunshots-cause-oil-spill-at-san-jose-pge-substation/ (last accessed September 30, 2013). 5 ASCE, “Failure to Act: The Impact of Current Investment Trends in Water and Wastewater Treatment

Infrastructure,” at p. x (2011).

9

the capital budgets used to install or retrofit key elements of their network and in the operations and

maintenance budgets that assure that the installed base of equipment continues to operate effectively.

Making choices and deciding where to invest (e.g., new capital projects, better information technology,

improved maintenance equipment) requires information. It requires information about the status of

the system in general and the performance of individual components in particular. Information about

when a component might fail, the type and severity of failure, and the potential impacts of a failure are

all required to develop an effective MRU strategy. No utility actually has or can realistically hope to

have perfect information or foresight to predict precisely when and where a particular device will fail.

However, there are advancements in predictive mathematical algorithms using real time data from gas

and electric infrastructure that are moving closer to this type functionality. Currently, utilities rely on a

combination of testing regimes, inspections, and expert assessments and past experience with various

types of equipment to develop an overall estimate the status of the network. Ultimately, each utility

must make judgments about the system and determine how to prioritize maintenance activities and

where to make new replace and upgrade investments across the enterprise.

Deterministic Valuation The maintenance investment problem for utilities is embodied in the process of developing a strategy to

define enterprise wide priorities, identify specific components and systems to be serviced, and then

establish a budget to provide the expected services. These budget decisions determine which projects

to invest in, which ones to defer, and which ones to abandon. In the traditional capital budgeting

exercise, the objective is to maximize the value of an investment, such as in Net Present Value (NPV)

analysis. In this paradigm, the total value of the investment is the sum of the future discounted returns

generated by the investment minus the cost of the initial investment

Equation 1: Generic NPV equation for evaluating a project

Where is the discount rate (often the cost of capital for the utility) and is the cash flow or the return

from the investment in each future year. For an MRU project, is ostensibly the avoided cost that can

be credited to that project. An increase in overall net present value to the utility, and ratepayers, occurs

when the avoided costs exceeds the investment costs. Some common types of avoided costs that MRU

projects can capture are:

Avoiding Failures

o Minimizing unscheduled maintenance

o Maintaining an adequate level of service (SAIDI, SAIFI, MAIFI)

o Assuring the safety of the users, utility employees and the public

o Minimize systematic risks and vulnerabilities

Equipment Life cycle costs

o Minimize retiring equipment before the end of its life

o Maintenance batching

10

o Spares and replacement management

o Minimize collateral damage during repairs

To make the connection between a specific MRU investments and specific avoided cost savings can be a

challenge at the utility scale. At the project level, however, the NPV modeling paradigm can and is often

used to assess the value of a project. This of course, as already mentioned, requires an assessment of

the state of the system: both current and future. The avoided costs are then estimated as the cost

difference with and without the investment.

In the regulatory environment, the starting point for assessing the state of the system, at least in the

General Rate Case (GRC)6 context of O&M budgeting, is not project specific. Rather, a portfolio of

projects is typically considered ostensibly as a single project and historical MRU costs are used as a

proxy for the state of the network.7 The costs are then adjusted for any expected changes in demand

and other potential improvements that may be needed and a new MRU budget is updated.

Additionally, these GRCs are submitted every 3 years which may overvalue short-term gains at the

expense of a longer-term planning horizon.

Once the utility budget has been established in the regulatory process, a utility then has the

responsibility to commit to investing in specific projects. In this project by project environment, NPV is a

tool in the utilities decision making toolbox. Due to the difficulty of estimating avoided costs, it is not

clear how accurately or consistently these NPV models predict the true avoided costs. There is evidence

from non-utility focused studies that managers recognize the potential for inaccuracy in NPV models. A

National Bureau of Economic Research study found that managers will not take on projects that are just

barely net positive.8 Rather, managers often require hurdle rates significantly higher than the nominal

cost of capital would indicate.

We suggest that there are three reasons why excessive valuation may be required to actually fund

projects in utilities. The first is that the information used to develop the NPV and future cost savings

models may not be sufficiently robust. Secondly, managers may perceive that the future cost savings

estimates are not in fact fixed, as deterministic NPV models imply. Rather, managers may see that

future costs can be influenced by their actions, i.e., the NPV models do not estimate the opportunity

cost of an investment. Thirdly, decision makers may be more risk averse than is implied by a risk free or

cost of capital rate. The first two factors have merit and are difficult, but not impossible to address, the

6 The General Rate Case establishes the utility cost requirements and the rates that utilities can charge to recover

these costs. A General Rate Case is the major regulatory proceeding for California utilities, which provides the CPUC an opportunity to perform an exhaustive examination of a utility’s operations and costs. Typically performed every three years, the GRC allows the CPUC to conduct a broad and detailed review of a utility’s revenues, expenses, and investments in plant and equipment to establish an approved revenue requirement. 7 The CPUC uses a five year historical average to establish a future budget.

8 Lawrence Summers, “Investment Incentives and the Discounting of Depreciation Allowances,” The Effects of

Taxation on Capital Accumulation, M. Feldstein, ed. (National Bureau of Economic Research 1987).

11

third is a cultural issue that may be resolved through changes in regulatory incentives regarding risk

taking.

Below we discuss the information and modeling concerns in MRU NPV models and how information may

be better used by these models. In the next section we discuss how managers may implicitly value the

opportunity costs of projects in excess of the NPV value. In the regulatory section we discuss the risk

aversion of utility managers.

Current Practices in Maintenance Decision Making The acceptance of the NPV models is largely driven by the strength of evidence of the future cost

savings or lack thereof. As has been mentioned, information regarding the state of utility infrastructure

can be difficult due to its inaccessibility. This lack of access to the state of the network stems from a) no

previous operating or policy requirement to invest and b) lack of network sensors and infrastructure to

collect real time data on assets. New information technology and data management tools are becoming

more available and can provide a new opportunity to improve information quality. However, there are

still questions in any modeling context. Some of the questions that arise are: How confident are we

that the avoided costs will actually be in year i? Are all the avoided costs captured by the model? Are

some of the avoided costs double counted? What are the rewards or penalties that utility managers will

be given for under or overstating estimates? The answer to these questions requires that models

accurately reflect the operational and economic processes, and that the data supporting the models is

both sufficiently accurate and precise. To the extent that better information can resolve these

uncertainties, it can give these models additional weight and provide additional grounds for managers to

rely on NPV rules.

Essential to estimating avoided costs within the MRU context is the ability of models to predict how

systems perform and how they degrade over time. As mentioned previously, information tracking and

monitoring devices can help in this regard, but analytic methods can also play an important role.

An Australian study9 developed a simple probabilistic model to estimate the lifecycle costs of

transformers: both new and currently installed equipment. Rather than use a simple point estimate for

the expected lifetime costs of a transformer, the study developed a probability distribution based on

transformer survey for each model vintage and then reweighted expected costs based on the age

distribution of their installed base. This more robust model gave them the ability to develop a new

decision rule that could more precisely identify transformers due for replacement.

Another group is the Virginia Department of Transportation which wanted to know when to retire heavy

road maintenance vehicles.10 They developed a regression model that was designed to generate a

comprehensive estimate of the expected total average costs for new equipment and compared that to

9 G Winsor & S Buncombe, “Repair/ Replace Decision Making Practices,” ICOMS Asset Management Conference

(2007). 10

James Gillespie, “Final Report: The Replace/Repair Decision for Heavy Equipment,” Virginia Transportation Research Council, VTRC 05-R8 (November 2004).

12

the expected marginal cost for existing equipment. They were interested in finding factors and

equipment characteristics that would be indicative of higher or lower costs. Developing these cost

scores allowed them to choose a tipping point for making replacement decisions.

Several multi-national vendors have developed or are planning to release systems which accurately

predict gas pipeline leaks, transformer failure, and device level voltage failures for solar PV and other

distributed assets for example. In addition to these efforts, utilities are also developing systems for

monitoring, targeted equipment tests, and sophisticated statistical modeling techniques. However,

none of these systems, even the most ambitious, contemplate having perfect information about the

system. Physical measurements must be supplemented with expert assessment. As one interviewee

mentioned, developing priorities is a bit “art and science.”

Despite the success of these models, and even recognizing some of their limitations, the NPV approach

makes some implicit assumptions that are often not valid. In general, the NPV models assume the

returns from investments are determined by the model, that is, once the investment is made the model

assumes that the future costs flow as predicted in the model. Furthermore, the deterministic NPV

models implicitly assume that the investment decision is a now or never option. This, however, is not

often true - particularly in MRU investments. Managers have many opportunities to learn about the

state of their systems and make decisions incrementally as new information becomes available.

Through R&D, managers can also develop better test methods and technologies that may change or

resolve uncertainties. NPV models simply do not capture or value these opportunities.

Stochastic Valuation A key assertion of this paper is that a snapshot NPV approach to valuing MRU projects implicitly

undervalues potential investments in new technology. This undervaluation occurs since NPV does not

account for the implicit, and at times explicit, flexibility of managers to respond to conditions on the

ground. This undervaluation, as we will argue, may consequently provide a perverse incentive for

managers to delay projects as a means of capturing, at least in part, this unaccounted for value.11 As

was noted in an article by Frayer:

Traditional valuation procedures, such as discounted cash flow analysis (DCF) ignore the value of managerial flexibility. These models do not capture the value embodied in the plant operator’s ability to dynamically react to changing market conditions; the real options methodology measures the value inherent in such adaptability. Under a real options approach, increased uncertainty … translates into a larger options-based value.12

11

By delaying projects and gathering better information, managers have an opportunity to exceed performance expectations established by NPV models. 12

Julia Frayer, “What Is It Worth? Application of Real Options Theory to the Valuation of Generation Assets,” The Electricity Journal, Vol. 14, Issue 8 (October 2001).

13

From the manager’s perspective, having the flexibility of waiting and resolving uncertainty before

committing to an investment is one type of option that may be attractive. In the real options paradigm,

a manager has the option, which is the right but not the obligation, to exercise a project. To illustrate

the value of options in utility MRU

decision making context, we adapt a

simple example from Pindyck and

Dixit.13 Consider an MRU

investment opportunity where the

investment in upgrading a

potentially leaky pipeline costs

$1600. If the line has a “big” leak

there is a $300 per year avoided

cost of leaking water per year. If there is only a “small” leak there is only a $100 per year savings per

year (Figure2). In the NPV paradigm, the value of the leak repair project is estimated as the expected

savings $200 per year: a 50% chance at $100 and 50% chance at $300. In this case, the NPV rule (Eq.1)

indicates that the value of the investment over all future years is positive and worth $600.

Equation 2

Yet this does not consider if the managers have the option to wait 1 year at which point they could

determine if the leak is big or small. In this case, the company only invests if the leak is found to be

“big.” Now the NPV for the investment in this year includes a waiting period of a year and then a 50-50

chance of investing next year, which in the NPV paradigm is valued at $773.14

= 773 Equation 3

When managers have the option of waiting a year and resolving the uncertainty associated with the size

of the leak, the NPV value raises to $773. Another way to write the total value of an investment

opportunity in year 0 is:

The extent to which utilities have options to alter their MRU investment decision implies that there is

potential value to be captured above the NPV estimated value - in the case above, the holder of the

option has an additional $173 in value.

Options and the Value of information A key tool that utilities could employ to manage the MRU investment challenge is to use a real options

approach. The real options paradigm is based on methods used widely throughout the financial sector.

13

Avinish Dixit, R. Pindyck, Investment Under Uncertainty, Princeton University Press (1994). 14

Waiting incurs other risks, as explained below.

Figure 1 Investment in pipeline leak loss prevention

14

In our discussions we found that utilities, at least implicitly, exploit the timing/delay option particularly

in non-critical assets. The irreversible nature of many investments, the lack of competition, as well as

the potentially very long lifetime of utility equipment seems to provide an incentive for MRU managers

to be highly risk averse and delay investments, if possible, to the very last moment.

We also found that the utilities do not seem to have a robust strategy to take on new risks in the MRU

context. Utilities do, of course, continually consider investing in new technologies that address these

emerging needs, but typically they use the static NPV-like rules to assess these technologies. While the

testing of the devices provides the basis for a benefit analysis, it is not clear that the additional value of

information and how that information can be used to resolve uncertainty and risk is part of the benefit

analysis. Indeed, without such information the electric and water utilities may not be effectively

utilizing their investment dollars. The ability to obtain, access, understand and use information

generated from advanced technologies may allow water companies to better assess the robustness of

their infrastructure and plan and prioritize based on array of possible future needs rather than on an

NPV snapshot version of the future.

Strategic Management of Real Options Portfolios As has been mentioned, the current methodology for evaluating MRU investments has been structured

around an historical review of costs. We believe, however, that this methodology is becoming

increasingly ineffective. The driver behind this change in effectiveness is change itself. These changes

and uncertainties include new types of energy and water supply sources, unknown impacts due to a

chronically aging infrastructure, revolutionary set of utility focused information technology, new market

structures and needs, and grid reliability threats due to environmental factors. These uncertainties are

also compounded by the fact that these uncertainties are often linked. For example, the risks to

transmission and distribution infrastructure from a wildfire may be more acute for an aging and less

resilient infrastructure than a more modern and robust infrastructure. An historical accounting of needs

simply cannot adequately develop insights and accurate cost and performance estimates in the face of

these growing and linked uncertainties.

This uncertainty (i.e., volatility in financial parlance) also comes with a high price. Uncertainty in itself

can lead to higher costs if only because managers increase the contingencies or reserves as a means of

managing uncertainty. Another way uncertainty can raise costs is when managers and regulators try to

“buy out of it” by locking in technical requirements or committing to large scale and high capital cost

projects. While this may be effective if the technical solution turns out as advertised, it can also be a

costly mistake if the technology underperforms. Additionally, without a clear map of the linked

uncertainties in large scale projects, unexpected consequences can also become an additional cost.

Uncertainty can also create yet another incentive to “overbuild” the system, a well understood but very

expensive mechanism to increase the high reserve margins and distribution hardware that places

financial pressure on ratepayers. Another hidden cost is the fact that some technologies that may

perform well in isolation may not turn out to be valuable when integrated in a utility.

We are not suggesting that big, bold or risky projects should not be undertaken. To the contrary, we

think that many of those types of projects can be quite valuable to utilities and customers. Rather, we

15

suggest that projects should not only be valued based on its current expectations of value to a utility

(the NPV rule), but also should be valued on the options that they present to the utilities.

The financial sector has already discovered that the option value of investments can be quite high and

typically increases significantly as either the uncertainty or cost of an investment increases. This is most

notably demonstrated by the Black-Scholes equation which estimates the value of a financial option

based on a measure of volatility and the cost of an investment.15

It seems intuitive that a project with managerial flexibility, i.e., real options, should also be more

valuable than projects without flexibility. What is unclear, however, is how this nebulous concept of

flexibility translates into real and quantifiable benefits for a utility, its investors, and its customers.

Capturing the additional benefits (e.g., decreased operational cost, lowered service outage rates, etc.)

from an option focused investment strategy, instead of an NPV maximizing strategy, requires that

utilities and regulators first develop a coherent strategy for evaluating the future.

Developing a future focused investment strategy is directed more by how managerial options can

resolve and respond to uncertainties. Rather than the rigid NPV model paradigm, where managers take

the uncertainties as they unfold, an options paradigm assumes more proactive managers.

Below we discuss three fundamental types of options that can be used by utilities: 1) timing investments, 2) expand investments, or 3) abandon investments.

1) Timing Option

The timing option is the most obvious course for utilities. With little or no competition, delaying MRU

investments provides utilities the ability to resolve uncertainties (e.g., big leak or small leak) before

making irreversible commitments to projects that may turn out to be unsuccessful. It also allows

utilities to make other, less risky investments while they wait. A significant problem arises here, because

utilities cannot delay indefinitely - at some point systems will fail. A utility’s decision comes down to

how long the utility wants to wait to ascertain the status of a system component. Regulators must be

aware of this incentive for utilities to delay and respond by developing a process for balancing the

performance of the network while also allowing for discovery and innovation. The consequence for not

recognizing the incentive to delay repairs and maintenance may be that systems will begin to age in

place, as has already been noted.16

2) Expand Option

Utilities may also find it valuable to expand projects even when the NPV approach shows a negative

value. The expansion option is desirable when a current project provides utilities with the ability to

make additional future investments that would not be possible absent the current investment. This

15

See, David Luenberger, Investment Science, Oxford University Press, at Chapter 13 (1998) (contains definition of Black Sholes option pricing formula). 16

There may be other financial or cash flow benefits to waiting or investing in other assets instead of investing in infrastructure maintenance or repairs.

16

option is perhaps less intuitive than the delay option, but can nevertheless have significant impacts on

the long term performance of a firm. As noted in Grenadier:

During the 1970s some manufacturing firms invested in automatic and electronically controlled machine tools. Returns on the initial investment were reported to be modest. However, microprocessor-based technologies arrived in the early 1980s, bringing about the opportunity for much more dramatic returns (greater performance at lower cost). Firms that had previously invested in electronically controlled machine tools were able to migrate quickly and cheaply to the new technology. Because operators, maintenance personnel, and process engineers were already comfortable with electronic technology, it was relatively simple to retrofit existing machines with powerful microelectronics. Companies that had deferred investment in electronically controlled machine tools quickly fell behind.17

This example points out another problem with the NPV rule as it relates to research and development

(R&D) and pilot projects: as new projects are inherently uncertain, it is difficult to justify them given the

potentially high failure rate. In addition, the static NPV rule does not include the practical value gained

even in failure. This value occurs because projects are rarely complete failures or complete successes.

Even in under-performing projects there are still some benefits generated by resolving uncertainties,

expanding the technical base of a firm, or developing new skill sets among employees. The NPV rule

simply does not give any value to the expanded potential for future investments given this enhanced

technical position of the firm, while an option based approach explicitly requires it.

3) Abandon Option

Managers clearly can and do abandon projects when they are not successful. The option to abandon a

project is valuable in itself since losses can be limited when projects do not payoff as expected. Having

this option can also provide an incentive to utilities to take on more risky projects since their downside

losses can be limited. When projects go south, the NPV rule may encourage managers and regulators to

adhere to prior decision instead of thinking about past investments as sunk costs, thereby causing

utilities to continue to operate subpar performing assets longer than financially or operationally

reasonable.

Barriers to Valuing Real Options When evaluating investment decisions in the MRU context, including the value of managerial flexibility

can make a significant difference. In many cases it can flip a decision from a “no-go” decision to a “go”

decision. This may be true particularly when uncertainties and investment costs are both high – as is the

case for many newer technologies - simply relying on NPV-like assessments can significantly undervalue

17

Steven Grenadier, A. Weiss, “Investment in Technological Innovations: An Option Pricing Approach,” Journal of Financial Economics, Vol. 44, at 398 (1997)

17

projects and lead to many missed opportunities. Forgoing these projects represents a loss in value to

utilities, their investors, and to the customers, as well as a loss in the general welfare of the public.

Utility Utility operations managers are usually tasked with establishing priorities in the MRU context. In our

discussion with the utilities, it was clear that keeping the lights on and the water flowing were the

underlying priorities and risk reduction was at the core of the managers objectives. The value of

developing new processes and installing new technology was seen in the context of how it could reduce

risk within the scope of their planning horizon and authority. As a result, the ability to value new

information enabled by advanced technology may be limited by a utility’s planning outlook. This can be

seen in several ways: 1) all else being equal, the manager has little incentive to invest in current or new

technology, since the payoff of new technology is likely beyond their planning horizon; 2) maintaining

current technology due to the familiarity or comfort with the technology is a low risk option; 3) taking

risks regarding newer technology may negatively impact reliability metrics within the scope of their

budget cycle; 4) historical budget constraints are biased towards using the current technology which

cost less than newer technology in the short run; and 5) the manager and employees may lack the

appropriate level of training to use the new technology. Each of these issues essentially undervalues

information and experimentation with advanced technology, and may unwittingly be perpetuating

unidentified or unknown risks to the operation of the grid.18

Understanding and valuing the capabilities of new technology is easier said than done. Managers, like

everyone else, have different comfort levels regarding adoption of new technology. Limited pilots can

help with this acceptance, but allowing and encouraging managers to trial these new technologies under

a variety of situations can both provide much needed real world experience with the technology as well

as provide managers an opportunity to have some experience working with the technology. Without

enabling these real world trials, breaking down this risk aversion at the local level will continue to be a

challenge, and may delay the eventual adoption of advanced technology.19

Regulatory Just as often, the regulator may act as the barrier to utility investment in advanced technologies. While

the Commission has encouraged and supported the investment in some advanced technologies, such as

Advanced Metering Infrastructure (AMI), it is not as simple with other technologies. Recently, the

Commission rejected providing SDG&E with full funding for several Smart Grid-related projects, siding

18

A related barrier is the GRC cycle itself which relies on a three year budgeting cycle; the extent to which that cycle may also provide a disincentive for longer term planning horizons in favor of shorter term costs is examined below. 19

There is an analog to this problem in cybersecurity where utilities manage cybersecurity to be compliant with standards and not necessarily to the actual risk. Recognizing the risks with managing cybersecurity protocols to meet compliance standards, the industry is moving towards a risk-based approach where utilities manage cybersecurity protocols to address the risks. See, e.g., “Electricity Subsector Cybersecurity Capability Maturity Model,” Department of Energy (May 31, 2012); “Electricity Subsector Cybersecurity Risk Management Process,” Department of Energy (May 2012); and “Guidelines for Smart Grid Cyber Security,” The Smart Grid Interoperability Panel- Cyber Security Working Group, NIST Intergovernmental Report 7628 (August 2011).

18

with the consumer advocates that the technologies were too risky and benefits were unknown.20 The

riskiness and unknowns about future needs played a role in determining appropriate funding levels for

these investments. Other states have made similar pronouncements on a number of other advanced

technologies, including AMI.21 Oftentimes, a regulatory determination is made primarily on a cost-

effectiveness or NPV-like rule determination. Unfortunately, the nature of these new technologies does

not readily submit to a simple NPV rule evaluation, nor do they fit in a 3 year GRC planning cycle. As we

have discussed, the NPV rule consistently undervalues investments, particularly when investment

uncertainty and costs are high. Some examples of undervaluation include; the cost savings from AMI

attributable to conservation that results from responding to AMI information such as more time

sensitive price data, or operational efficiencies from predictive modeling of future demands such as

electric vehicle. Attempting to value advanced technologies based on time static and simple NPV tests

quite frankly misses that vast bulk of the opportunity that these technologies offer.

Regulatory risk is present in all utility programs, but infrastructure investments pose challenges to this

model. Since many infrastructure investments have costly upfront expenses, ensuring that the utility is

including all the potential value in technological risks is important and should be understood and

encouraged by regulators. Encouraging utilities to consider, test, and plan to invest in new technologies

at the urging of a PUC, then rejecting the funding for those programs is not productive in building the

grid, or water system, of the future. None of this is to say that regulators should provide a blank check

to utilities to test any new technology; rather, allowing the utility to manage a risk portfolio,

understanding that some products will fail and others will succeed, will lead to better utility planning

and execution, as well as provide the utility better opportunities to try out new technologies in a more

responsible manner.22 These trials also can be a proving ground for new technologies and provide for a

clearer road map for future investments and needs. Regulators should not hide from technology or

failures of technologies, for without these trials and errors we lose the ability to improve.

Current Metrics The ability to measure performance of the system is critical to making wise MRU investment decisions.

Metrics of technical and operational performance provide regulators, suppliers, utilities, and the public

with a basic understanding of the quality of service provided to customers. It is imperative, however, to

have metrics that are understandable, usable, actionable, and meaningful in order to be useful. Indeed,

without metrics it is difficult to understand how successful, or unsuccessful, utility investments in

20

See, D.13-05-010 at 100-101, 132, 208-232 (May 14, 2013). See also, D.12-11-051 at 118 and 129 (November 29, 2012) (decision on Southern California Edison Company GRC reducing funding for several Smart Grid projects). 21

See, e.g., Proceeding on Motion of the Commission to Consider Regulatory Policies Regarding Smart Grid Systems and the Modernization of the Electric Grid, “Smart Grid Policy Statement,” New York Public Service Commission, Case 10-E-0285, at 30-31 (August 19, 2011) (finding that “[m]any new smart grid technologies are still expensive, and the risks associated with such technologies can be significant. We believe it is most appropriate to start with technology investments that can provide a relatively certain return, and continue to evaluate technologies whose potential benefits remain uncertain, particularly for technologies such as smart meters that are dependent on intensive customer engagement of behavior changes to produce those benefits.”). 22

Allowing this portfolio approach should also span beyond a single GRC cycle precisely because needs and priorities may change over time; flexibility itself should be considered valuable.

19

operations have been. The conventional way to evaluate investments was based on historical cost

benchmarks along with an NPV-like analysis, but when the option value of investments is included,

information quality becomes more important. A by-product of including real options in investment

decisions is that uncertainties can be identified and resolved more effectively than with a simple NPV-

like investment rule. Detailed metrics with sufficient resolution and granularity are an important

prerequisite for determining if uncertainties have in fact been resolved. Determining which

uncertainties can be resolved by which technologies will be increasingly difficult in the increasingly

dynamic and complex utility environment. New systems and technologies will have different types of

impacts on the network and understanding how these technologies influence performance is critical to

understanding the effectiveness of the new technical options. The ability of our current set of metrics to

provide the requisite level of information to assess new investments should be reviewed as new

technology can provide more detailed information about the performance of the grid, network, and

associated investments.

1) Electric service

In 1996, the Commission issued D.96-09-045 which adopted reliability reporting requirements for the

electric utilities in California under Commission jurisdiction. That decision adopted SAIDI, SAIFI, and

MAIFI as the three metrics by which the Commission (and the public) can measure how reliable utility

service is for the utilities. In adopting these standards, the Commission noted that

Uniform and measurable system standards are an important first step in defining reliability. They will help us fulfill our commitment to preserving reliability at levels that we have previously accepted as reasonable. In the absence of any statute or Commission rule defining statutorily acceptable performance measures of reliability, and recognizing that reliability is strongly tied to costs, we refer to "statutory reliability" as those levels of reliability historically accepted as reasonable, as measured by indices then in use at the time. It is our goal to encourage improvement in this area.23

Although this decision was adopted at the beginning of California’s experiment with restructuring and

retail choice, many of the same concerns raised in that decision are still valid today, notably that costs

associated with reliability may be getting short shrift as utilities (as directed by state law and

Commission policy) focus on other areas of service.24

The metrics adopted by the Commission are the three most commonly identified metrics in the

electricity industry for measuring the reliability of the distribution grid. These metrics, defined below,

23

D.96-09-045 at 11-12 (July 21, 2001). It is also worth noting that the Decision identified that the adopted reliability measures may act as a floor and provided details regarding a performance-based ratemaking mechanism that was to counter the utility desire to reduce costs. 24

Indeed, D.96-09-045 also noted that “[a]lthough system measures of reliability may give us these means for holding utilities accountable to measurable criteria, satisfaction of system measures is not a shield … System measures may mask more localized problems, and the utility may still be found to have acted unreasonably with respect to maintenance or replacement of some portion of the system.” D.96-09-045 at 12.

20

form the basis of our understanding of the distribution grid. However, these metrics really only tell part

of the story as advanced technology has the capability to provide greater visibility down into the

distribution grid.

SAIDI is defined as follows:

The total minutes of sustained customer interruption divided by the total number of customers, expressed in minutes per year. It may be expressed in smaller time periods (month or quarter) or smaller portions of the system (region or circuit) upon request. It characterizes the average length of time customers were without power during the time period.

SAIFI is defined as follows:

The total number of sustained customer interruptions divided by the total number of customers, expressed in interruptions per customer per year. It may be expressed in smaller time periods (month or quarter) or smaller portions of the system (region or circuit) upon request. It characterizes the average number of sustained power interruptions for each customer during the time period.

MAIFI is defined as:

The total number of momentary customer interruptions divided by the total number of customers, expressed as momentary interruptions per customer per year. It may be expressed in smaller time periods (month or quarter) or smaller portions of the system (region or circuit) upon request. It characterizes the average number of momentary power interruptions for each customer during the time period.

2) Water Service

Unlike with electricity service, service or reliability metrics for water companies are less developed. For

the most part, metrics associated with water tend to focus more on quality than on reliability. However,

there are other performance measurements that could be used to better monitor water service since

water quality, while very important, fails to address service quality. For example, advanced metering

infrastructure (AMI) could be used by a water company to gain insight in water loss through the

network, especially if used in conjunction with other information. AMI can also support better billing

practices, provide customers with more detailed information about their usage, and support new,

innovative rates for customers. Most importantly, AMI can be used to reduce non-revenue water losses

and associated delivery costs. Indeed, many municipal water utilities cite these savings as support for a

water AMI use case.25

25

See, e.g., City of Albuquerque and Bernalillo County (http://www.publicworks.com/doc/albuquerque-bernalillo-county-water-utility-0001), City of Dubuque (http://www.cityofdubuque.org/DocumentCenter/Home/View/3116), and City of San Francisco (http://www.sfwater.org/index.aspx?page=51).

21

Clearly, advanced technology provides utilities and the regulator with the potential for a new and vast

array of metrics and an understanding of the utility’s system. Yet, advanced technologies, like AMI for

water, is typically considered expensive and not worth the cost primarily because water can be stored,

has less “peak/off peak” issues compared to electricity, and water has fewer reliability concerns than

electricity. However, service quality and operational savings still matter, and should be measured. For

example, advanced technologies have the ability to help water utilities reduce non-revenue water

losses, which provide direct savings to customers. Information can be used to help reduce leaks as well

as assist in identifying more leaky pipes or homes, which can provide significant cost savings to the

utility and the customer.

A second potential measurement that can be implemented would be water pressure at certain points

throughout the network. Assuring sufficient water pressure at a location or premise is a basic quality of

service responsibility for water utility; in much the same way as an electric utility ensures a level of

voltage at each premise. Indeed, electric meters can (and do) measure voltage, as well as current; it is

expected that advanced water utilities will be able to measure water pressure at a premise.

Fundamentally, adequate and accurate metrics for service and quality require additional information

which is currently unavailable, not for lack of technology, but, rather, for lack of investment and

installation across the service territories of the utilities. Until better information can be obtained, any

measurements must, by necessity, be at the macro level, which may help with overall system

performance, but not necessarily at the customer level.

Conclusion Maintaining critical electricity and water infrastructure is of tremendous value. Reliable electric and

water services establishes a fundamental base for the economy, but an expansion of our population,

changes to our environment, and an explosion of new technology are presenting new threats and

opportunities to utilities. The challenge we now face is how to leverage new technologies in order to

mitigate these threats. This new uncertainty for industries that have been weaned in an environment of

certainty may be a challenge. It will require that both utilities and regulators think about risk in new

ways. What risks should be avoided and which ones should be taken?

Fortunately methods for managing new and emerging risks already exist. The financial sector has

developed a rich science around risk management and financial options. There is also an emerging field

of real options that can be applied to utilities; adopting these new methods will require a new way of

thinking and acceptance of some failures in the process. While there are certain financial risks

associated with investments in new technologies, some level of failures should be expected. Failure is

part of learning; when one falls down, one must get back up. Without some amount of failure, the

utility may not fully understand what makes an investment succeed. This ability to learn from ones

failure is an important and necessary step in the utilities’ technological modernization. That is, of

course, not to say that utilities should be investing in every technology that comes its way; utilities and

regulators must continue to be good stewards of customers and shareholders money, but all parties

should be willing to accept that sometimes an apparently prudent investment in technology does not

22

work out as anticipated. Understanding why something failed can then allow the utility to learn from

that experience, understand where the technology failed, and move forward. The use of NPV methods

tends to undervalue the usefulness of data and information that could be gained from the adoption of

new technologies. For example, as discussed above, while NPV methods may tend to discourage

investments in R&D, real options valuation encourages R&D by capturing the intangible benefits of R&D

to a firm.

Learning to accept failures, and recognizing the value of developing real options and robust strategies, is

usually frowned upon by regulated utilities but can be tremendously valuable in a changing

environment.

Recommendations The main assertion of this paper is that utilities have a dis-incentive to take risks and proactively develop

a robust and technologically advanced network. It is our belief that this dis-incentive derives from the

fact that utilities possess an option value that regulators do not recognize or measure. As a

consequence, utilities have an opportunity to capture part of this value by adjusting the timing of

investments. In many cases, utilities do not in fact have these options, e.g., in critical systems, but to

the extent they do have them and can exercise them represents an untapped opportunity to improve

the overall system. In order to optimally utilize this potential value we make several recommendations:

In their GRCs, utilities should explain how they value information and new technology as part of

their overall investment strategy, including defining the likelihood of an substantial impact event

occurring and the consequence of it happening

In their GRCs, utilities should also explain how they may use advanced technologies, either

already in the field or proposed to further reduce the risk of a substantial event on their grid or

system.

In their GRCs, utilities should describe the methodology they use to prioritize risks and

technology, and based on that methodology, explain how far along they are in implementing the

prioritization.

The Commission should develop more effective metrics for water and electric grid that can be

used to assess the impact of new technologies. Identify systems as critical or non –critical. For

example, the Commission should develop new metrics for a “risk based” portfolio that allows

utilities to outline specific new technologies they would like to value but do not due to current

constraints.

The Commission should re-assess the budgeting process and rates of return to reflect the

assumption that utilities have an accounted for option value built into some budgets. This may

also include allowing for budgets to span across GRC cycles.

23

References American Society of Civil Engineers (ASCE), “Failure to ACT: The economic impact of current investment

trends in water and wastewater treatment infrastructure,” 2011.

Allenman, J., “A New View of Telecommunications Economics,” Telecommunications Policy, Volume 26,

2002.

Allenman, J.Rappoport, P., “Regulatory Failure: Time for a New Policy Paradigm,” Communications &

Strategies, No. 60, 4th quarter, 2005.

Bellahah, M., “Real Options with Information Costs: A synthesis,” International Journal of Business,

Volume 12(2), Spring, 2007.

Dayanik, S. Gurler, U., “An Adaptive Bayesian Replacement Policy with Minimal Repair,” Operations

Research, Volume 50(3), 2002.

Dixit, A. Pindyck, R., “The Options Approach to Capital Investment,” Harvard Business Review, May-June,

1995.

Durango-Cohen, P., “Maintenance and Repair Decision Making for Infrastructure Faculties without a

deterioration Model,” Journal of Infrastructure System, March, 2004.

Frayer, J. Uludere, N., “What is it Worth? Application of Real Options Theory to the Valuation of

Generation Assets,” The Electricity Journal, Volume 14(8), 2001.

Gillespie, J., “The Repair Replace Decision for Heavy Equipment,” The Virginia Transportation Research

Council, VTRC 05-R8, Nov 2004.

Gollier, C.Proult, D, et al., “Choice of nuclear power plant investments under price uncertainty: Valuing

modularity,” Energy Economics, Volume 27, 2005.

Grenadier, S. Weiss, A., “Investment in technological innovations: An options pricing approach,” Journal

of Financial Economics, Volume 44, 1997.

Haddad, G. Sandborn, P. et al., “Using Real Options to Manage Condition-Based Maintenance Enabled

by PHM,” Center for advanced Lifecycle engineering - University of Maryland, 2011.

Luehrman, T., “Investment Opportunities as Real Options: Getting Started on the Numbers,” Harvard

Business Review, July-Aug, 1998.

Luehrman, T., “Strategy as a Portfolio of Real Options,” Harvard Business Review, Sep-Oct, 1998.

Luenberger, D., “Investment Science,” Oxford University Press, 1998.

24

Schellenberg, J., “Evaluating the Total Cost of Outages” Presentation to IEEE Power & Energy Society

Meeting, July 2012.

Sullivan, M. Mercurio, Shellenberg, J., “Estimated value of Service Reliability for Electric Utility

Customers in the United States,” LBNL-2132E, June 2009.

Sullivan, M. Mercurio, et al., “How to Estimate the Value of Service Reliability Improvements,” Power

and Energy Society General Meeting IEEE, July 2010.

Summers, L., “Investment Incentives and Discounting of Depreciation Allowances,” The Effects of

Taxation on Capital Accumulation, NBER books ed. Feldstein, M. 1987.

Tsui, M., “Valuing Innovative Technology R&D as a Real Options: Application to Fuel cell vehicles”

(Master’s thesis, MIT, 2005).

Wang, T., “Real options “in” Projects and Systems Design Identifications of Options and Solution Path

dependency” (Ph.D Thesis, MIT, May 2005).

Winsor, G. & Buncombe, S., “Repair/Replace Decisions Making Practices,” ICOMS Asset Management

Conference, 2007.