VFM GUIDE WEB.pdf

8/10/2019 VFM GUIDE WEB.pdf

1/23


2/23

ASSESSING VALUE FOR MONEY: A GUIDE TO INFRASTRUCTURE ONTARIOS METHODO LOG Y- PAGE 2 -

Queens Printer for Ontario, 2007

This is a technical document. Definitions for many concepts andterms may be found in the glossary at the end.


3/23


Table of Contents

Introd uction .........................................................................................................................................................................4 How is Value for Money Estimated? ................................................................................................................................5 Estimating the C ost Components....................................................................................................................................6 Bringing it all Together to Calculate Value for Money...............................................................................................14 More on Notiona l Public Sec tor Financ ing Rate and Discounting ..........................................................................15 More on Risk Quantification and Statistica l Simulation ..............................................................................................16 Final Comments ................................................................................................................................................................18 Glossary .............................................................................................................................................................................. 19 Appendix: Deloitte Letter ................................................................................................................................................21


4/23


Introduction

O BJEC TIV E OF THE GUIDE

Infrastructure Ontario (IO) delivers publicinfrastructure projec ts using a projec t de liverymodel called Alternative Financing andProc urement (AFP). This document provides thedetailed methodology by which IO d etermines ifvalue for money is demonstrable by procuring aprojec t using AFP. A value for money analysisconsists of a compa rison between the total costs ofde livering a n infrastructure projec t using thetrad itional public sec tor projec t procurementmodel and AFP.

TYPES O F AFP PROJECTS DELIV ERED BY IO Currently IO uses several AFP models to d eliverprojects. These are the Build-Financ e (BF), Build-Finance-Maintain (BFM), Design-Build-Finance (DBF),and Design-Build-Finance-Maintain (DBFM) models.

The current list of projects assigned to IO consists ofbuildings of various types (hospitals, courts, etc.).

The methodology set out in this guide thereforerelates to BF, BFM, DBF and DBFM buildings.

WHA T IS V A LUE FO R M O NEY? In simple terms, a value for money (VFM) analysisrefers to the proc ess of developing a nd compa ringthe total projec t costs, expressed in dollarsmeasured at the same point in time, related to thefollowing:

1. Traditional Projec t Delivery: Estimated costs tothe public sector of delivering an infrastructureproject using traditional procurementprocesses (under which total estimated costsare known as the pub l ic sec to r co m pa ra to r, o rPSC ), and

2. Alternative Financing and Procurement:

Estimated costs to the public sector ofdelivering the same projec t to the identicalspecifications using AFP (under which totalestimated costs are known as the ad jus t edsha do w b id, or ASB ).

The difference between the public sec torcomparator and the adjusted shadow bid isreferred to as the value for money. If the adjustedshadow bid is less than the public sector

comparator, there is positive value for money byprocuring a project using AFP.

WHEN A ND HO W IS VA LUE FOR MO NEY ASSESSSM ENT

USED?

STAGE #1 - Authorization to release the Request forProposal (RFP)

The release of all RFPs by IO must be approved byits Boa rd of Direc tors. The IO Boa rd does notapprove release of an RFP unless, among otherfactors, positive VFM is demonstrated by procuringa project using AFP.

STAG E # 2 - Auth or iza t ion to ente r in to the Projec t

A g r e e m e n t

Upon c lose of the RFP process, bids are evaluatedby an evaluation committee. The preferred bid isthen compared to the public sector comparatorand presented to the IO Board of Directors. At thispoint the PSC is updated to reflec t the most currentcost information. Again, the IO Board of Directorswill not approve proc eeding with AFP procurementunless positive VFM is demonstrable using AFP.

STAG E # 3 - Pub lica t ion of the va lue for mo ney

a na lysis

After the projec t agreement has been finalized, IOreleases a public report that contains the final VFManalysis, along with details on the projec t, theprocurement process and the project agreement.

The objective of the report is to provide the publicand others with an understanding of the projectand the basis for the dec ision to deliver the projec tvia AFP.


5/23


How is Value for MoneyEstimated?

The value for money analysis is prepa red inaccordance with the methodology detailed in thisdocument by an external advisory firm withrelevant experience, such as a professional servicesfirm, in collaboration with IO management forpresentation to the IO Board of Directors.

The VFM assessment is based on detailed projec tspecific information from multiple stakeholders. Thisinformation is used to develop two financial models:

Model #1 Public Sec torComparator (PSC)

Model #2 Adjusted Shadow Bid(ASB)

Total estimated costs tothe public sector ofdelivering aninfrastructure projec tusing traditionalprocurement processes

Total estimated costs tothe public sector ofdelivering the sameproject to the identicalspecifications using AFP

The difference between the estimated total projectcosts under each model is the VFM.

In the illustrative BF VFM analysis (figure #1), the PSCis shown as the stacked bar on the left of the graphand the ASB is shown as the right hand ba r. Bothare expressed in terms of dollars measured at thesame point in time.

Figure #1Illustrative BF VFM ($s millions):

=$62.0=$60.0

=$17.0=$15.0

=$7.0=$20.0

=$14.0

=$12.0

$0

$20

$40

$60

$80

$100

Pub lic Se cto r C o mp a ra to r ( "PSC ") Adju ste d Sh ad ow Bid ("ASB")

Ba se Co sts Fina nc ing C osts Reta ine d Risks Anc illa ry C osts

A: Base Costs A: Base Costs

(includingPremium)

} Value for Money = $7.0

B: Financing CostsB: Financing Costs

C: Retained RisksC: Retained Risks

D: Ancillary Costs

D: Ancillary Costs

Total PSC = $107.0 Total ASB = $100.0

The comparative cost components will vary slightlyin magnitude between the two procurementmethods (as shown by the coloured segments inthe figure above). The difference between theestimated total project costs is the VFM and isca lculated as:

Sample Value for Money Calculation

(Traditional Projec t C osts) - (AFP Project C osts) _____________________________________________________________________________________________________

(Traditiona l Projec t Costs)

=

(Tota l PSC ) - (Total A SB) _____________________________________________________

(Tota l PSC )

=

$107.0 - $100.0 ____________________________________

$107.0

=

$7.0 _______________

$107.0

= 0.0654

Stated in percentage terms, the VFM for the sampleproject above is an estimated 6.5%


6/23


Under AFP, the estimated base costs (includingpremium) and the estimated financing costs aretogether known as the shadow bid . It is when theother cost components such as retained risks and

ancillary costs are added to the shadow bid thatone arrives at the adjusted shadow bid (i.e.adjusted for risks and ancillary costs).

When bids are received, they consist of base costs(including premium) and financing costs (i.e.exac tly the same components as the shadow b id).When reassessing VFM at this stage, the shadow bidis replaced by the preferred bid, and adjustmentsare once again mad e for risk and a ncillary costs toarrive at the adjusted preferred bid. This adjustedpreferred bid is compared to the updated PSC.

The following sec tions will detail the methodologythat is followed on IO projects to develop each ofthe cost components that make up the PSC andASB, leading to the VFM c alculation. The c ostcomponents in the VFM analysis include only theAFP portions of the project costs. Non AFP relatedproject costs, such as land acquisition costs, thatwould be the same irrespective of the deliverymethod are excluded from the VFM calculation.

Estimating the Cost Components

The method of calculating each of the individualcost components is as follows.

A. BASE COSTS and PREMIUM

Figure #2

=$62.0=$60.0

$0

$20

$40

$60

$80

$100

Pub lic Se c to r C o mp a ra to r ( "PSC ") Adju ste d Sh ad o w Bid ( "ASB")

A: Base Costs

A: Base Costs(including Premium)



D: Ancillary Costs

D: Ancillary Costs

Premium

The estimate of base costs generally includes: construction c osts; lifecycle costs 1; hard and soft fac ility management costs 1; and premium (under ASB only)

Unless there a re compelling rea sons to a ssumeotherwise, the base costs under the PSC and theASB should be assumed to be identical as a startingpoint, with the exception that the AFP base costswill include a premium (as shown by the dotted linein Figure #2). This premium is included by theprivate-sector pa rty as compensation for theadded risks transferred to them under the A FPcontract.

Where clear base c ost advantages to proceedingusing AFP can be anticipated or are revealed inbids, they should be factored into the VFM Analysis.A couple of examples may serve to illustrate. Were

1 Applicable only in projects which include a post-construction maintenance phase


7/23


a hospital to consider putting out to tender a fullydesigned addition project, it might rightfullydetermine that there were few opportunities forinnovation in the actual construction of the project.

Consequently, the ba se construction c osts couldbe assumed to be identical under trad itional andAFP delivery. Unless the bids contradicted thisassumption, it would underlie the VFM analysisthroughout the process. Conversely, were ahospital to c onsider putting o ut to tender a projec tfor design, build and maintain, it might assume thatthere were opportunities for innovation andefficiency in the delivery and operation of thefacility under AFP. However, prior to receipt of bids,it might conservatively determine not to factorthese into the analysis. Once bids were received, if

there were clear bases for using different base costsfor the PSC and ASB, such as unanticipated designinnovations, construction techniques or operatingexpenses, the base costs would be different underthe PSC and ASB.

Prior to the receipt of bids, the best means ofestimating base project costs is to have professionalindustry experts estimate the:

Cost element External Source of DataConstruction Costs Construction cost consultantLifecycle Costs Lifecycle cost consultantFacility Ma nagementCosts

Facilities management costconsultant

The relevant co st consultants also estimate, basedon their industry knowledge and expertise, themagnitude of any premium in respec t of riskstransferred in connection with the goods or servicesbeing proc ured . The premium, if any, will varydepending upon the degree of risk transfer, as wellas market conditions.

The illustrative VFM shows base co sts under PSC of$60.0 million, whereas the ba se costs under AFP are$62.0 million. The cost of $62.0 million under AFPrepresents $60.0 million for base costs, plus a $2.0million premium to account for the risks that thepublic sector has transferred to the private sector.

For the VFM upda te at the preferred bid stage(stage #2), it is IOs policy to update the PSC withthe ba se costs (after stripping out the embedded

private-sec tor pa rty premium) contained in thepreferred bid. Recall that the preferred bid isbroken into base costs (including premium) andfinanc ing c osts. However, the actual bid does nottypically break down the base c osts (includingpremium) into base costs and premium separately.It is thus nec essary to make an assumption aboutthe magnitude of this embedded premium in theobserved costs and so extract it to arrive at theba se c osts to use for the PSC. Unless there is clear

justification to believe that the percentagemagnitude of the risk premium has changed since

the relevant cost consultant first estimated it, theoriginal assumption about the magnitude of the riskpremium (in percentage terms) is used to c alculateand back out the premium embedded in theprivate-sector parties bid. The resulting base costinformation is then used to update the base costs ofthe public sector comparator while the actualpreferred bid information is used to update thecosts for AFP procurement. Upda ting the PSC in thismanner is consistent with the principle o f using thebest cost information available. Information taken

from actual bids is considered the most ac curate,up-to-da te market information available, especiallyin light of rapidly changing market pricing.

Competitive NeutralityIn certain instances, the base costs under AFPdelivery will include a provision for certain taxes andinsurance premiums. The equivalent costs will notappear under the PSC, as the public sec tor may beexempt from paying certain taxes and may selfinsure. These perceived cost advantages couldbe misleading. As a result, an adjustment called the

competitive neutrality ad justment is required tonegate this potentially misleading cost of AFPdelivery. The adjustment consists of a dding suchcosts to the PSC. The perceived advantages aremisleading because taxes are costs that ultimatelyresult in revenues to the public. It might be possibleto distinguish among the va rious levels of


8/23


government to whom taxes are paid, so that taxespaid to the Government of Canada were trea teddifferently from provincial taxes. It is IO policy not todraw such a distinction on the basis that tax

revenues paid to the Government of Canada alsobenefit the Province. A similar adjustment isrequired in respec t of insurance. When thegovernment chooses to self-insure, there is aperception that the government has saved oninsurance premiums. In fac t, the government istaking on risks otherwise covered by insurance, andthe government should account for this additionalrisk. An adjustment is made to the PSC by addingan amount equivalent to the premium otherwisepaid by the private sector under AFP to account forthe additional risks.

InnovationAssuming the same base c osts under the PSC andthe ASB (with the exception of the premium underAFP) leads to a conservative assumption of thevalue for money under AFP. It is conservativebecause it assumes that the private-sec tor partydoes not introduce any value-added innovations toreduce the ASB. Such innovations can oc cur whenthe responsibilities for design, construction,financing and maintenance are assumed by a

single party - who will then op timize the trade-offsthat are availab le between these different projec tcost elements. This is po ssible in the case of AFPdelivered projec ts where the responsibilities areassumed by a single party, but not in the case oftraditionally delivered projec ts. Until IO hascompiled sufficient empirical data regarding theprobability and magnitude of such innovations it willassume that the base costs under AFP and ASB arethe same (with the exception of the premium). Thisassumption will be revisited if, as projec ts arecompleted, experience indicates that the base

costs under AFP are reliably less than undertraditional delivery as a result of innovation.

B. FINANCING COSTS

Figure #3

=$15.0=$17.0

$0

$20

$40

$60

$80

$100





D: Ancillary Costs

D: Ancillary Costs

One of the c ommon elements of a ll the AFP modelsused by IO is private finance for some period of theproject:

Figure 3 illustrates how AFP financing costs aretypically greater than Traditional financing costs.

Trad itional Model Financing:When projects are built using a Traditionalprocurement method, the public sec tor makesprogress pa yments throughout the constructionperiod, and thereafter pays annually for facilitymaintenance . Depending on which public sec torentity procures a projec t, construction funds are

Traditional Model AFP Model

The public sector makesprogress paymentsthroughout construction.

The public sector incurs anopportunity cost for havingpa id ea rlier (paymentsthrough the c onstructionperiod) than under AFPdelivery.

Either the governmentmakes one lump sumpayment after constructionor makes a series of regularunitary pa yments to pay fo rthe projec t, starting a t

substantial completion andstretching over the post-construction term of theagreement. In either casethe private party borrows atprivate financ ing rates topay for the project costsuntil repaid by the publicsector.


9/23


either wholly or in majority provided by theProvince. While the Province may not borrowmoney direc tly from the market on a project-by-project 2 ba sis to make these payments, it incurs an

opportunity cost of having to pay earlier than itwould under AFP (under AFP, payment forconstruction is delayed until substantial completionor later). The government could have used thefunds used to ma ke these progress payments forother public purposes. A key alternative use for thefunds, one that can be used to measure thisopportunity cost, is to pay down existing publicdebt (thus avoiding interest payments on the paid-down debt) or alternatively, to avoid incurringad ditional borrowing c osts to finance governmentexpenses. It is important to note that since this

financing cost is not directly linked to project-specific borrowing, this financing cost is anallocated or notional cost. This notional publicfinancing cost is calculated at the current Provincialcost of borrowing (the notional public sec torfinancing rate). The Provinces cost of borrowingcan be estimated through readily available data.IO uses the simple average of yields on provincialbonds with a term of one year or longer as theestimated current borrowing cost (or weightedaverage cost of capital).

Though the VFM analysis methodology is co nsistentacross the AFP delivery models described earlier, akey difference is the c hoice of the point in time(referred to as the base date) at which the PSC andASB costs are compared. This choice has animportant effec t on how the public sec tor financ ingcosts are presented in the VFM a nalysis, though itdoes not affect the outcome of the VFM analysis.

Since, in the BF or DBF model, the public sec tormakes payment at projec t completion (a future

2 Since the current po rtfolio of A FP projects assigned to IOrepresent a very small po rtion of Provincial indebtedness,and since the c urrent AFP projects are themselvesindividually relatively small in magnitude, it is reasonableto a ssume that irrespec tive of the delivery model,

Traditional or AFP, no incremental public sec tor borrowingwould oc cur solely on ac co unt of such projec ts.

date); this is the da te that becomes the ba se datefor comp arison of PSC and ASB costs. Thus all BF orDBF PSC costs, such as the multiple c onstructionpayments made o ver the c onstruction term, have

to be future-valued at the public sec tor borrowingrate to the base date. The difference between thefuture value of each co nstruction payment and theconstruction payment itself represents the notionalcost of financing that the public sec tor incurs as aresult of having made the construction payment.For example, assume that the public sector makesa c onstruction payment of $20 million one yea r intoa three-year construction term. Assume further thatthe public sec tor borrowing rate is 5% a year. Bymaking the $20 million c onstruction pa yment, thepublic sector does not pay down public debt of $20

million. By construc tion end, this $20 million deb twould have grown to $22.05 million (i.e. the futurevalue at 5% compounded annually for two years).

Thus the difference of $2.05 million represents thenotional cost of financing a ssoc iated with theconstruction pa yment that the public sec tor made.

This calculation is done for each constructionpayment made by the public sec tor to arrive a t thetotal notional public sector financing cost that isadded to the PSC in a BF or DBF model. The timingof the construction cash flows is estimated and

provided by the external cost consultant.

In a BFM or DBFM model, the public sector makes aseries of unitary payments to the private sec tor,starting from construction completion andstretching over the post-co nstruction period (i.e.typically a maintenance term of 30 years). Sincethere will be not one but many future-da tedpayments in the BFM or DBFM model, the date onwhich the RFP closes 3 and all the private-party bidsare rece ived is used as the base da te forcomparison of the costs in the PSC and ASB models.

Thus all PSC c osts (and ASB costs) have to be

3 At the VFM publication stage (stage #3), the base dateis the date on which financial close of the project isac hieved. C osts are contrac tually locked d own a tfinanc ial close, making it a good point in time for thecomparison.


10/23

ASSESSING VALUE FOR MONEY: A GUIDE TO INFRASTRUCTURE ONTARIOS METHODO LOG Y- PAG E 10 -

present-valued ba ck to the base da te using thetechnique of discounting and using public sectorborrowing rate as the appropriate discount rate 4.Discounting the payments made by the public

sec tor in the PSC model explicitly accounts for theimplied public sector financing cost. To understandwhy this is so, c onsider the previous example wherea $20 million payment is to be made a year into thefuture. To finance a n expenditure o f $20 million in ayears time, the public sector has two equivalentchoices. It can either (A) borrow $20 million in ayears time to finance the expenditure oc curringthen or (B) it can borrow $19.05 million today, investthe borrowed money in an account bea ring 5%interest (e.g. buying its own debt that pa ys 5%interest), earning $0.95 million in interest (= $19.05 x

5%) over the next year so as to have $20 millionavailable just in time to finance the expenditure 5.

Since under choice B, the $19.05 million borrowedtoday would itself ac crue interest of $0.95 million (=$19.05 x 5%, recall that we assume that public debtpays interest of 5%) the opportunity co st or publicsec tor financing cost of $0.95 million is reflec ted inthe discount rate used in the discountingtec hnique. By borrowing an equivalent smalleramount (i.e. discounted) earlier ($19.05 million

today vs. $20 million in a year), the public sec torincurs a financing cost reflected in the discount rate(equal to the public sector financing rate). Thus noseparate public sector financing c ost line itemappears in the discounted PSC model for a BFM orDBFM VFM analysis (i.e. there would be no financingbox on the PSC side in the sample VFM figure, whendrawn for a BFM or DBFM project). It should benoted that if it were assumed that projec t-specific

4 The technique of disco unting and why the public sec tor

borrowing rate is the a ppropriate discount rate a re furtherelabo rated in a later sec tion. 5 Thus, toda ys $19.05 million is the present value of the $20million a year from now. Stated differently, the b orrowing(expenditure) of $20 million in a years time is equivalent toa borrowing (expenditure) of $19.05 million today (in theregime of a 5% interest rate and 5% discounting rate).

debt were to be raised by the public sec tor tofinance a traditionally-delivered project, then thefinanc ing c osts assoc iated with that specific debtwould be calculated and would appear as a

separate line item in the PSC model. However, thenet present value of tota l projec t costs would beidentical unless the project specific debt was issuedat a rate different from the public sector financ ingrate.

AFP Financing:Under the AFP model, the private sector is not paidby the public sector until the project is completeand thus in the interim, the private sec tor has toraise financing in the private markets to meetproject expenditures. At the initial VFM stage, IO

often engages independent, external financ ialadvisors who provide assumptions on the financingcosts and fees that a private party is likely to becharged (and will pass through to the public sec toras a cost) by the private markets for undertakingthe AFP projec t being analyzed. In addition IOstudies the financing costs and fees observed in thebids rec eived on ea rlier projec ts to develop privatefinancing assumptions. Financing costs aremodeled through the development of detailedfinanc ial models based on the c onstruction (and

lifecycle and operating c ash flow schedules in aBFM or DBFM) as developed by the relevantexternal co st consultant. When VFM is rea ssessed(stage #2), the actual private financing cost in thepreferred bid is used to replac e the estimatedprivate financing costs in the AFP model.

Total financing costs under AFP are typically higherthan public sector financ ing c osts because theprivate sector borrows at a higher rate than theProvince. This is a common criticism of the AFPmodel, but it is important to consider the overall

VFM analysis when evaluating which is theappropriate procurement model of choice. Higherfinanc ing costs are offset by the risk transfer to theprivate sector and mitigation of public sector risksunder AFP.


11/23


12/23


The development of a risk matrix co nsists of thefollowing steps:STEP #1: Identify the project risks

STEP #2: Allocate the risksSTEP #3: Estimate prob ability of risk oc currenc e

and resulting cost impact rangesSTEP #4: Run statistical analysis to qua ntify tota l

risks retained by the public sec tor

STEP #1: IDENTIFYING THE PROJ ECT RISKS The first thing that risk workshop participants do isidentify the individual risks that are inherent in theproject and group them by ca tegory. Generally,these are:

Planning/ strategic; Financial/accountability; Design and construction; Maintenance; and Life cycle.

STEP #2: ALLOCATING THE RISKSOnce the major risks have been identified, theworkshop participants allocate each of the riskseither to the public sector, or to the private sectoror as a risk shared by both public sector and privatesec tor, depending on the nature of the specific risk

in question, as well as the delivery approac h andrelated project agreement terms.

The following table shows how a risk is allocated tothe appropriate stakeholder depending on theprocurement model. The risk Design Coordinationand Completeness is one of several key risks thatare transferred to the private sector under AFP (asindicated by the Xs).

AllocationTrad itiona l AFP

RISK Pub. Pvt. Shrd. Pub. Pvt. Shrd.DesignCoordinationCompleteness

X X

STEP#3: ESTIMATING PROBABILITY OF RISKOCCURRENCE AND RESULTING COST IMPACTRANGES

The next step will determine the proba bility under

each de livery model that a risk will oc cur causingadditional costs (over and above base costs) to beincurred. For example, if one out of every twoprojec ts incurs costs due to a pa rticular risk, theprobability of oc currence would be 50%.

Onc e the proba bility of eac h risk occurring hasbeen determined, a range of potential co sts is thenestimated for each risk that is retained 6 by thepublic sec tor under either delivery method . Therange is expressed as a percentage of ba se c ostswith a range from (i) unlikely, but low additional

cost (10 th percentile); through (ii) most likelyadditional cost; to (iii) unlikely, but high additionalcost (90 th percentile).

For example, participants would be asked toestimate wha t the risks are associated with the riskDesign Coordination and C ompleteness. Sincethis is a risk that is reta ined by the public sec tor onlyunder the traditional delivery model, the range ofpotential c ost impacts to the public sec tor isestimated only under the Traditional delivery model.

The resulting impac t estimates may be as follows:

Traditional Impact Range

Risk

Probabilityof Risk

Occurring 10 th

MostLikely

90 th

DesignCoordinationCompleteness

90% 1.0% 3.0% 8.0%

The table illustrates that if the public sec tor sponsorswere to proc eed with the projec t using the

Trad itional delivery method, there is a 90% chancethat costs for the public sec tor would exceed thebase project costs on account of designcoordination and c ompleteness issues, and that the

6 The public sector may retain risks in full or in part (sharedwith the private pa rty)


13/23


range of cost impact is from 1.0% of base costs atthe low end to 8.0% at the high end. Under AFP, thisrisk is not retained by the public sec tor and so itsmonetary impact to the public sec tor under AFP is

zero. If this risk was reta ined by the public sec torunder AFP, estimates of the impact of this risk to thepublic sector would be developed as well.

STEP #4: QUANTIFYING TOTAL RISKSOnce the probability and impact ranges havebeen established for eac h risk, the cost of eachpotential impact can be calculated. The formulasfor calculating the cost of a particular risk are:

Cost of Risk PSC =Base Costs x Probability of Occurrence of the Risk under

Traditional delivery x Impact of the Risk under Traditionaldelivery

Cost of Risk AFP =Base Costs x Probability of Occurrence of the Risk underAFP x Impact of the Risk under AFP

Since, on any project, the actual impact of anyindividual risk may fall somewhere along acontinuum of impacts that includes the low, mostlikely and high ranges, and since this will notnec essarily be the same for each risk, statistical

analysis 7 is required to calculate the average totalcost impact of risks retained by the public sectorunder either delivery model.

Standa rdization and benchmarksIt is important to note that, while p rojec t specific riskestimations are developed and provided byindustry experts who have significant relevantexperience and knowledge , IO continues to workto minimize subjec tivity to the grea test extentpossible. One of the ways to reduc e this is by usingbenchmark probabilities and impact ranges thatare grounded in historical data and developed byindustry experts. AFP projec t agreements are highlystandardized, a nd as such, the variation in riskranges should be minimal ac ross projec ts.

7 Please refer to a later section for details on this statisticalanalysis

However, every project has its own uniquecharacteristics, and therefore every risk workshopwill yield slightly different results. Benchmarks aredeveloped by experts in the relevant field of cost

estimation: construction, building maintenance andbuilding lifecycle, and are used as a starting pointfor risk workshops. Risk ranges are then modified toreflec t projec t-specific risks.

Unquantifiable risksWhile considerable time and effort are allocated toquantifying risks in order to more a ccuratelycompa re the two delivery methods, the VFManalysis should also rec ognize those risks that aremore difficult to quantify. For example, the benefitsof having a project delivered on time cannot

always be accurately quantified. It would bedifficult to put a dollar value on ensuring that thepeople of Ontario ge t access to reduced waittimes in a new hea lth fac ility. These are importantqualitative benefits that, while not quantified in theVFM analysis, are important to considernonetheless.

D. ANCILLARY COSTS

Figure #5

=$14.0

=$12.0

$0

$20

$40

$60

$80

$100





D: Ancillary Costs

D: Ancillary Costs

There are significant costs assoc iated with theplanning and delivery of a large complex projec tthat could vary depending on the project deliverymethod . For example, there are costs related toeach of the following:


14/23


Projec t management: internal and/ or externalproject management fees, whether direc t orindirect, including the incremental costs of

Infrastruc ture O ntario providing its servicesunder alternative financing a nd proc urement.

Transac tion costs: additional transac tion costsare typically incurred under alternativeprocurement and financing, including lega l,capital ma rkets, fairness, transaction,architectural and engineering advisory fees.

Ancillary costs are quantified and added to eachmodel as applicable. Project management andtransaction costs are likely to be higher under AFP

given the grea ter degree of up-front due d iligence.As illustrated in figure #5, anc illary costs are higher,at $14 million, under ASB while they are $12 million inthe PSC.

Bringing it all Together toCalculate Value for Money

Onc e a ll the cost compo nents and ad justments aredetermined, the total costs assoc iated with eachdelivery model can be calculated, and expressedat the same point in time, as the ASB and the PSC.Separate c ash flow models are prepa red for theASB and the PSC, reflec ting the different costco mponents allocated to each model and whenthey will be incurred .

The PSC model would include the base costs,notional public financing c osts (in a BF), risksretained under traditional delivery, competitiveneutrality ad justment (where applicable), andancillary costs. The ASB model would typicallyinclude base costs (along with the private sec torpremium), financing costs, risks retained by publicsec tor under alternative financing andprocurement, and ancillary costs.

Once the adjusted shadow bid a nd the publicsector comparator are calculated, the positivedifference between the PSC and ASB representsthe estimated value for money proposition of using

AFP. Since the risk components in the models areexpressed as a statistical mean, the VFMproposition can also be expressed as a statisticalmean of a range of savings.


15/23


More on Notional Public SectorFinancing Rate and Discounting

The cash flow streams differ between the PSC (e.g.progress payments through construction) and theASB (e.g. lump sum pa yment at substantialcompletion or through post-construction paymentsduring the maintenance p eriod). In order tonumerically compare the cash flow streams, therespec tive cash flows must be expressed in dollarsas at a single date in time, known as the ba se date,by the technique of d iscounting c ash flows.

Bringing c ash flows forward in time (future valuing)or back in time (present valuing) is known asdiscounting and follows the concept of time valueof money the premise that a dollar today is worthmore than a dollar in the future. This reflec ts theopportunity cost of cap ital: funds available earliercan earn a return, or be used for other capitalexpenditures and therefore reduce the assoc iatedcost of borrowing.

Discounting hinges on the rate used to estimate thevalue of a future dollar in todays terms. Since theproject costs are in future dollars, and are

estimated costs that may turn out to be different(e.g. higher) than projec ted, the d iscount ratechosen should match the uncertainty inherent inthese c ash flows. Since higher risks require higherreturns, one could argue for a higher discount rate(i.e. risk-free rate plus risk premium) to capture theuncertainty in the project costs. However, this leadsto the counterintuitive result of future uncertaincosts being heavily discounted leading to a projectappearing less costly in present-day dollars as aresult of this increased risk. An appropriate methodto avoid this result is to quantify the embeddeduncertainty in costs through a comprehensive riskassessment. The quantified risks (i.e. cost of risk) canbe added to the estimated project c osts resulting invirtually risk-free costs. This risk-free cash flowstream can then be discounted back andexpressed in dollars as at bid submission date a t arisk-free rate. As the public sector financing rate

reflec ts the virtually unlimited taxing po wer of thecrown to repay its debts, crown borrowings areviewed as being risk-free.

Since crown borrowings are viewed as risk-free, theappropriate rate to use for discounting project costsis the public sector financing rate.

Infrastructure Ontario has chosen to beconservat ive and t ransparent by accounting forrisks exclusively through risk qua ntifica tionworkshops, rather than adding a risk premium to thediscount rate.

The public sec tor financing rate simply reflec ts theProvinces most current weighted average c ost ofcapital (WAC C). In consultations with the OntarioFinancing Authority (OFA), IO has determined thatthe best proxy for the Provinces most currentWAC C is the simple average of the long-termProvincial debt (bonds with terms of one to 30years). To neutralize the effects of daily fluctuationson the discount rate, a ten-day rolling average ofthis simple bond yield average is used as thestandard discount rate.

The advantages of computing the disco unt rate this

way c an be summarized by the following: Readily availab le, as market rates are public

and easily ac cessible when required; Reflects the market cost of funds as opposed to

static historical costs; and Rec ognizes the OFAs general borrowing

prac tices rather than being purely based onevery projects duration.

The choice of the disco unt rate has a significantimpact on results. Generally speaking, the higherthe discount rate, the higher the c alculated VFM.IOs choice of the lowest reasonable risk-freediscount rate ensures a conservative estimate ofVFM.


16/23


More on Risk Quantification andStatistical Simulation

Once the probability and impact ranges havebeen estab lished for each risk, the cost of riskretained by the public sec tor under Trad itional andAFP delivery are calculated using the followingformulas:

Cost of Risk PSC =(Base Costs x Probability of Occurrence of Risk #1 underTraditional x Impac t of Risk #1 under Traditional) + (BaseCosts x Probability of Occurrence of Risk #2 underTraditional x Impac t of Risk #2 under Traditional) + +(Base Costs x Probability of Occurrence of Risk #N under

Traditional x Impact of Risk #N under Traditional); wherethe risk matrix has N defined risks under Traditionaldelivery

Cost of Risk AFP =(Base Costs x Probability of Occurrence of Risk #1 underAFP x Impact of Risk #1 under AFP) + (Base Costs xProbability of Occurrence of Risk #2 under AFP x Impact ofRisk #2 under AFP) + + (Base Costs x Probability ofOc currence of Risk #N under AFP x Impact of Risk #Nunder AFP); where the risk matrix has N defined risks underAFP delivery

On any project, the actual impact of any individualrisk may fall somewhere along a continuum ofimpacts that includes the low, most likely and highranges (the 10 th , Most Likely and 90 th percentileimpacts quantified in the risk workshop). Since theimpact will not necessarily be the same for eac hrisk, without knowing in advance the exactcombination of risks that might occur in the projectbeing analysed, there are an infinite number ofsolutions to the above equations depending on thecombination of impacts that are plugged into theequations. A well-established mathematicaltec hnique for dealing with such problems is themetho d of statistical simulation. Statistical simulationfollows the following steps:

Step 1: Create a pa rametric model, y = f(x1,x2, ..., xN). In our problem, y is the c ost of risk

and the xes are the risk impacts for each ofthe N risks.Step 2: Generate a set of random inputs, xi1,xi2, ..., xiN. This is done by randomly picking a

risk impa ct number for each of the N risks,from within the defined range for that risk 8.Step 3: Evaluate the model and store theresults as yi. In other words, plug therandomly chosen set of impacts for each riskinto the two equations above and recordthe resulting c ost of risk number for

Trad itional and AFP delivery.Step 4: Repea t steps 2 and 3 for i = 1 to aminimum of 10,000 times.Step 5: Analyze the results using summarystatistics, confidenc e intervals, etc . Thestatistical simulation exercise generates a fulldistribution of c ost of risks under Traditionaldelivery and under AFP, as we now have10,000 different po ssible costs of risks ea chunder Traditional and AFP delivery. Thisdistribution can be statistically ana lyzed forthe mean (i.e. average or 50 th percentile)cost of risk retained by the public sectorunder Traditional delivery and under AFPdelivery. This mean cost of risk is used in theVFM analysis.

Most risk impact ranges, such as the DesignCoordination and Completeness risk discussed inthe sec tion on reta ined risks, are positively (orrightward) skewed 9 so the mode (Most Likelyoutcome) of the distribution is less than the mean(average or 50 th percentile) of the distribution. Usingthe Most Likely impact to calculate the cost ofthe risk would thus understate the true cost of therisk on average.

Optimism Bias:

8 Risks are assumed to be completely unco rrelated a ndimpact ranges are assumed to follow a triangulardistribution. 9 In probability theory and statistics, skewness is a measureof the asymmetry of the proba bility distribution o f a real-valued random va riable.


17/23


Underestimating the costs associated with projectrisks is common and is referred to as optimismbias. In statistica l terms, projec t planners tend tovalue risks according to their Most Likely

probability of oc currence, ignoring extreme caseswhere the incidence of a seemingly remote risk hashad a significant impact on a projec t. Forexample, if project budgets expanded by exactly1% over the life of the project eight times out of ten,it would not be unusual to learn that those planninga similar project would budget for only thatadditional 1%. If the remaining two in every tenprojects experienced budget overruns of anextreme 10%, however, the planning andbudgeting for a project should take this intoconsideration (i.e., should recognize that a 1%

contingency for budget overruns will not always besufficient). If the projec t managers are op timisticand expand the budget by only 1%, two out ofevery ten projects will experience overruns onaverage. One should, in fact, expec t an averagecost expansion of 2.8% (the average of eightprojec ts at 1% and two projec ts at 10%). IO avoidsoptimism bias by presenting all risks at their truestatistical mean, which takes into account not justthe M ost Likely risk outcome, but also all possiblerisk outcomes.

The conc ept of optimism bias can be furtherillustrated with our example of Design Coordinationand C ompleteness risk. Assume that this was theonly risk that appeared in the risk matrix. While wedefined the Most Likely impact under the

Traditional model at 3.0%, the average or mean forthis risk is actually 4.3%, owing to the skewed natureof the distribution 10. This is shown in the chart below.

10 Commercial Statistical programs are readily ava ilab lethat allow a user to selec t an appropriate impac t rangedistribution (for example a triangular distribution as shownin the example) to c alibrate to the 10th, Typica l and 90thpercentile impacts as established in the risk workshop s inorder to generate the Mean of the defined distribution.

Optimism Biasof 1.3%

If one were to budget only 3.0% for this risk, onewould underestimate the mean (or average) riskimpact by 1.3% (= 4.3% - 3.0%).

Continuing with our earlier example of Design

Coordination and Completion risk, for a project withestimated base costs of $60.0 million, we cancalculate the cost estimates for this risk at the meanof the risk impact range.

Calc ulating the Averag e (or Mea n) Cost of a Risk

Design Coordination and C ompletion Risk:

Mean Impact (Traditiona l): 4.3%

Cost for Risk =Base Costs x Probability of Occurrence x Mean Impact

Estimated Mean (Traditional):= $60.00 x 90% x 4.3% = $2.32 million

Under AFP, this risk is not retained by the public sector sothe impact to the public sector is $0 million. This exampleillustrates a cost savings under AFP for this risk of $2.32million.

The above example works only bec ause of theassumption tha t a single risk existed on the projec t.It was used only to illustrate optimism bias. Since aplethora of risks occur in concert on projects, it is

nec essary to use a statistica l simulation method toestab lish the true mean impact of the combinedrisks.


18/23


Final Comments

The methodology as set out in this doc umentapplies to building projects assigned to IO by theProvince. The VFM analysis is prepa red at multiplestages of proc urement by an external advisory firm.

The VFM analysis serves as a decision tool for the IOBoard of Directors at multiple stages ofprocurement ensuring that the choice ofproceeding via AFP is the best value proposition forthe public sector at a g iven po int in time based onbest information available.

IOs methodology and approach to VFM may besubject to change as new and be tter information

becomes available, but the ultimate methodologyand all underlying assumptions will be based onwhat is the most conservative, accurate andtranspa rent approach to estimating VFM. Thisensures that public interest rema ins pa ramount.

For additional information and greaterunderstanding of IO practices, this documentshould also be read in conjunction with any otherIO documents on VFM, such as VFM reports onindividual projects.


19/23


Glossary

Adjusted shadow bid : The shadow b id of aparticular project adjusted for risks retained by thepublic sector under AFP and for ancillary costs.

Alternative Financing and Procurement (AFP) : Arange of infrastructure project delivery methodswhich use private expertise and financing tostrategica lly rebuild vital infrastructure, on time andon budget, while ensuring a ppropriate publiccontrol and ownership.

Ancillary costs : The soft costs of delivering a projec t. These costs normally include: project management,

legal services, architectural and engineering,advisory and other professional fees, transaction,capital markets and fairness advisors.

Build Finance (BF): Typically considered for smallerprojec ts that involve renova tions or significa ntinterconnections to existing infrastructure (e.g.,shared HVAC , build-out of existing floors). Theprivate sec tor is generally responsible forconstruction and financing during the c onstructionperiod and the project is paid for by the publicsec tor at the completion of construction.

Build Finance Maintain (BFM): An AFP model inwhich the private sec tor is generally responsible forconstruction, maintenance a nd long-termfinancing. The projec t is pa id for in instalments overa fixed period, usually 25 to 30 years. The publicsec tor sponsor is responsible for developing thedetailed design of the fac ility.

Competitive neutrality : An ad justment mad e toremove certain perce ived ad ditional costs of

delivering a project using AFP. In certain instances,the base costs under AFP delivery will include aprovision for certain taxes and insurance premiums.

The equivalent co sts will not appea r under the PSCas the public sec tor may be exempt from payingcertain taxes and may self insure. The

adjustment consists of adding such costs to thePSC.

Construction costs: Costs incurred in completing the

construction of a project, including labour,materials, construction equipment, site preparation,construction management, typical contingenc ies,etc.

Design Build Financ e (DBF): A delivery model inwhich the private sec tor is generally responsible forthe design, construction and financ ing during theconstruction period. The projec t is pa id for by thepublic sector at the completion of construction.

Design Build Financ e Maintain (DBFM): Typically

considered for large projects involving newconstruction on a vac ant site (greenfield orbrownfield). The private sec tor is generallyresponsible for design, c onstruction, long-termfinanc ing and maintenance. The project is pa id forin instalments over a fixed period, usually 25 to 30years.

Discount rate : The interest rate a t which future c ashpayments are discounted to a base date todetermine their value at the base date.

Discounting is the process which allows costs to beassessed in current-value dollars.

Fac ility management : This typically includes theprovision of management, maintenance and repairservices related to the building and buildingcomponents to allow the facility to be used for itsintended purposes throughout the term of theProjec t Agreement, in addition to soft fac ilitiesmanagement such a s grounds maintenance ,pa rking, security, retail services like a food court orcafeteria, and dispatch services (e.g. one-call

help desk).

Lifecycle costs : Costs typica lly assoc iated withplanned or scheduled replacement, refreshmentand/ or refurbishment of building systems,equipment and fixtures that have reached the endof their useful service life during the projec t term.


20/23


Notional public sector financing cost : An estimateof the notional financ ing c osts that the publicsec tor would incur when a project is to be delivered

using a Traditional delivery method .

Optimism bias : A tendency of those planninginfrastructure projects to fail to take into accountthe full magnitude of risks retained by the projectsponsor.

Private sector financing costs : The financing costsincurred by bidders (and ultimately passed on tothe public sec tor) under a project deliveredthrough alternative financing and procurement.

Private sector risk premium : The premium (exclusiveof the private sector financing rate) charged bybidders to compensate for the risks transferred tothem under AFP in connection with the goods orservices being procured.

Project risks : Risks are events that can lead toserious cost increa ses, construction de lays, or bothshould they occur. Risks can be quantifiable (e.g.construction cost overruns) or qualitative (e.g.social, political or economic risks associated with

the delayed delivery of a project).

Public sector comparator (PSC) : Estimated totalcosts (including adjustments for risks retained a ndancillary costs) to the public sector of delivering aninfrastructure projec t using Traditional proc urementprocesses.

Risk matrix : A detailed table or chart that lists theconceivable quantifiab le risks for eac h project.

These risks range from cost overrun and design risksto p lanning and regulatory risks. Eac h risk is

described in detail along with the probab ility of therisk occurring and a range of probable costimpacts as a result of the risk oc curring.

Risks retained under traditional delivery : The projectrisks which are borne by the public sector when a

projec t is delivered using a Traditional deliverymethod.

Risks retained under alternative financing and

procurement delivery : Any projec t risk retained bythe public sector when a project is delivered usingalternative financ ing and procurement.

Shadow bid : An estimate of the expec ted private-party bid (including financing costs) for a particularAFP project.

Traditional delivery : Procurement of a project usinga Stipulated Sum Contract (usually the C CDC2 formof contract) for construction and, if applicable, aseries of short-term maintenance contracts post-

construction.

Value for money : The difference between thePublic Sector Comparator and the AdjustedShadow Bid is referred to as the Value for Money.

There is said to be positive Value for Money byprocuring a project using AFP when the AdjustedShad ow Bid is less than the Public Sec torComparator.


21/23


22/23


February 22, 2007

Infrastruc ture Ontario777 Bay Street, Suite 900

Toronto, Ontario M5G 2C8

Attention: Anurag GuptaSr. Ma nager, Projec t Finance

Dear Sir

ASSESSING VALUE FOR MONEY: A GUIDE TO INFRASTRUCTURE ONTARIOS METHODOLOGY

We have been pleased to be associated with the development of your Value for Money methodology. We havereviewed " Assessing Value for Money: A Guide to Infrastructure Ontario's Methodology " (the "Guide").

A value for money assessment consists of a comparison of the estimated total costs of delivering a publicinfrastructure project using alternative finance and procurement as opposed to the traditional public sector projectdelivery method.

We confirm that the value for money methodology described in the Guide will, if properly applied using validassumptions, yield fair and accurate results.

We also confirm that the methodology is consistent with best practices observed in other Canadian and international jurisdictions.

Yours very truly,

Deloitte & Touche, Canada LLP

Deloitte & Touche LLP79 Wellington Street WestSuite 1900, PO Box 29 TD CentreToronto, ON M5K 1B9

Tel: (416) 601-6150Fax: (416) 601-6690


23/23

Intentionally Left Blank

VFM GUIDE WEB.pdf

Documents

Transcript of VFM GUIDE WEB.pdf