128

Special Report

Post-projectappraisals pay

Frank R. Gulliver

li your company is like most,you spend thousands of hours planningan investment, millions of dollars im-plementing it-and nothing evaluatingand learning from it. As a result, youmay not have answers for the most ba-sic questions: Was the investment suc-cessful? What made it go according toplan? Did it go according to plan at all?As easy as these questions seem, theanswers aren't always ohvious.

British Petroleum (BP) builta plant in Australia to convert gas intoa component of high-octane gasoline. Itcame in under hudget and ahead ofschedule. A similar facility in Rotter-dam went over budget and v/as a yearlate. BP's managers first drew the obvi-ous conclusion: the Australian plantwas a success and the Dutch one a fail-ure. But a second look challenged thatfirst impression.

At the time the Australianproject was proposed, that country wassuffering from a balance of paymentsdeficit, and the product was expectedto help the country reduce its gasolineimports. The plant was completedearlier than expected. But by that time,Austraha's economic situation hadchanged, and gasohne demand turnedout to he lower than predicted.

Mr. Gulliver is group inter-nal auditor with the British PetroleumCompany He was a founding memberof its post-project appraisal unit, andit remains part of his responsibilitytoday.

Although the Rotterdam proj-ect had ohvious prohlems, the marketfor the product remained strong in Eu-rope. Thus that project's return on in-vestment was in line with predictions,while that of its Australian counterpartwas much lower. The Rotterdam proj-ect's success taught top managers at BPa valuable lesson: the planners neededto improve their market forecastingtechniques.

At this moment,managers in every

g company aremaking mistakes thatno one thinks

could be made.

There is an independent unitat British Petroleum's London head-quarters responsible for identifyingthese kinds of issues-the post-projectappraisal unit (PPA). It examines thethinking behind selected investmentsas well as their management and theirresults. PPA's sole mission is to helpBritish Petroleum worldwide learn fromits mistakes and repeat its successes.

Since its inception at the endof 1977, PPA has appraised more than

80 of BP's worldwide investments, in-cluding onshore and offshore construc-tion projects, acquisitions, divestments,project cancellations, research proj-ects, diversification plans, and shippingactivities. The appraisals are not aca-demic exercises; the unit seeks to im-prove company performance.

Through PPA, BP managershave learned how to formulate invest-ment proposals more accurately, ap-prove them more objectively, and exe-cute them more efficiently than everbefore. As a result, most projects nowgenerate returns on investment at leastas high as those forecast. These im-provements have naturally boosted thecompany's overall financial perfor-mance: in 1985, BP's profits reached anall-time high of £1,598 million aftertaxes. While PPA isn't tbe only reasonfor tbis performance, managers at BPhelieve the appraisal unit has yieldeddramatic results.

Wide-angle inquiry

In talking with husiness-people from large British and multi-national corporations, I have found thatfew companies examine tbeir com-pleted projects in any depth. Most au-dits are narrowly focused attempts tocheck tbat proper controls are in placewbile a project is in operation. Whenour managers audit an oil refinery, forexample, tbey gatber detailed informa-tion about how the oil and gas is col-lected, measured, shipped, and account-ed for.

A post-project appraisal,bowever, takes a much larger view. Itfirst looks at the hig questions: Whywas tbe project started in tbe firstplace? Is it producing as mucb oil asthe proposal predicted? Is tbe demandfor oil at tbe forecasted level? Did thecontractors deliver wbat they prom-ised? Does the project fit well into BP'soverall corporate strategy?

In "post-completion re-views," some U.S. corporations attempta similar sort of wide-angle evaluationof past projects. But these differ fromBP's post-project appraisals in twoways: objectivity and applicability. Be-cause project members usually con-duct post-completion reviews, they aremore likely to have preconceived ideas

Harvard Business Review MarLh-Aprill987 129

or even a vested interest in the reviews'outcomes. The members of BP's PPAunit have no affiliation with the proj-ects they appraise and so can evaluateinvestments more objectively.

Moreover, post-completionreviews usually don't guarantee thatthe lessons will reach the people whoneed them most, because the informa-tion spreads by word of moutb. PPA, incontrast, is a centralized departmenttbat can inspect any type of mvest-ment in any part of the far-flung BPgroup and transmit information fromone site to anotber. It can learn lessonsfrom an oil refinery project in Franeeand teaeh them to planners working ona similar plant in Australia.

PPA is also part of BP's in-vestment proposal procedure. Tbe unitreviews all new investment proposalsto make sure tbat no one repeats mis-takes. Wben tbey have time, unitmemhers will even work with projectplanners to formulate proposals.

Appraisal operations

Tbe PPA unit consists of amanager and four assistants, reportingdirectly to BP's board of directors. Inthe unit's nine-year history, the compo-sition of tbe staff has, of course, changeda few times. PPA managers, however,have to meet tbe same criteria: theymust be acceptable to the most seniorechelons of management and musthave at least 15 years of broad-basedexperience at BP Tbe company choosestbe other staff members for their spe-eific expertise. Tbey might be engineers,chemists, pbysicists, economists, or ac-countants. A team of two or three unitmembers investigates eacb project.

An appraisal of a large invest-ment generally takes ahout six montbsto complete. Because the company canabsorb only so mucb information at atime, the unit limits its major apprais-als to six per year. Tbe most valuablelessons come from tbe largest projects,where BP stands to lose or gain tbemost money. PPA selects its projectscarefully, looking for those that willyield the most valuable results.

The unit does not thereforeinvestigate a project if its lessons willduplicate those drawn from a previousappraisal. Nor does it evaluate a project

that BP is unlikely to do again. Tbeunit once considered appraising a largecrude-oil sale contract that BP badmade with anotber hig oil company. Be-cause tbe Middle Eastern nations hadnationalized their oil fields, however,BP no longer made such sales. Tbe unitconsequently decided not to study tbeproject.

Getting started. BP is dividedinto U businesses, each with its ownboard of directors and chief executive.These businesses report to BP's centralmanagement, wbich is headed by tbemain board of directors. A corporatereview committee of BP's main boardmust approve each PPA appraisal. Thiscommittee both oversees tbe unit's ac-tivities and examines all proposed capi-tal investments for compatibility withBP's corporate strategy. PPA suhmitsproposals for projeets it could appraisein 18 months to two years, and thecommittee generally accepts them,though it occasionally adds or deletesone or two.

The unit staff then deter-mines the order in which to carry outthe appraisals and, witb the chief exec-utive of the project's business, sets abroad timetable for each investigation.

A PPA team examines aproject from its conception-before theproposal is even written - usually untiltwo years after it bas become opera-tional. The team tries to determine sys-tematically how a project was bandied:at the proposal stage; during the proj-ect's construction (or, in the case of anacquisition, during tbe target compa-ny's purchase); during tbe project'soperation (or the acquired company'sintegration into BP); and during tbepost-operation (or post-integration)stage. PPA always tries to determinethe important factors that contributeto a project's problems or success.

Altbougb it usually learnsmore by seeing bow problems devel-oped, tbe unit also finds it useful topinpoint the causes of success. The pur-chase of a Dutch nutrition companycalled Hendrix, top management agreed,was one of the smoothest acquisitionsever. PPA ascrihed its success largelyto the precision with which the plan-ners had determined tbe extent of Hen-drix's integration into BP

Files and interviews. At anappraisal's outset, the team relies on

tbe files to become familiar with theproject. This avoids wasting people'stime. The team learns ahout the eco-nomic climate at tbe time, tbe identityof tbe contractors, or the chemieal pro-cess used. Team members might spendtbe first two months of a six-month in-vestigation just looking at files-bothat project files and at material in relat-ed corporate files, in sueh departmentsas accounting, legal, or planning.

While the PPA manager willprobably already know the senior man-agers who sbould be interviewed, thefiles provide a complete list. Tbe teamgenerally tries to interview everyoneinvolved in the project. Since most proj-ects have been completed for at leasttwo years before tbe unit begins itswork, bowever, tbe proieet membersare working all over tbe world on otherthings. In one investigation, the PPAteam talked to 80 individuals; the aver-age is usually around 40.

In their interviews, the PPAteam members make an effort to un-derstand tbe psycbology of the projectmemhers and managers, Tbey inter-view in pairs so that one team membercan ask questions while the otberwatches tbe interviewee. A furtive lookoften tells as much as a direct answer.

After tbe interview, the twoteam memhers compare notes and rec-oncile differences in their perceptions.Tbe full story usually emerges in sepa-rate pieces: senior managers in Londonwill give up one piece of information;engineers on an oil rig in the middle oftbe North Sea will give up another. Bymelding project members' differentperspectives, tbe PPA people can comeup with the whole picture.

PPA team members realizetbat project employees sbed light on is-

130 Harvard Business Review March-Apnl 1987

Four lessonsOver the past ten years, PPAhas taught BP management fourmain lessons. These are:

Determine costs accuratelyBefore PPA existed, BP's man-agement approved unreatistical-)y low budgets because plan-ners inaccurateiy predicted thescope of the project when theysubmitted the budget. Now BPapproves budgets in phases, andeach phase becomes more accu-rate as planners work out theproject's details.

In the first phase, the engineersoffer an approximate figure forthe project's budget that couldbe off by as much as 50%. Theboard then approves about 1 %of this sum to pay outside engi-neers and consultants to developthe case more fully. The engi-neers then submit a more accu-rate budget. Eventually, at thetime the board approves the en-tire project, it adopts a final bud-get, which should be off by nomore than 10%.

BP now pays more attention tothe technical requirements oflocal health, safety, and environ-mental legislation. Companymanagers look beyond simplywhat the legislation requires; toestimate costs accurately, BPplanners solve any design prob-lems created by such regula-tions in the proposal stage.

Managers now are careful not torush a project's approval so itcan qualify for a governmentgrant or other bonus. A rushedproject is often inadequately de-fined and therefore out of controlfrom the start, runs very late, andcomes in over budget-so muchover budget that costs substan-tially exceed the incentive.

The corporation no longer auto-matically awards a contract tothe lowest bidder. Many low bidscome in because contractorsdon't fully understand what BPneeds. PPA has found a correla-tion between low bids and poorcontractor performance.

Anticipate and minimize riskFearing that a competitor wouldsnatch the opportunity, BPbusinesses wishing to acquireanother company would oftentry to speed up the examinationand the decision-making pro-cess. According to the PPA unit,such self-imposed deadlines areusually illusory. Moreover, theunit has found that if the com-pany is not satisfied about thesoundness of an acquisition pro-posal, BP will probably notregret the missed opportunity.

BP used to expand plant capac-ity without knowing whether itcould sell a!! of the product thenew plant could then produce.Now before adding capacity orintroducing a product, the com-pany requires planners to sub-mit a full market survey to verifythat a market will exist and beprofitable.

Evaluate contractorsBP now has a contractor evalua-tion unit that monitors potentialcontractors' performance. Whenit solicits bids, the corporationalready knows which contractorwould be most likely to performto its satisfaction. Formerly, BPused an unsophisticated methodto select contractors. It wasignorant of contractors' deficien-cies and performance tor othercompanies in different parts ofthe world.

To make certain that a contrac-tor has expertise in a project'sprocess technology, BP nowpays careful attention to the cali-ber of the contractor's key staffmembers and insists that they re-main with the project to the end.

improve project managementEngineers do not automaticallymake good managers. The com-pany frequently used to send anengineer from a project in onepart of the world to one halfwayaround the world. No one askedwhether the engineer was famil-iar with the project, the country,or even the main contractor. Atthe recommendation of PPA, BPset up a projects departmentthat helps engineers developappropriate control techniquesand procedures and ensuresthat the right person managesthe right project.

To make project progress reportsmore constructive, the projectsdepartment has set up a proj-ects control division. This divisionuses software programs, linkedto each project, to help the proj-ect manager issue reports thatidentify likely problems and givereasons for missed milestones.These reports can be fedthrough the project control divi-sion's computer center for eval-uation on a day-by-day, or evena minute-by-minute, basis.

The projects department ensuresthat project managers are ap-pointed early enough to involvethem with the design considera-tions, project strategies, andcontrol mechanisms. With theprojects department's guidance,project managers can makemore independent decisions,

Capital investment analystshave usually swamped manag-ers with advice based on well-meaning academic research,but it has been limited to ques-tions about acquisition. Now,through post-project appraisal,managers can get sound adviceon questions about many kindsof projects from the experienceof their own companies.

sues that may seem unrelated to tbeirareas of expertise. Those working outin tbe field often live together, eat to-gether, and go out drinking togetber.Not surprisingly, an accountant mayoffer a cogent insight about the headengineer, even though they did not ac-tually work closely witb eacb otber.

Sending PPA teams into thefield to conduct investigations is farmore expensive tban sending out ques-tionnaires - and far more effective. Be-cause a questionnaire is a set collectionof questions, it can elicit only a limitedview of tbe project. In an interview,people offer unexpected information;

also, tbe PPA team can lead an inter-viewee away from digressions.

Conclusions and reactions.Tbe post-project appraisal unit bas badvery little trouble getting cooperationfrom BP's staff. In tbe unit's nine-yearlife, tbe PPA teams have found tbat peo-ple genuinely want to help the companygrow more profitable by joining in anexamination of performance. Even indi-viduals wbo bave been singled out forblame continue to see the unit's value.In one case, an appraisal concluded thata senior manager had not done his johwell. The corporate review committee

called bim in and raked him over thecoals. For some time, relations betweentbe manager and tbe PPA unit werecool. But a few months later, he tele*phoned tbe PPA manager to ask if theunit had appraised any projects simi-lar to one he was beginning. He wantedadvice.

The staff cooperates with theunit partly because it gives them achance to take issue with conclusionsbefore tbey appear in PPA's report. It isa testament to tbe fairness and accura-cy of the unit's work tbat no one has ev-er taken advantage of tbis opportunity.[Continued on page 132]

132 Harvard Business Review March-April 1987

After the team has exhaust-ed the files, interviewed everyone in-volved, and digested and assemhled tbeinformation in a preliminary draft tocirculate to key managers, it submits afinal draft to the business board andthen to the corporate review committee.

The most valuablelessons come from the

biggest projects,where the most

money is at stake. ^

The committee carefully considersPPA's work and almost always supportsthe conclusions: it has received manyhundreds of recommendations and hasrejected only one. Tbis suggestion-tbat BP maintain a staff of experts indifferent metallurgical technologies tosupplement contractors-was simplytoo expensive.

BP does not circulatethroughout the corporation the fullreports on each appraisal, althoughthese do go to relevant managers, butcollates tbeni into three booklets-oneon acquisitions, anotber on joint ven-tures, and tbe last on project develop-ment and control. PPA regularly updatesthese booklets—adding lessons leamedfrom later appraisals and occasionallydeleting a lesson tbat no longer applies.One was a recommendation to buildrefinery plants on the Continent ratherthan in Britain because of poor labor re-lations in the United Kingdom. But la-bor relations have improved greatlysince then.

BP's upper management ex-pects project planners to use tbe infor-mation in the hooklets as guidelineswben writing proposals. A proposaltbat does not meet all tbe guidelinesshould not necessarily be abandoned.But if planners cannot comply with theguidelines, the corporate review com-mittee will want to sec that the pro-posal accounts for tbe possible risks.

PPA sends the hooklets totbe London beadquarters of eacb ofBP's 11 businesses and to eacb of theapproximately 30 major BP associatecompanies worldwide. If any section of

the corporation needs more copies, tbeunit willingly sends them along. ThePPA philosophy is that the company'sinvestment performance will only im;prove as more BP people learn whatwent wrong and what went right in tbepast.

Frt)m its experimental andtentative beginnings a decade ago, PPAhas grown into an integral part of BP'splanning and control process. It suc-ceeds because of its consistent reputa-tion for digging out the truth. Tbe unitenjoys tbe full confidence of BP's seniormanagers and directors because tbeybelieve tbat both the facts and the con-clusions in the reports are accurate.Tbis accuracy is based on the investi-gating team's thoroughness, its under-standing of tbe technical issues, its fair-ness in evaluating tbe evidence, and itssensitivity to the psycbological forcesmotivating the staff. In that accuracylies the usefulness of the lessons to thecorporation and tbe success of the post-project appraisal unit.

Appraisal lessons

There is a big difference he-tween classroom lessons about busi-ness and lessons drawn from experi-ence. Wbat might seem self-evident orunlikely in theory may he the most im-portant factor in an actual event. To il-lustrate, let me describe a project froma time before BP implemented many ofthe procedures PPA recommended.

In 1967, a director at BP re-sponsihle for engineering and refiningwanted to explore a technology tbatExxon and others were using but thatwas new to BP Tbe man was well re-spected witbin tbe company and had agreat deal of influence. By tbe force ofhis personality, he pushed through aproposal for tbe construction of thebiggest plant of its kind tbat BP badever built. Exxon had a plant thatturned out 30,000 barrels of oil per dayon three produetion lines; tbe BP in-stallation would produce tbat volumeon a single line. This line required thelargest compressors and pumps that BPbad ever used and completely newtechnology in the reactor vessels.

During construction andtesting, tbe company had difficultywitb all tbree. It had particular trouble

witb tbe reactor vessels, wbicb, be-cause of tbeir size, had to be thinnertban conventional vessels and thusneeded lining with stainless steel. De-spite assurances to management thatthe joh was easy, BP's contractor raninto one problem after another. Finally,BP's own engineers solved the prob-lems at great expense.

BP learned mucb from PPA'sinvestigation of this experience. Itlearned that it must assess proposalsmore carefully. It leamed to assess anew technology's risks more tborough-ly and more objectively. It learned thatit had to improve its method of select-ing contractors. Perhaps the companysbould have learned these lessons al-ready. But obviously it had not-and thepost-project appraisal process broughtthem to light, formalized them, andcollected them in one place.

Managers in every companyare making mistakes no one thinkscould be made. Time after time, thepost-project appraisal unit has uncov-ered these kinds of mistakes andhelped British Petroleum avoid repeat-ing tbem. ^