Problem formulation for multi-criteria decision analysis: report of a workshop

JOURNAL OF MULTI-CRITERIA DECISION ANALYSIS

J. Multi-Crit. Decis. Anal. 7: 242–262 (1998)

Problem Formulation for Multi-Criteria Decision Analysis: Report ofa Workshop

SIMON FRENCHa,*, LISA SIMPSONb, ELIZABETH ATHERTONa, VALERIE BELTONc, ROBYN DAWESd,WARD EDWARDSe, RAIMO P. HA8 MA8 LA8 INENf, OLEG LARICHEVg, FREERK LOOTSMAh,ALAN PEARMANi and CHARLES VLEKj

a School of Informatics, Coupland 1, University of Manchester, Oxford Road, Manchester, M13 9PL, UKb School of Computer Studies, University of Leeds, Leeds, UKc Department of Management Science, University of Strathclyde, Strathclyde, UKd Department of Social and Decision Sciences, Carnegie-Mellon University, Pittsburgh, PA, USAe Departments of Psychology and of Industrial and Systems Engineering, University of Southern California,California, USAf Department of Systems Engineering, Helsinki University of Technology, Helsinki, Finlandg Institute for Systems Analysis, Russian Academy of Sciences, Moscow, Russian Federationh Department of Mathematics, Technical University of Delft, Delft, The Netherlandsi Leeds University Business School, Leeds University, Leeds LS2 9JT, UKj Department of Psychology, University of Groningen, Groningen, The Netherlands

ABSTRACT

This paper reports on a workshop on Problem Formulation in Multi-Criteria Decision Analysis held at SPUDM97.The focus of the workshop was the problem formulation phase which occurs between the analyst meeting a personwith a ‘mess’ and the time he or she begins to analyse a structured problem with several alternatives scored againstseveral attributes or criteria. The objectives were: to share experience on procedures which might be transferablebetween the methodologies; to demonstrate different skills used by the analyst in structuring decision problems;and to catalyse a discussion on the problem formulation phase of an analysis.

Three analysts, who generally approach problems using multiattribute methods, addressed the same problem.The problem used was constructed to be realistic to three decision makers, who had been trained in the issues ofconcern. There were two sessions. Each analyst was assigned a decision maker and formulated the problemindependently in the first session, held in parallel. They were each observed by two observers and many of theaudience at the workshop. The three formulations were presented along with the comments of the observers anddiscussed at a second plenary session.

This paper reports the three formulations and observations, remarking on the ‘tricks of the trade’ employed bythe analysts in formulating the problem. The analysts also describe their thinking and their aims in adopting theirapproach and style of interaction. More general remarks on the process of decision analysis are also offered.© 1998 John Wiley & Sons, Ltd.

KEY WORDS: multi-criteria decision analysis (MCDA); problem formulation; process of decision analysis

1. INTRODUCTION AND FORMAT OFTHE WORKSHOP

A workshop on Problem Formulation in Multi-Criteria Decision Analysis was organized bySimon French and Lisa Simpson at the Subjecti6eProbability, Utility and Decision Making(SPUDM97) conference held in Leeds in August

1997. The idea behind the workshop was to lookat the problem formulation phase which takesplace when an analyst and client explore a ‘mess’of concerns and issues and structure these into adecision problem with several alternatives scoredagainst several attributes/criteria. In the earlyplanning of the workshop we had hoped to drawtogether analysts from very distinct schools ofdecision analysis. We hoped to see if their differ-ing methodologies led to distinct approaches toproblem formulation. However, the diaries ofsome of those invited precluded their attendingSPUDM97. The three analysts who took part

* Correspondence to: School of Informatics, Coupland1, University of Manchester, Oxford Road,Manchester, M13 9PL, UK. E-mail: [email protected]

CCC 1057–9214/98/050242-21$17.50© 1998 John Wiley & Sons, Ltd.

Recei6ed 9 June 1998Accepted 10 July 1998

PROBLEM FORMULATION FOR MCDA 243

approach problems in roughly the same mannerusing value trees (attribute hierarchies). Accord-ingly, the purpose of the workshop became toshare experience on problem elicitation and for-mulation. In the event the observation and discus-sion of these sessions brought out many pointsranging from ‘tricks of the trade’ through obser-vations on the management of the social interac-tion to the philosophy and ethics of decisionanalysis.

The decision analysts who kindly agreed to takepart and formulate a problem in front of anaudience were Valerie Belton, Freerk Lootsmaand Raimo P. Hamalainen, who was assisted byMari Poyhonen. Each was introduced to a deci-sion maker (DM)1. The DMs were three lecturersfrom the School of Computer Studies at the Uni-versity of Leeds. They had no prior training orexperience in decision analysis. The DMs hadbeen schooled in a problem ‘mess’, related to areal issue within the School of Computer Studiesbut with additional hypothetical events to increaseurgency and raise the complexity. Simon Frenchand Lisa Simpson briefed the subjects in this‘mess’, but held back from suggesting alternativesor objectives or any other aspects that should beestablished during the formulation of the mess asa MCDA problem.

The workshop was run in two 2 h sessions onthe 19 and 21 August, respectively. The first ses-sion comprised three parallel events in which eachdecision analyst met one of the DMs and dis-cussed the issues in order to formulate a decisionmodel. The analysts had been given a very shortbriefing note (see Figure 1) a few days beforewhich outlined the ‘mess’, but other than thatthey arrived at the session completely unaware ofthe situation before them. In this way we tried toensure that the analysts were observed in the veryfirst stages of problem elicitation and formula-tion. It was, to draw upon a metaphor used byone of the analysts, an ‘unseen examination’.

The analysts were encouraged to push the anal-ysis as far as they could in the 2 h, actually a fewminutes less since there were a number of openingannouncements at the beginning of the session.Each session was observed both by delegates atSPUDM and by two designated observers. Allobservers were forbidden to interact at all duringthe session. They just watched.

The second session was plenary in which allanalysts, observers and audience met together.Each analyst presented his or her approach todecision analysis and ‘guiding principles’ in prob-lem formulation. They also presented their initialformulation of the issues in the scenario, indicatedhow the analysis would have progressed, whatjudgements would need to have been elicited,what further data would be needed, etc. Theobservers each made a short presentation on theirthoughts. The remainder of the session was anopen discussion.

In the next section an outline of the problem isgiven. Subsequent sections describe the formula-tions of the problem produced by the three ana-lysts and the comments made by the observers.The penultimate section records the general dis-cussion at the workshop. We close with sometentative conclusions.

2. THE PROBLEM ‘MESS’

The problem was set in the School of ComputerStudies at the University of Leeds, where the threeDMs actually lectured. In that way it was hopedthat they would share a common context whichwould help in their role playing. Moreover, theproblem was related to a real problem. Aspects ofit had occurred a couple of years back. Neverthe-less, the scenario we set up involved many hypo-thetical aspects, particularly hypothetical actionsof certain others in the School. We would like toemphasize that the current system for providing

Figure 1. Outline of problem ‘mess’ sent to the DMs in their briefing note.

© 1998 John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 7: 242–262 (1998)

S. FRENCH ET AL.244

coffee is not in crisis and those organising it haveno reason to resign en bloc. The School is verygrateful to them. That being said, the remainingtext in this section is a verbatim copy of thesummary of the scenario given to and discussedwith the DMs before the first session.

The School of Computer Studies at the Univer-sity of Leeds is relatively large among UK univer-sities. There are 6 professors, 52 academic andresearch staff and over 60 postgraduate researchstudents. The undergraduate programmes have anenrolment of about 550 full time equivalent stu-dents: in number of bodies, this means about 1000students are taught in various lectures, laborato-ries and classes by the School. In addition, there isa growing Masters programme, currently withsome 40 students. The School is sited in roomsand laboratories on Levels 6, 7 and 9 in the E.C.Stoner building, with some staff and postgradu-ates accommodated in an Annex some 400 maway. There are plans to move the staff from theAnnex to further rooms on Level 6, but this moveis at least a year in the future. The main adminis-trative area of the School is on Level 9 and mostsenior staff are accommodated near there.

The common room is on Level 7. It is a rela-tively large room with several small tables, 40 orso comfortable chairs and a small kitchen areawith sink, refrigerator, dish washer, cupboardsand kettle. In addition, on Level 9 there is a smallkitchen with sink, refrigerator, microwave andkettle. On Level 10, in the nearby MathematicsDepartment, there is a coffee bar run by theUniversity Refectory, which provides a range ofsnacks and beverages between 10.00 h and16.00 h. There are a number of cold and hotdrinks machines on nearby staircases and corri-dors.

The School common room is open to staff andresearch postgraduates, but not taught postgradu-ates (i.e. Masters students). Taught postgraduatesmay use the Level 9 kitchen to make their drinks.There are no common room or drinks facilitiesfor undergraduates within the School. They mayuse the Mathematics coffee bar or the drinksmachines. A coffee and tea club is run by theSchool’s common room Committee. Staff pay asubscription to this and can make tea or coffee asthey wish. Supplies are put out each morning inthe common room. (Few supplies make itovernight, either because of consumption or ‘ac-quisition’ for use at someone’s home.) The School

pays the subscription of all research postgraduatesto encourage them to mix with staff other thantheir supervisors. The School investigated payingstaff subscriptions, but the Tax Office insistedthat this would need to be taxed as a benefit-in-kind and thus would require a very complexaccounting system. The School also pays for acleaner to straighten the common room eachmorning and wash up cups, etc. Many staff makedrinks with their own supplies, either in theSchool common room or Level 9 kitchen.

For some time now there have been a numberof issues rumbling in the background. Firstly, thelevel of social interaction in the School is low. Inparticular, only about 25% of staff use the com-mon room regularly and then seldom every day.Attendance is at its highest for the Friday Morn-ing Doughnut Club. On a weekly rota someonebuys 50 doughnuts, which are consumed on a firstcome first served basis. The Head of School andmany senior staff feel that, if there was moreinteraction, there could well be benefits in devel-oping new research. Informal discussion oftenleads to new insights. However, it should be notedthat the proportion of senior staff who go to thecommon room is well below 20% because, theyplead, of the continual need to attend administra-tive meetings. The clerical and administrative staffwould be very welcome in the common room too,but they claim that the time to get from Level 9 toLevel 7 (by a contorted set of staircases) is toogreat. Moreover, the route from Level 9 to Level7 is not easy for the disabled, of whom there area small number.

Taught postgraduates have indicated their wishto be invited to the common room to mix withstaff and research postgraduates. Undergraduatestudents have long campaigned for a ‘social cen-tre’ in the School where they can meet each other.Many Departments at Leeds have some areawhere students may congregate, chat and drinkcoffee.

There have been heated debates in the Schoolover the use of the microwave on Level 9, mainlyoccurring after someone has prepared an aromaticmeal. A dish of curried cabbage and the literalstink it caused are part of the School’s folklore.Only marginally less heated are the discussionsabout the source of the coffee used by the coffeeclub and by others using their own supplies.Firstly, some are concerned at the price paid tothe Third World growers by the food conglomer-ates. Secondly, some are concerned at other trade



in baby milk carried out by the largest instantcoffee manufacturer. On the other side of thecoin, are discussions of price and quality.

Last week two things occurred which have ledto the problem. Firstly, the kettle in the commonroom blew up. One member of staff complainedrather unpleasantly to a couple of members ofthe common room Committee. It was unfortu-nate that this came on top of a growing numberof demands upon them and also at a time whenperchance they were all frantically trying tocomplete a project report. Whatever the cause,the complaint blew up into a row and the Com-mittee resigned en bloc, claiming no one appreci-ated the amount of effort they put in. Thus, thereis no kettle in the common room, no replenish-ment of supplies and no one willing to take onthe task. Secondly, the University Refectory hasnotified the Head of School that the Mathematicscoffee bar is too small to be viable and theywill be closing it at the end of the calendaryear.

Three lecturers have been asked by the Head ofSchool to prepare a report on ways forward. Theyhave been told to think very widely and report in2–3 weeks.

3. THE FORMULATION CONSTRUCTEDBY VALERIE BELTON

3.1. Her formulationMy interpretation of the remit for the workshopwas to focus on the formulation, or problemstructuring, phase of the decision aiding process.At this stage there is no assumption about whattype of analysis, if any, would best help the DM,Matt Morley. The process is initially relativelyunstructured in order to encourage divergentthinking. An understanding of whether or notmulti-criteria analysis is an appropriate tool formore detailed evaluation should emerge from thisproblem structuring phase. If the answer is ‘yes’,and given the situation it seemed reasonable topresuppose that some aspect of this issue wouldbe amenable to MCDA, then the problem struc-turing should lead naturally into a more struc-tured phase of multi-criteria model building andanalysis. Indeed, much of the structure of themulti-criteria model should emerge from the moregeneral model of the problem structure (Belton etal., 1997).

Facilitation of the problem structuring discus-sion was guided by the following objectives:

� to establish the key issues: What were thecauses of the problem as outlined? Was theimmediate issue symptomatic of a longer termproblem?

� to identify stakeholders and the extent of theirpower and interest in the problem;

� to identify key uncertainties;� to explore possible ways forward;� to establish the values of important

stakeholders.

The process adopted was influenced signifi-cantly by the specific situation. The most impor-tant objective was to be able to capture thediscussion in a flexible way which would allow forinteractive structuring of ideas as they emerged. Asignificant constraint was the need for the processto be visible to the audience. Given these factors Idecided to make use of Post-Its to capture aspectsof the problem as they arose, organising these onthe whiteboard to reflect the areas of interestoutlined above. An alternative, more hi-tech ap-proach would have been to map the discussionlive using Decision Explorer (Banxia Software,1996). However, this would have focused the at-tention of both the facilitator and the client onthe computer screen, largely excluding the audi-ence (in the absence of projection facilities). Also,as the workshop was a one-off occurrence thebenefits of having the problem structure in elec-tronic form would not be realized in the longerterm.

The client was soon at ease with the use of thePost-Its and participated in recording ideas andbuilding the picture emerging on the whiteboard.

Figure 2 shows the problem representation as itstood at the close of the problem formulationsession. The first hour was spent primarily onbuilding a picture of the current facilities for theprovision of tea and coffee and the users of thosefacilities, together with issues surrounding theiruse including the recent events. From the discus-sion to this stage it emerged that in addition tothe immediate problem, which was to restore asatisfactory system for the operation of the coffeeroom facilities, there were also a number of longerterm issues which should be addressed. The longerterm issues included opening up the coffee roomto use by other stakeholder groups, but alsobroader issues relating to the geography of thedepartment and a wish to encourage greater inter-


S. FRENCH ET AL.246

Figure 2. Problem representation as captured with Post-Its (and subsequently drawn in Decision Explorer).

action amongst staff and students, a view heldstrongly by the Head of School. At this stage,the client was asked whether he would prefer tofocus on issues relating to use of the coffee roomor on the broader issues. He opted for the for-mer. (He was also offered the opportunity of abreak, but declined.) From that point the discus-sion centred on alternative ways of operating thecoffee room facilities and factors which shouldbe taken into account in deciding a way forward.The needs of the different stakeholder groupswere also considered.

The options which were identified are illus-trated in the table at the bottom of Figure 3.The main emphasis is on ways of restoring thecurrent system, perhaps at the same time encour-aging its wider use by staff and postgraduateresearch students. Options for provision for Mas-ters (taught postgraduate) students and under-graduate students are listed separately. For theMasters students one possibility is to invite themto make use of the coffee room, thus interactingwith decisions about the current system. It wasnot considered feasible to invite undergraduatestudents to join the coffee room scheme and thusprovision of facilities to meet their needs couldbe considered as an independent issue.

It is clear from the presentation of the optionsin Figure 3 that rather than being a choice be-tween a number of clearly defined discrete alter-natives, the decision is comprised of a numberof, possibly inter-related, sub-decisions. It is notclear that this decision would be best handled bythe classical approach of identifying all possiblefeasible ways forward and evaluating each inde-pendently according to specified criteria. Thiswould give rise to at least 70 possible combina-tions and to evaluate these individually wouldseem to be unnecessarily clumsy. An alternativeapproach may be to use an EQUITY type ofmodel (Hall, 1986), but this would ignore anypotential synergies or complications that mightarise from particular combinations of subdivi-sions.

A number of goals or criteria pertinent to theevaluation of different ways of organising thesystem emerged naturally during the discussion.The client was prompted to add to these byconsidering the relative strengths and weaknessesof different ways of working. The following keyfactors were identified:

� the extent to which the system encouragedparticipation;



Figure 3. Common room facilities—options available.

� the extent to which the system encouragedinteraction;

� the ‘quality’ of the service provided, including– choice offered– reliability of the system– degree of congestion/crowding;

� the possibility that service providers would feeltheir work was not valued;

� the cost of the system, to individuals and to theSchool.

To close the workshop it was suggested thatthere were two ways in which the work couldprogress. Firstly, a more detailed evaluation ofthe options outlined in Figure 3 against the crite-ria identified above would serve to highlight thestrengths and weaknesses of different systems withthe aim of identifying the best way forward forthe School. Some preparatory work to establishthe feasibility and costs of particular aspectswould be required. Secondly, in order to explorethe broader issues in more detail it would beworthwhile bringing together a cross-section ofinterested parties in a problem structuringworkshop.

3.2. Remarks by Elizabeth AthertonOne of the most notable aspects of the analysiswas the relaxed way in which it was conducted.Valerie guided the focus of the DM while allow-ing him to speak freely and explore aspects of theproblem. The DM was allowed to write when hefelt motivated to and participated in the processin a relaxed and open way. This approach enabledthe DM to consider the issues at his pace and inhis way rather than being ruled by the analyst.The process was controlled by the analyst, in thatthe DM’s attention was focused on different as-pects of the problem, but once focused the DMwas encouraged to explore the aspects in his ownway.

To start the whole process off Valerie asked theDM to describe the problem as he saw it. I believethis helped the DM to realize that Valerie wasinterested in his feelings and thoughts and gaveValerie a background to the problem. During theanalysis all the main points were placed on Post-Its which were put on a whiteboard and movedaround as the analysis progressed. The Post-Itswere grouped into stakeholders, alternatives and


S. FRENCH ET AL.248

attributes. This helped to structure the problemand identify how the different aspects related toone another.

During the discussion Valerie focused the DM’sattention on different aspects of the decisionproblem and their interaction. The analysis beganby focusing on the stakeholders, their role in thesystem and the resources available to them. Aimswere then established and the means to meet theaims were discussed. This helped to create newalternatives which were then related to the stake-holders. The DM was encouraged to think ofways of combining the alternatives and alteringavailable resources to improve them. This alsohelped to create new alternatives.

The analysis started off quite generally andfocused in on the important issues. As the analysisprogressed it was clear that there were twoimportant aspects to the problem: the immediateproblem of restoring the coffee club and thelong term problem of the coffee facilities availableand how they are used. As time was limitedValerie asked the DM which part of the problemhe wanted to focus on. He decided to focus onthe immediate problem in the system. In this waythe problem structuring focused on the issueswhich the DM thought were important, andgave him power in the decision process. Atseveral points in the analysis Valerie reviewedthe information uncovered. In this way shechecked her understanding of the DM’s com-ments and enabled him to recap on what hadbeen said. The process helped to clarify the issues.Valerie asked the DM to think about how otherstakeholders may feel about aspects of the deci-sion and the alternatives available. This encour-aged the DM to look at the problem fromdifferent perspectives and to focus on other peo-ple’s opinions. The DM was encouraged to thinkwidely and was asked about other coffee systemshe had experienced. This helped to create newalternatives.

The process ended with a summary of whathad been discussed and possible ways forward,which helped to clarify what had been achievedand what could be done in the future. Duringthe second session comments were made aboutthe problem structuring and the DM was quickto defend the analysis. This shows that he feltthat he played an active part in the analysisand so any criticism of it was a criticism ofhim.

3.3. Remarks by Robyn DawesValerie’s attempt to structure the problem byeliciting relevant considerations in a non-directivemanner illustrated the difficulty—if not the im-possibility—of using a totally non-directive ap-proach to structuring a decision problem in ahour and a half’s time. Consideration after con-sideration was mentioned by the person reportingthe problem, but Valerie did not ‘force’ the re-porter to make a judgement of which were moreimportant than which. Assigning weights is, ofcourse, the final—not an initial—step in con-structing (formally or informally) a decision tree,but in my view the approach taken led to adivergent analysis that never came together at theend. This divergence was reflected in 53 Post-Itsnotes involving possibly important considerationsstuck to the whiteboard, notes that were occasion-ally moved around and regrouped, but whichwere never discarded as specifying unimportantfactors. Thus, in the absence of a formal weight-ing system, the conclusion was—in my view—a‘weak’ one.

Specifically, it did not appear to address theproblems that led to the resignation of the com-mittee. Kettles fail (‘blow up’) all the time, butsuch failure does not usually lead to a social ‘blowup’ of the group sharing the kettle, especially notto the point of mass resignations. I was frustratedwatching the interaction Valerie structured, be-cause I wanted to shout at the reporter ‘for real,what the hell is going on?’ and then subsequentlyassess his response for indications of its potentialinaccuracy, or the role he himself might haveplayed in the social (as opposed to the kettle)blow up. (I had the distinct impression that hewas ‘too glib’ and so prone to bringing up newissues—rather than pursuing issues already men-tioned—that I was suspicious of his motives.) Butof course, I wouldn’t give into this impulse, eitheras a decision analyst or as a therapist. And Va-lerie certainly didn’t make any such confronta-tional inquiry, and shouldn’t have, whether shefelt an urge to or not. I believe, however, that inthe absence of such direction, I and others neverhad sufficient information to make a judgementabout ‘what the hell went on’. The reporter keptbringing up a new factor, and then another newfactor, and then yet another one that might berelevant to setting up a new system. If the analysis(or ‘therapy’) could have gone on for weeks, thenthe approach of allowing the reporter to reach theconclusions about the problem on his own with-



out confrontation would—many (Rogerians andothers) believe—have been superior to a confron-tational one. But we had an hour and a half.

The considerations brought up during this pe-riod were equally relevant whether an old systemhad existed and broken down or no system at allhad existed. Often, it is helpful to frame a prob-lem as if there were no history. ‘Just starting fromnow, as if nothing had happened before, what doyou think the relevant issues are and what mightyou do if indeed you had happened upon thisproblem—or your life—straight off a spaceshipfrom a distant planet?’ Such reframings can bevaluable (like the ‘why aren’t things worse?’ or‘given what you’ve told me, why don’t you com-mit suicide?’ reframings I just discovered I hadfilched from the late Victor Frankel). My concernis that relying exclusively on an historical frame,as Valerie did, when the interaction is limited toan hour and a half, leads to a lot of vague andsometimes platitudinous considerations. Concen-trating on this possible function of a coffee club,another possible function, a location here, a loca-tion there, and so on may have avoided thecritical factors that led to the problem in the firstplace.

A personal note. In the debriefing sessionSimon assured us that while aspects of the sce-nario were historically accurate, others were not.Moreover, there are no serious residual problemsin the department. So perhaps my concern that wenever got to some less than pleasant ‘guts of theissue’ is just a reflection of a type of paranoia thatis too often reinforced elsewhere.

4. THE FORMULATION CONSTRUCTEDBY FREERK LOOTSMA

4.1. His formulationWhen I prepared the session with the limitedamount of information which was given to us, Iplanned to proceed as I would have done in Delftwith a new project. I assumed that an appoint-ment had been made via the phone, for instance,and that the speaker had very briefly sketched thepurposes of the visit. The discussion should beinformal since a project had not yet been defined.

So, at the beginning I offered Tony Jenkinssomething to drink, and thereafter I asked him togive a more detailed sketch of the problem. Hetold me that there was a serious conflict about the

common room in the School of Computer Stud-ies. After some unpleasant incidents the volun-teers who organized the coffee and tea service andwho collected the financial contributions refusedto continue their activities.

The conflict might have deeper causes, how-ever. Otherwise it would be incomprehensible whythe volunteers so suddenly dropped what they hadbeen doing so far. I first tried to find out who theusers of the common room were. Postgraduatestudents frequently came to drink coffee, and theyconsumed the doughnuts which were usuallyavailable on Friday mornings. Staff members,however, came less frequently. Certainly thosewho had many commitments did not show up.They even resented the common room because ofthe predominant presence of the students. Thecollection of the contributions was irregular, andthere was a feeling among staff members that thestudents did not really pay their fair share. More-over, the location of the common room was farfrom ideal. The offices of the School were ondifferent floors, so that some staff members al-most never came to meet their colleagues aroundthe coffee kettle.

At this stage, I had to be careful. Otherwise Iwould be swept away by my prejudices. It seemsto me that volunteers should not run a commonroom. If a clear allocation of the responsibilitieshas not been made, if there are no staff membersor if there is no (elected or appointed) committeeexclusively responsible for the common room,difficulties will always arise. Secondly, althoughmany university departments assert that a com-mon room is essential for informal communica-tion, there are sometimes empire builders whokeep their subgroup isolated from the rest. Thereasons are not always clear, but it happens.Therefore I had to follow the track and to verifywhether my assumptions were correct.

First, I tried to get an idea of the atmosphere. Iasked Tony to tell me some anecdotes. He men-tioned again the explosion of the coffee kettle andthe smells in the building because someone usedthe common room for the preparation of a spicymeal, but he did not say that the conflict was dueto the actions of certain characters in the School.

Next, I tried to explore the problem by dis-cussing alternative options for solving the conflict.This step sometimes shows the boundary condi-tions in the client organization. I asked Tonywhether it would make sense to maintain thestatus quo. The Head of School could ask the


S. FRENCH ET AL.250

Figure 4. Freerk Lootsma’s formulation.

volunteers to take up their job again. He couldalso try to find new volunteers, but the systemwould not change. That seemed to be a feasibleoption. Thereafter, I suggested that the commonroom could be run by a committee elected orappointed by the School. That was also feasible.Furthermore, the common room could be relo-cated to a more central position, possibly anexpensive operation but, given the importance ofimproved contacts within the School, not totallyunacceptable. Again, Tony agreed that this wasan option to be considered. Then I introduced thezero option: don’t do anything. The commonroom could be closed, no coffee, no tea, theremight be enough coffee shops on campus, and thecoherence of the School was perhaps already toolow to maintain the coffee and tea service. So, wejointly tried to view the problem from differentperspectives, that is, we set up various optionsand we came up with several criteria to judgethem. At this stage, I started to use the white-board in order to set up a tableau which wouldshow the strengths and the weaknesses of thealternative options. Of course, this is precisely thestart of the screening phase in MCDA, where wedraft the performance tableau which shows theperformance of the alternatives under the respec-tive criteria. Eventually, we had the survey shownin Figure 4.

Some cells in the tableau contain tentative plusor minus signs in order to indicate the perfor-mance of the options under the criteria listed in

the first column. Of course, this would not be thestarting tableau for a complete analysis. The op-tions still had to be combined into possible strate-gies for the School, and the criteria still had to bedefined more carefully, but the drafting of thetableau gave me some idea of the room to play,and it hopefully gave Tony some idea of theapproach. I told him that the DMs in the Schoolcould all be invited to make up their mind. Wecould ask them to fill the cells of their ownindividual tableau and to weigh the criteria.Thereafter we could analyse the tableaux in orderto identify the predominant criteria as well as acompromise solution. We could even carry outthe analysis in a group decision room with ampleopportunities for electronic brainstorming andweighted voting.

Lastly, I arrived at the crucial questions whichI always have to ask at the beginning of a project.Who is (are) the DM(s), and what could be myrole? Thus, I asked Tony whether the Head ofSchool would take the final decision or whetherthe decision would be made somewhere else. Hethought that the Head of School would submit aproposal to the Staff Meeting. ‘Do you expect thedecision to be difficult’, I asked him. ‘Yes, I guessso’, he said, but he could not sketch me whichmembers would object to which possible solu-tions. Then I suggested to him that I could sup-port the decision process if the Head of Schoolwould define a project and if he would open abudget for it. I did not specify something like a



lump sum or a daily rate because that is mostlyunimportant. What matters is the commitment ofthe client organization. The decision to hire anoutside analyst is enough to give him/her aspecific role in the decision process, at least in myexperience.

In summary, although I might have an idea ofthe feasible alternatives, I did not have a clearpicture of the power game in the School. I did notknow whether the Head of School was cleverand/or powerful enough to impose his solution tothe problem, or whether there were other keyfigures in the School who had to be handled withcare. Perhaps I should have asked Tony to drawthe building on the whiteboard, to show the loca-tions of the subgroups, and to describe the leadingpersonalities. The first meeting with the represen-tative of the client organization is not decisive,however. Usually, I have to wait and see whetherthe client returns in order to set up a joint project.The second meeting is more important. It enablesme to work out a full-scale MCDA with variantsof SMART and the AHP (possibly with weightedDMs), in the particular role of an outside consul-tant hired by the DM(s) in the client organization.

Afterwards, when I saw the case description, Ifelt sorry for Tony Jenkins because I asked himseveral questions for which he was really notprepared.

4.2. Remarks by Oleg LarichevThe organization of the workshop around animaginary problem created some challenge to theanalysts. The analysts represent different schoolsin decision analysis and have different personalskills. It was assumed that they would try to usedifferent strategies at the stage of the problemanalysis. In order to examine this assumption weconsider how the analysts used the available infor-mation. They were all provided with thefollowing:

(a) a short description of the problem whichcould be called the ‘coffee room problem’(CRP);

(b) a face-to-face meeting with a substitute forthe DM.

My personal evaluations of how Freerk workedwith the data are as follows.

4.2.1. Positi6e features(a) Freerk clearly understood that the initial re-quirement was to structure the problem. He did

not try to use any particular techniques (e.g. theAHP method) at this stage of the problem’s solu-tion. He posed question after question trying tomake clear the relationship of the DM to theother participants in the CRP, to find alternatives,criteria, evaluations of alternatives and so on. Thediscussion looked very natural from my point ofview. Freerk used the language of the DM with-out giving him artificial tasks like ‘define theweights of criteria’ and so on.

(b) Freerk finally found the main alternatives ofthe problem’s solution. They could be representedas:

Status Quo (SQ): to find the way to support theexisting system of organization.Vending Machines (VM): to install vending ma-chines in the School.Small Coffee Rooms (SR): to organize manysmall coffee rooms in different places.

There were also some unrealistic alternatives suchas: to have no coffee room at all or to take aroom from the Mathematics Department. But myimpression is that they were introduced by Freerkto support the discussion.

(c) Freerk found the list of criteria importantfor the CRP:

� Relationships between the members of staff(R). There were professors who did not giveany value to the coffee room. They had theimpression that other people behaved in inap-propriate ways: preparing ‘an aromatic meal’,taking milk home and so on. They mistrustother people.

� Money (M). ‘Staff pay the subscription’ butsome professors are too busy with administra-tive meetings to visit the coffee room. They donot want to pay for others.

� Relation to personal habits (PH). Some profes-sors are not in the habit of using the coffeeroom to discuss research and teaching activi-ties. They prefer to have short talks with onlytheir students.

4.2.2. Negati6e features(a) Freerk asked again and again: ‘Please, give mea new alternative’. It is useless from my point ofview to ask the DM about new alternatives. He/she is usually restricted by a traditional vision ofthe problem. What is possible for an analystduring this stage of analysis? To define the re-quirements for new promising alternatives and


S. FRENCH ET AL.252

organize the search for ideas for their develop-ment (Larichev et al., 1994).

(b) Freerk did not make a clear representationof the three-dimensional box: alternatives–crite-ria–stakeholders. There are three stakeholdergroups, different in terms of their relation to theCRP: two kinds of staff (users and non-users ofthe coffee room) and postgraduate students. Itwas possible to describe their relation to SQ, VMand SR. It was also possible to find for eachgroup the evaluations of each alternative in theterms of R, M, PH. Having such a presentationwould make it easier to see the essence of theconflict in the CRP. It would be possible to findthe possible changes in the existing situation (SQ)desirable for all stakeholders.

Finally, I see the similarity between the taskgiven to the analysts in this workshop and one ofretrospective protocols used to study medical de-cision-making (Patel and Groen, 1991). This latterapproach gave interesting results in a differentfield. Maybe, it would be interesting to repeatsuch a task in the future again on a wider scale: togive to several analysts the description of a prob-lem (history of a ‘patient illness’) and ask them tostructure it using given information in the bestpossible way.

4.3. Remarks by Alan PearmanMy comments on Freerk’s formulation of thecoffee club problem address principally the mod-elling process, but also to some extent the result-ing model structure itself. In the problemformulation phase, there is a clear interdepen-dence between the two, in this application andmore generally.

4.3.1. Confidence, relaxation and a little bit ofirrele6anceFreerk’s time with the DM was split into threeapproximately equal phases. The first was anentirely oral exploration of the surface issue, lead-ing on to a gradual deepening and widening toprobe for more fundamental concerns. Critical tosuccess in this key early phase is to build theconfidence of the DM. This is in part a personalconfidence, the analyst as confidante, but also aprofessional confidence. The analyst was in thiscase (and needs to be seen to be) confident in thedirection in which she/he is taking the analysis,without appearing unduly prescriptive or hurried.

A few irrelevant, peripheral and particularlyopen-ended early questions proved a good invest-

ment of time in developing an effective and free-flowing conversation about the problem.Listening is every bit as important as asking. Astime passed and an initial formulation of theproblem began to develop, a useful technique wasto ask ‘Can you tell me any general anecdotesabout the School which in some way relate to thepoints we have touched on?’. A question like thislightens the tone of the conversation, breaks themould and potentially opens up new insights be-yond the surface appearance of the problem.‘Why is this your problem?’ was another usefulway to induce a change of perspective.

4.3.2. So what are we talking about?A point is reached where the momentum of theinitial conversation starts to flag. When it did so,Freerk moved into the second phase, a white-board-based approach towards a formal problemspecification. He used the change simultaneouslyto do three things, essentially by setting down anunderstanding of the problem in terms of thematrix-based alternatives and attributes formula-tion which would ultimately be used to guidechoice. Firstly, this process enabled him to checkthe accuracy of his own understanding of theproblem. Secondly, it served to introduce the DMto the formal analysis procedure. Thirdly, it beganto identify alternative solutions to the problem.Overall, this was a very effective and time-eco-nomical way of shifting from discursive mode tomore structured and formal analysis.

During this phase, Freerk was willing to suggestalternatives, in a general way. He specifically in-troduced the zero option, ‘Why bother to do any-thing?’. Although there may be an argument forthe analyst not suggesting alternatives, my feelingis that a total ban on such activity is excessive.Surely part of facilitating the DM in identifyinggood options is to suggest possibilities? As op-tions were introduced, so too were attributesagainst which the alternatives might be assessed,usually through exploring their ‘feasibility’.

Again, eventually the impetus from this line ofdevelopment started to flag. A final check withthe DM for further options was made and theanalysis moved into the final phase of the initialsession.

4.3.3. A stab at e6aluation‘Who is the final DM?’; ‘Can you guess whichoption would gain majority support?’ were usefulways of better understanding the decision in ques-



tion. Then pairwise comparison of alternativeswas used as a basis to score alternatives on a− −/−/0/+/+ + scale. Although in principlethis might have led to the identification of apreferred option, it did not and was not reallyexpected to. In this particular case, the analysiswas in fact halted prior to a full evaluation of allalternatives on each attribute.

4.3.4. How far is far enough?Freerk’s formulation was notable in using onlyabout 1 h of the total time allotted. This wasdeliberate, essentially a recognition that, after anintroduction, a period for quiet reflection can nowbe useful to both parties. The DM can reconsidermatters in the light of potential solutions whichhave been aired and on the basis of a moreformalized structuring of the problem. She/he caneven consider whether examining the problem inthis way deserves any further investment of timeand possibly money! The analyst can reflect onthe nature of the problem presented and on howanalysis might best be pursued in any subsequentsession.

Although circumstances will clearly differ, andthe pure practicalities of operating as a consultantwithin an organization may preclude it, there ismuch to commend this approach of a rather briefinitial meeting, followed by a period forreflection.

4.3.5. Discussion and suggestionsOverall, the analysis was professional and effec-tive. However, some possible avenues for im-provement are:

1. The analyst could perhaps have set down ini-tially a ‘route map’ of broadly how and wherehe saw the session going. This would haveclarified things for the DM and perhaps haveencouraged further confidence in the processas a whole.

2. Some time devoted to probing the DM moredeeply for potential options might have beenfruitful.

3. Inevitably, given the short initial session, someof the attributes derived were somewhat am-biguous and potentially overlapping. Clarifica-tion in subsequent sessions prior to any formalevaluation would be necessary.

4. Although to some extent an artefact of theworkshop situation, having a one-to-one ini-tial meeting with a single DM was arguably

less helpful in many respects than a one-to-many meeting between the analyst and a fullcross-section of stakeholders from within theSchool.

5. THE FORMULATION CONSTRUCTEDBY RAIMO P. HA8 MA8 LA8 INEN

5.1. His formulation

5.1.1. BackgroundIt is very important to note that the session wasstarted with almost a complete lack of advanceinformation about the case problem. This is asetting which has received very little attention inthe literature. Yet, it is quite realistic to seek andbenefit from decision support in such cases too.We have recently observed this in the parliamentof Finland. In our study (Hamalainen andLeikola, 1995, 1996) small groups of parliamentmembers were offered the opportunity to get sup-port by spontaneous decision conferencing abouttopical issues. They were assisted in problemstructuring and prioritization by using multiat-tribute value models. This approach was verypositively received.

In a standard facilitation setting I would dohomework beforehand to get an understanding ofthe problem area and potential interests of therelevant stakeholders. In the current case wecould not be prepared at all and thus the situationwas like a spontaneous decision conference.

5.1.2. Goal dri6en approachIn the following description I will summarize themain principles and steps of the approach which Itried to follow in this session. I assumed the roleof a decision analysis consultant who had beeninvited to help the customer in his problem solv-ing. Thus, there was no need to sell my ideas andservices to the customer, but to start the processdirectly. My facilitation process was very stronglydriven towards the goal of problem solving andproducing some kind of deliverable. This is anapproach which I use when the DM or customercan specify the task in which help is needed andwhen there are tight time constraints present. Ifelt that we had a situation like this, however,very often this is not the case. The facilitator canalso be invited at a phase in which the DMs,stakeholder groups and the problem scope are stillvery poorly understood. In that case, I would


S. FRENCH ET AL.254

start with a different process which would havelonger brainstorming and scenario generation ses-sions at the beginning.

Keith Hobley was my customer whom I refer toas the DM even if he would not be the final DMin this case. I had received a very short problemdescription where his task was defined to be thepreparation of a report to the Head of the Schoolwithin 2 weeks. I decided to try to use this as thegoal. The report would not need to include arecommendation. It could equally well turn out tobe a problem description as the DM saw it.

In the beginning I explained to Keith the work-ing goals I had set for myself and the session:

� He is my customer and the king of the sessionand I will try to advance and serve his interestsunless he wants to take another perspective.

� Introduce the value tree analysis approach.� Create personal credibility and a confidential

relationship with the DM. This would hope-fully be transferred into the methodology to beused, as well.

� Emphasize the DM’s problem solving, i.e. helphim produce the report to the Head of School.

� This first session should answer the followingquestions and result in the first definition ofthe problem framework.– what are the issues creating the need to do

something?– who is the true DM?– who else is involved, who are the

stakeholders?– what are the scales and ranges of impacts

and interests?� Keep the time constraint of 2 h in mind and

produce some useful deliverables. This wouldguarantee that the DM would be well moti-vated for the next session and could feel thatthe time was not wasted and that he hadachieved something.

5.1.3. Procedural methodThe facilitation technique used could be called aninteractive computer supported decision analysisinterview (Marttunen and Hamalainen, 1995) or aspontaneous decision conference (Hamalainenand Leikola, 1995, 1996). It was a face-to-facedialogue between the analyst and the DM. I amused to running such sessions together with atechnical assistant. Such help is essential in situa-tions when time is a limiting factor. It also helpsin general as the customer is not distracted from

the dialogue by technicalities. The assistant keepsa record of the issues and keywords identifiedduring the session and visualizes the value treesaccording to the structures arising in the facilita-tion. He/she also operates the computer. An assis-tant needs to have basic knowledge of the valuetree methodology. He/she should not actively par-ticipate in the process, although this happened alittle in our session. The assistant was my doctoralstudent Ms. Mari Poyhonen and I would like toacknowledge her efficient help.

The session followed the steps listed below:

� Problem/task statement.� Identification of the true stakeholders and their

general goals and values.� Option screening and generation by scope

expansion.� Formulation of an ideal solution and candidate

alternatives by the win-win philosophy.� Re-specification of the problem.� Searching for stakeholder criteria to be in-

cluded in a preliminary value tree.� Rough performance estimation to learn the

ranges of the attributes.� First attempt in numerical weighting of criteria

to find out the essential discriminating onesand to learn the scope of the problem.

� Produce printouts of the model as deliverables.� Describe future steps and what would be done

in the next session.

5.1.4. Task statement and issues defining theproblemThe most important step of the process was toidentify the problem task which would then deter-mine the angle of attack, i.e. the perspective of theanalysis. What really is the problem and who arethe true DMs?

Keith seemed to see his role as a representativeof the staff members who had come to be inter-viewed by me. I did not find him a strong interestholder himself. I suggested that a possible objec-tive statement might be to help him produce areport in such a way that the staff reach theirgoals maximally. Keith accepted this as the pre-liminary working principle. We would alwaysrefer to this task when discussing the decision-making problem. It seemed that originally Keithhad not thought that the preparation of the reportfor the Head of School was so important, al-though this was clearly stated in the problemdescription. To me it looked as if he had not



thought that he was responsible for the report andthat 2 weeks was very little time for it. Eventhough I emphasized this perspective, it did nothave a strong effect on the development of oursession.The primary issues which arose in thediscussion included:

� Location of the staff and the coffee room andthe way of operating it. The coffee room iscurrently at Level 7 and the rooms of staffmembers and postgraduate students are both atLevels 7 and 9, Level 6 is likely to be used inthe future.

� The users of the coffee room are staff membersand postgraduate students. The undergraduatesalso had an interest in a socialising space.

� Administrative questions: What is the role ofthe staff room committee? Who is paying whatfor the coffee?

� What are the possible location solutions: newbuilding, cafeteria, different levels? Could theSchool provide free coffee?

� The decision-making procedure: the Head ofthe School is making the final decision basedon the recommendation given by the SchoolManagement Board (SMB).

5.1.5. Identification of stakeholders and theirobjecti6esThe identified groups of stakeholders were: thestaff members located at the two levels; postgrad-uate students; undergraduate students; and theSMB. The objectives of each group were listed asthey appeared during the discussion:

� The actual DMs, Head and the SMB, seem tobe interested in the costs only. They are willingto accept any economically reasonable solutionproposed by the staff members. The SMB alsoincludes staff members.

� The main issue for the staff members is thelocation of the coffee room and kitchen whichare used frequently. The location of the staffroom is not as important. Yet, these two facili-ties are likely to be the same room. The loca-tion has an effect on the communication andsocialization among all staff members as wellas with student groups. The staff members alsorequire an efficient system to collect paymentsbecause they have to pay for the coffeethemselves.

� The postgraduate students get free coffee. Theywould like to improve the socialization with

staff members. They are also worried about thespace issues. They do not have much room atthis moment and new arrangements may takespace away from their other activities.

� The undergraduate students want to have cof-fee machines and they would probably not usea coffee room. The socialization with staffmembers, however, is considered to beimportant.

Overall the main objective turned out to be theimprovement of the communication and socializa-tion among staff members and between staffmembers and student groups.

5.1.6. Scope expansion with ideal and win-winsolutionsThe first thing discussed was the problem. I triedto change the perspective from the alternatives tothe goals with the question ‘Is this really a coffeeroom problem?’. To me the real ultimate goalseemed to be the improvement of the working andcommunication atmosphere in the department. Iasked Keith to think of the possibility of having acompletely new building or location for theSchool with a special design emphasis on thesocial areas. Could this be a realistic, even if, anextreme policy alternative? This idea seemed to betoo radical and it was left for future sessions. Thealternative of a coffee shop or cafeteria run by anoutsider was brought up. This alternative was alsoincluded to get a clearly different reference for theevaluation. I tried to emphasize that the specifica-tion of the alternatives is a crucial point because itdefines the problem scope.

Based on the stakeholder interests we started tothink what an ideal staff room would look like. Ifwe are able to present a set of alternatives for thereport which all have a built in win-win philoso-phy then Keith’s goals are guaranteed to be metto some degree at least. By win-win I mean solu-tions which would have some positive elementsfor each of the stakeholders. Based on the discus-sion it was clear that it would be possible todesign a staff room and coffee delivery systemthat would satisfy most of the people involved.The issue raising conflicts between the interestgroups seemed to be the location: whether thestaff room should be on Level 9 or on Level 7.The pros and cons of these two solutions werelisted and evaluated with respect to the objectivesidentified earlier.


S. FRENCH ET AL.256

Figure 5. Raimo P. Hamalainen’s value tree for the coffee room problem.

5.1.7. Value tree and deli6erablesWe tried to capture the stakeholder’s objectives ina simple value tree to be included in the report.Next we tried to come up with some clear mea-sures, like money for costs, and their ranges foreach objective. It was hoped that the value treewould clarify the problem and alternatives to thetrue DM. We had very little time for the priori-tization but I wanted to carry it out to get apreliminary feeling of the relative merits of thealternatives. If one of them would turn out clearlysuperior then this fact should direct our nextsession. We used the HIPRE 3+ software(Hamalainen and Lauri, 1996) running on aportable computer accompanied with a portableprinter. The main objectives in the value tree (seeFigure 5) are costs, socialization and the effective-ness of communication. The attributes are costs toSchool and costs to staff members, socializationamong staff members and between staff membersand undergraduate students. The three alterna-tives evaluated were a cafeteria, a coffee roomeither on Level 7 or on Level 9.

The performance of the alternatives with re-spect to each attribute were estimated subjectivelyand very quickly (see Figure 6). The first idea of

the relative importance of the objectives was alsoasked and thus we could produce weighted perfor-mance scores for the alternatives. Direct weight-ing was used. This initial analysis showed thatboth coffee room alternatives are realistic. Therecan be more interest in moving the staff room toLevel 9 because that would improve the commu-nication among the staff members. However, thedifferences are not very strong. The printoutsfrom the value tree model showed this and theywould remain as the direct deliverables from thissession.5.1.8. Future stepsIn the next session we would rethink the alterna-tives keeping the improvement in communicationin mind as one of the main goals. Yet, we shouldstill also look for measures that could improve thecoffee delivery system.

A more complete value tree may need to begenerated to give credibility to the report. Thiswould show that the objectives of all the stake-holders have been considered. Representatives ofstakeholder groups should perhaps be given anopportunity to do the weighting of objectives toimprove joint problem ownership and commit-ment to the report and its recommendations.



Figure 6. Raimo P. Hamalainen’s comparison of coffee room alternatives.

We should interpret the model results verballyand explain the benefits of each alternative.Keith should prepare a summary of these pointsas well as a final recommendation and explain themerits of this alternative by the win-win philoso-phy.

5.2. Remarks by Ward EdwardsAs I watched Raimo toiling away at a quickelicitation of an MAU, my overwhelming impres-sion was of the differences between what I wasseeing and what I have seen when doing MAUelicitations for real, with real clients. Some of thedifferences:

1. Raimo was working under intense time pres-sure, and showed it. He did a minimum ofstructure elicitation, and it was never clear tome that his elicitee understood what a valuetree is or why it is useful. He had no time todo cross-checks of his elicited numbers.

2. The ‘stakeholder’ didn’t have much real stake,and indeed it wasn’t even apparent that a realdecision needed to be made. It was quite clearthat there was no organizational commitment

to the elicitation I was watching. In most realcontexts an organizational commitment is im-plied by the elicitation because a commitmenthas been made to pay for the elicitation.

Given all that, I thought Raimo did a quitecreditable job. He got to number elicitation ratherquickly; I think that, even in this artificial situa-tion, I might have tried for more situational andstructural underpinnings. But if Raimo had donethat, he couldn’t have finished in time.

As happens every time I do or watch a weightelicitation, I am struck with the importance andslipperiness of the notion of ‘importance’. Raimoused that word a lot, and explained it very little.In similar situations, I do much the same. Noadvocate of swing weighting (like myself) has everfound really good language to replace that wordin explaining what weights mean to an elicitee orto the users of MAUs.

All in all, a very creditable and hard-workingeffort on Raimo’s part. It is simply amazing howpervasively relevant the techniques of additiveMAU elicitation have become in many, manypsychologically savvy decision analysts’ hands.


S. FRENCH ET AL.258

5.3. Remarks by Charles Vlek

5.3.1. Decision theory, support and psychologyThree points to begin with concern decision the-ory, decision support and decision psychology.Classical decision theory primarily is a theory oftask environments rather than of cognitive pro-cesses, and it revolves around individual ratherthan social representations, probabilities andutilities. Thus, both the individual and the socialpsychology of decision-making are not ac-counted for in the classical theory which is nor-mative only in so far as the underlying axiomsare accepted.

Decision support may be practically dividedinto information provision, problem structuringand option evaluation, where any former stageconditions any latter (Vlek et al., 1993). Thus,problem structuring underlies option evaluation,while information provision underlies problemstructuring. Without an overall model of gooddecision-making it is difficult to tell the differ-ence between relevant and irrelevant informa-tion, as well as the difference between usefuland useless elements of the problem structure.Obviously, a specific decision rule is needed toevaluate ‘good’ against ‘bad’ decision alterna-tives.

Decision psychology rests upon descriptivemodels of human decision-making, for differenttypes of problems, DMs and social contexts. In-dividual decision psychology deals with limitedinformation-processing capacities, the guidingrole of goals and values, cognitive and be-havioural effects of decision stress, and the fun-damental distinction between cognitivelycontrolled (reasoned) versus automatic (habitual)choice behaviour. Social decision psychologydeals with inter-individual differences in problemdefinition, values and expectations, with powerrelations among concerned individuals, and withconflict management and consensus formationamong differing stakeholders. These and otherindividual-cognitive and social-deliberative vari-ables and mechanisms play a role in any multi-party decision situation.

5.3.2. Problem formulation on the common-coffeeissueFirst of all, Raimo’s attempts to ‘formulate’ thecommon-coffee decision problem at the Univer-sity of Leeds, were remarkable for the great

amount of time spent on gathering informationfrom the DM. This was aimed at determiningthe nature and the background of the currentproblem situation, the various ideas about itwhich seemed to exist among different involvedgroups (staff, graduate and undergraduate stu-dents, and the Head of School), and the valueobjectives which one seemed to be cherishing.During this information acquisition, it becameclear that valid empirical knowledge about themany stakeholders’ own problem diagnoses,goal-directions, and feasible solution-scenarioswas missing, whilst the DM could only providehis own perceptions and impressions of what theothers found problematic and/or desirable.

A second outstanding phenomenon duringRaimo’s interrogation was his apparent back-ground model of ‘a well-defined decision prob-lem’ as optimally structured in terms of anoptions by attributes matrix. Here, the presump-tion seemed to be that a multiattribute evalua-tion model would be most effective, and that,for example, no major uncertainties (requiringsome kind of expected-utility analysis) were in-volved. In fact, major uncertainties seemed to bethere all right, in association with future hous-ing rearrangements and the degree to which anystakeholder group (staff, graduate and under-graduate students) would actually make use ofany new arrangement for the provision of coffeeand tea. These and other uncertainties cannot beexplicitly accommodated in an options by at-tributes structure.

Thirdly, in his attempts to identify feasibleoptions Hamalainen reminded me of an essayby the Polish philosopher Szaniawski (1980)who emphasized the goal-driven character of de-cision-making in general and of option genera-tion in particular. Szaniawski specified twoconditions for any alternative to be included inthe choice set: (1) it must be acceptably feasible,i.e., the DM must be reasonably able to imple-ment it, and (2) it must be goal-relevant, i.e.,there must be a causal connection between thealternative and the goal to be achieved. Thesetwo conditions imply that the DM validlyknows his or her own capabilities for carryingout what he/she has decided to do, and thathe/she also has valid expectations about thelikely future situation emerging from any choicealternative. Particularly the latter should havebeen, but could not sufficiently be explored dur-ing the workshop session.



5.3.3. Scope of problem and goal-setRelated to the previous point is the scope of boththe problem diagnosis and the desired goal situa-tion. To the extent that these are both limited tothe operational level of ‘regularly having coffeetogether’, a wider problem context and a broaderset of goals may remain obscure. During theanalysis it appeared, for example, that socializa-tion and communication were important putativeeffects of ‘regularly having coffee together’. How-ever, no diagnosis was performed of the degree ofinformal work-related socialization and communi-cation that actually took or would take placeunder different past and possible future coffee-having arrangements. Also, no alternative social-ization and communication strategy wasconsidered in which the words ‘coffee’ and ‘tea’did not figure prominently. Perhaps the deepergoals and values of the departmental staff andstudents would have been served at least as well(for the time being) by leaving ‘coffee’ and ‘tea’out of the debate.

Finally, the value attributes considered for theoptions identified were also set on the initial goalsof the problem-formulating operation. Again, so-cialization and communication seemed important,next to practical and financial feasibility. Butmissing were individual-actor attributes such aswork efficiency and small-group embeddednesswithin the larger department. As already noted,perhaps the scope of the problem and of the goalstate was too restricted for more strategic valueattributes to emerge. For one thing, an attributelike ‘future controllability of common-coffeestrategy’ was not considered, so that the riskcould arise of a strategic decision that would setirreversible meeting conditions for some time tocome.

5.3.4. A post-hoc suggestionIn pondering over the common-coffee problem ofthe School of Computer Studies at Leeds I myselfmight have hypothesized that perhaps the depart-ment had been struck by a tendency towardssmaller-group informalities during shorter breakswhich would occur at varying moments of the dayanyway, and that something unusual would haveto be contrived to get staff and graduate studentstogether, if only once every 2 months, for both aprofessionally and socially rewarding meeting. So,my instruction to the assigned DMs might havebeen: ‘find me some good socialization and com-

munication strategies which do either or not in-volve whatever kind of refreshment’.

6. GENERAL OBSERVATIONS ANDDISCUSSION

6.1. The scenario and the experimentPerhaps one of the first conclusions is that, de-spite a folklore among decision analysts that DMsare only motivated to make realistic and usefuljudgements in the context of a real problem,which de facto focuses their attention, there wasno doubt that the DMs and the analysts all‘bought in’ to the role playing in the exercise. Ineach of the problem formulation sessions therewas an ‘emotional’ involvement on the part of theDMs with the issues. They were able to—anddid—state judgements that ‘mattered’ to them.There is one qualification to this: at least oneanalyst admitted to a feeling that ‘Simon wasMacheavellian enough to have buried rather morenasty issues than there actually were in the mess!’.

The mechanism by which the problem was de-veloped depended upon the modification of anearlier problem 2 years before. Simon had facedthis earlier problem when he had been Head ofSchool in his time at Leeds. Two issues which hadled to frictions then—the use of the microwaveand the source of the coffee supply—did not riseto the surface in any of the sessions, despitehaving been noted in the briefing of the DMs.Subsequent discussion showed that they haveceased to be important to the members of theSchool. This suggests that the DMs really didbring the current context to their role playing andthus introduced realism into the exercise. Furtherevidence of this was provided when one of theDMs provided some background on the socialtensions between two research groups in theSchool.

6.2. Different skills of an analystThe workshop was constructed to examine threeapproaches to formulating a ‘decision mess’ into asoluble problem. Some common skills and tech-niques were apparent in all the analysts’ ap-proaches. Initially, all probed the problem area.They encouraged the DMs to widen their view ofthe problem, to devise and explore many alterna-tive solutions. In essence, the analysts were tryingto bound the problem space. This done, each


S. FRENCH ET AL.260

analyst was able to determine the perspectivewhich the DM would like to take on the issues.Without this formulation stage being performedsuccessfully, it is unlikely that a satisfactory solu-tion could ever be agreed. Hence, our overall aimin the workshop to demonstrate and explore somepossible approaches.

This aim was achieved, in that everyone in-volved in the workshop witnessed three differentapproaches from the analysts. Interestingly, thedifferences which arose were not necessarily dueto the alternative requirements of each of theresulting methodologies. It is more likely that theyarose out of the differing personalities of the threeanalysts. Each was working towards a similarstructure, and yet each led the DM through a verydifferent experience. The role which the analystplays in a decision support session clearly influ-ences the event. It is undoubtedly central to thenature of the experience and support offered tothe DM.

Some analysts approach their task as ‘business-men’: they are efficient, demanding and challeng-ing. Alternatively, some analysts act as‘counsellors’: they are calming, sympathetic andcoaxing. In essence, this personality may be cap-tured by the level of direction used by each of thethree analysts. This is not to say that any of theanalysts acted more or less professionally thanany other, but is more a comment on the pressurewhich was exerted on each DM. The personalitywhich is employed probably stems from the objec-tives of the analyst: reflecting, for example,whether placing the DM at ease is more impor-tant than completely solving their problem. Suchobjectives will drive the analyst, and thereforedictate the character of the session. As such, theseobjectives should be made explicit at the outset inorder that the DM has some prior knowledge ofthe nature of the support process he or she willexperience.

An essential skill for an analyst is the ability towithhold personal opinions. The analyst mustmaintain an objective or detached position. Theirrole can be jeopardized by intervening inappropri-ately in a decision analysis session (Phillips andPhillips, 1993). All of our analysts demonstratedthis ability, though admitted afterwards that theyhad practised restraint. Further, the DMs mustrealize that they have ownership of the problem.They must not look to an analyst to pro6ide asolution. Rather, the DMs should be prepared to

work with the analyst to find an acceptable an-swer. It is likely that any solution will performbetter if the DMs feel that they own it.

Another observation relates to the pace of thesession. Even in a somewhat reduced event, last-ing under 2 h, it was apparent that the dynamicsof the modelling varied. The analysts were vigilantthroughout the session. Sometimes a slowing ofpace indicates that the DM is puzzling over anissue, but it could also mean that the DM hasnothing more to say on the subject. The analystneeds to be able to tell the difference betweenthese two events. In our workshop, when ananalyst felt that an avenue had been exhausted, ornoticed that the DM’s interest was beginning towane, they took control and moved the sessiononto alternative ground. This enabled the DM tore-involve himself with the problem and sustaininterest. Further, it promoted confidence in thedecision support session, adding value to theDM’s experience.

6.3. Transferable procedures?We have highlighted some of the skills employedby the analysts. While trying to formulate theproblem, each explored stakeholders, objectives,and the scale of problem. Furthermore, eitherexplicitly or implicitly, they tried to establish waysforward, criteria and values. Clearly, there aremany ways in which this information can beestablished. In this section, we highlight some ofthe procedures which, although not used by all ofthe analysts in the workshop, might well be con-sidered as transferable. That is, the final choice ofdecision model would not influence whether theseapproaches could be used. However, it would notbe fair to suggest that any analyst could use anyapproach. There must be a personal attachment;the analyst must feel comfortable with his/herapproach to use it confidently and in order thatthe DM is put at ease.

Freerk’s approach, whilst helping the DM tofind a simple problem representation, also seemedto be based upon trying to establish the level ofcommitment to solving the problem. His was acautious approach, not wishing to give the im-pression that he would be involved unless therewas an equal undertaking from the DM. Such acommitment could be demonstrated by agreeingto pay a fee to an analyst, but this is not suffi-cient. There must also be a desire to solve theproblem, rather than placating others by giving



an impression of addressing it. Through provisionof an initial representation of the problem, Freerkwas offering the DM a chance to reflect on theproblem and ways forward before pursuing a fullanalysis. The analyst was keen to make use ofmultiple sessions, and therefore broke off at thepoint he felt appropriate for reflection.

Raimo was driven by a desire to produce adeliverable for the DM: to provide the DM with aresource to take back to the School for discussion.He was pursuing his own objective in some sense,though this objective was aimed at supporting theDM. As a consequence, Raimo progressed muchfurther into the decision modelling phase than anyof the other analysts. His approach to encouragingthe DM to appreciate the scope of the problemrequired the DM to generate numbers. Raimoargued that only through providing weights andpreferences could the scope of the problem beunderstood. He supported the gathering of thisdata with software, in order to quickly compute theimpact of such beliefs.

Valerie’s session seemed to be split into twoactivities—acquiring and structuring knowledge(cf. Ackermann and Belton, 1994). Initially, sheand the DM discussed issues, ideas, people, prob-lems and solutions. This approach was used tocapture all thoughts as they surfaced. As such, theDM was allowed to move from one topic toanother, and back again as certain revelations ledto other overlooked thoughts. Because of theanalyst’s approach, there needed to be a flexibleform to hold the information. Once this process ofacquisition of information was close to complete,Valerie moved on to its organization. Clearly, theDM may still add information to the problem, andtherefore knowledge acquisition will progressthrough the later phases of the analysis. Thestructuring phase involved clustering ideas, con-structing alternative stakeholder’s views and build-ing up logical solution strategies. This provided awide view of the problem, enabling the DM tocritically reflect on the various problems which heactually faced. Again, the DM was encouraged tofocus on addressing one of these, and the analysisprogressed.

Eden et al. (1983) suggest three styles of help thatmight be offered by (decision) analysts: coercive,empathetic and negotiative. Valerie’s style wasempathetic moving a little in the direction ofnegotiative. Freerk also operated in the empathetic-negotiative mode, although a little more towards

the negotiative. Raimo clearly leant to the coercivestyle, beginning with some empathetic interactions.

6.4. Discussion about problem formulationReflecting on the workshop, all participants agreedthat it takes a lot of time and effort to get a problemmess into an agreed matrix of values reflectingcriteria, preferences and beliefs. Further, all possi-ble solutions need to be scored against this matrixin order to apply a normative solution procedure.The dynamic nature of problem formulation maylead to divergent working, further complicating theinitial problem. However, this will encourage theDM to consider fully the focus which he/she wouldlike to take. Problem formulation is complicated bythe fact that people’s preferences evolve. There isa danger of an analyst forcing preference values onto the DM in order to perform the analysis. Thiswas not something which was observed in theworkshop, but is a temptation which analystsacknowledge in stressful, demanding situations.

In addition to his/her preferences changing, it islikely that many other aspects of the DM’s perspec-tive will develop during the analysis. DMs are likelyto mention new constraints at each suggestedmodel. As certain issues become more concrete intheir minds, it is likely that they will have revela-tions concerning the problem. Therefore, problemformulation is likely to be cyclic, and each recogniz-able phase could be revisited at any stage. A goodindicator that the model is becoming complete arethe feelings of the DM: unease on their part impliesthat the problem needs further exploration.

A conclusion from the workshop was that it isworth investing a little time at the beginning of thesession in social interaction to build a relationshipbetween the DM and the analyst. The early phasesof any problem formulation are very much basedon good communication between the analyst andDM. Therefore, anything which can facilitate arapport will be beneficial. Further, a few frivolouscomments can give much away about the truenature of the DM’s feelings. A request for anec-dotes or stories regarding the problem will providebackground for the analyst, and may allow the DMto ‘let off some steam’ with regard to the problem.

Further, it was agreed that analysts should bewary when discussing the problem with the DM.The analyst should not take for granted the statedproblem. Indeed, there was a lot of suspiciondisplayed by the analysts at our workshop as theytried to establish the ‘real’ issue. The experiences


S. FRENCH ET AL.262

of those involved showed that there is often moreto a problem than is initially made explicit. TheDM may be encouraged to reveal this if they feelsecure in the session, and if they have confidencein the analyst’s ability to help them face theconsequences of raising the issue. What is gener-ally agreed is that any solution found to thesuperficial problem is not likely to survive unlessthe more fundamental one is addressed.

Some comments arose from the artificial natureof the workshop. Cautionary statements weremade about assuming that MCDA would be theright way to solve a problem. Given the brief ofthe workshop, the analysts involved were safe toassume that the problem would be of an MCDAtype. However, this is an aspect which an analystmust normally establish. The DMs involved com-mented that a ‘route map’ would have been use-ful. They felt unsure about where the session wasgoing. The fact that each of the analysts failed tomake the session outline clear may have been afactor of performing in front of an expert audi-ence. The analysts were in an unrealistic session,being watched by people who were experienced indecision analysis, and therefore may have felt thisunnecessary. A final point emerged due to thehypothetical nature of the scenario. All key play-ers should have been present, including the DM,face-to-face with the analyst for a decision sup-port session. Our analysts and observers notedthat their work would have been constrained hadthey not been able to get all of the problemowners together for the decision formulation.

ACKNOWLEDGEMENTS

We are grateful to Keith Hobley, Tony Jenkinsand Matt Morley for acting as DMs, to MariPoyhonen for her assistance, to the other partici-pants at the workshop, and to the organizers ofSPUDM97 for agreeing to an unconventional for-mat for one of their workshops.

NOTES

1. It is arguable that the term decision maker is incorrect:maybe problem owner would be a better term. But wehave chosen to use the former both because we believethat problem formulation is provided with its motiva-tion from its role in the decision making process andwe wish to emphasize our concern to compare ap-proaches to the support of decision makers.

REFERENCES

Ackermann, F. and Belton, V., ‘Managing corporateknowledge with SODA and V.I.S.A.’, Brit. J. Man-age., 5, 163–176 (1994).

Banxia Software, Decision Explorer, Glasgow: BanxiaSoftware, University of Strathclyde([email protected].), 1996.

Belton, V., Ackermann, F. and Shepherd, I., ‘Inte-grated support for problem structuring through toalternative evaluation using COPE and V.I.S.A.’, J.Multi-Crit. Decis. Anal., 6, 115–130 (1997).

Eden, C., Jones, S. and Sims, D., Messing about inProblems, Oxford: Pergamon, 1983.

Hall, P., ‘Managing change and gaining corporate com-mitment’, ICL Tech. J., 7, 213–227 (1986).

Hamalainen, R.P. and Lauri, H., HIPRE 3+ DecisionSupport Software 6s. 3.13e, Helsinki University ofTechnology, Systems Analysis Laboratory, 1996.(Distributed by EIA Ltd., Tekniikantie 17b, 02150Espoo, Finland. Tel.: +358 9 70018680; fax: +3589 7001 8682.)

Hamalainen, R.P. and Leikola, O., ‘Spontaneous deci-sion conferencing in parliamentary negotiations’, inProceedings of the Twenty-Se6enth Annual HawaiiInternational Conference on System Sciences, Hawaii:IEEE Computer Society Press, January 4–7, Vol. IV,1995, pp. 290–299.

Hamalainen, R.P. and Leikola, O., ‘Spontaneous deci-sion conferencing with top-level politicians—discov-ering the hidden demand for support in politicaldecision making’, OR Insight, 9(1), 24–28 (1996).

Larichev, O., Brown, R., Andreeva, E. and Flanders,N., ‘Categorical decision analysis for environmentalmanagement: a siberian gas distribution case’, inCaverni, J., Bar-Hillel, M., Barron, F. and Junger-mann, H. (eds), Contributions to Decision Research,Amsterdam: North Holland, 1994.

Marttunen, M. and Hamalainen, R.P., ‘Decision analy-sis interviews in environmental impact assessment’,Eur. J. Oper. Res., 87, 551–563 (1995).

Patel, V.L. and Groen, G.J., ‘The general and specificnature of medical expertise: a critical look’, inEricsson, R. and Smith, J. (eds), Toward a GeneralTheory of Expertise, Cambridge: Cambridge Univer-sity Press, 1991.

Phillips, L.D. and Phillips, M.C., ‘Facilitated workgroups: theory and practice’, J. Oper. Res. Soc., 44,533–549 (1993).

Szaniawski, K., ‘Philosophy of decision making’, ActaPsychologica, 45, 327–341 (1980).

Vlek, C., Timmermans, D. and Otten, W., ‘The idea ofdecision support’, in Nagel, S.S. (ed.), Computer-Aided Decision Analysis; Theory and Applications,Westport (Conn.)/London: Quorum Books, 1993, pp.33–68.


Problem formulation for multi-criteria decision analysis: report of a workshop

Documents

Transcript of Problem formulation for multi-criteria decision analysis: report of a workshop