Multi-attribute decision support in the event of a nuclear accident

JOURNAL OF MULTI-CRITERIA DECISION ANALYSIS, VOL 5, 39-57 (1996)

Multi-Attribute Decision Support in the Event of a Nuclear Accident

SIMON FRENCH School of Computer Studies, University of Leeds, Leeds LS2 9Jb U.K.

ABSTRACT Retrospective studies of nuclear accidents such as those at Three Mile Island and Chernobyl have emphasized the need for preparing structured decision support methodologies for use in any future emergency. This paper discusses the multi-attribute aspects of such decision support, i.e. methods for supporting decisions when there are several conflicting objectives. Considerable progress has been made in application of these methods over the past 4 or 5 years. In particular, the decision-conferencing format has proved successful in helping senior decision makers understand and deal with the issues that arise in considering long-term countermeasures made. Within the RODOS project, a European initiative to build a decision support system for emergency response, multi-attribute value and utility techniques for short-term and medium-term countermeasures are being developed. However, much remains to be done, particularly in relation to the incorporation of uncertainty when there is a risk of an imminent accident.

KEY WORDS: Bayesian decision analysis; decision support for environmental emergencies; decision trees; multi-attribute resource allocation; multi-attribute utility analysis; multi-attribute value analysis; radiation protection decisions; RODOS

1. INTRODUCTION

Major nuclear accidents such as those at Three Mile Island and Chernobyl have focused attention on the need for coherent decision support decision making on countermeasures. I became involved in such matters during the International Chernobyl Project which was run during 1990-1991; see Section 2. The Project demonstrated that decisions on countermeasures are driven not just by the need to avert dose to the population and hence reduce radiation-related health effects, but also by the need to tackle population stress and to devise strategies that are acceptable to the public. The varied response to the Chernobyl accident both in and beyond the former Soviet Union led the Commission of the European Communities (CEC) to support a number of projects to further our understanding of the issues raised by such accidents and to build decision support systems (DSSs) and methodologies for use in the event of a future accident. One such is the RODOS Project beal time on-line decisiGn Support system) which is outlined in Section 3. The aim of this system is to support decision making from the moment that an accident threatens, when immediate precautionary countermeasures may be necessary, through to the longer-term decision making which may need to consider stress and public acceptability issues as

0 1996 by John Wiley & Sons, Ltd CCC 1057-9214/96/010039-19

well as any radiation-related health effects. Later sections of the paper consider the different decision support needed over the life history of an accident and suggest how multi-attribute value and utility theory may help. (French et al. (1995) consider the issue of uncertainty modelling. Related work on forecasting the spread of contamination is reported in companion papers by Smith and French (1993) and Smith et al. (1994).)

There are several themes running throughout this paper.

Multi-attribute methods are extremely useful in structuring the problems faced by decision makers in formulating and evaluating countermeasure strategies. Cost benefit methods alone are not sufficient, because they cannot easily deal with the ‘softer, intangible’ issues such as stress and public acceptability. Modem decision support systems gain from the use of graphical interfaces to communicate with a variety of decision makers who may possess qualitatively different skills and perspectives, e.g. scientists, medical personnel, engineers, emergency planners, government officials and senior politicians. The format of a decision conference (facili- tated workshop) is a very effective way of

Accepted 3 October 1995

40 S . FRENCH

formulating and evaluating the broad strategy for long-term decision making.

Throughout, familiarity is assumed with multi- attribute value and utility theory, see e.g. Keeney and Raiffa (1976), French (1986, 1989) and Keeney (1992).

2. THE INTERNATIONAL CHERNOBYL PROJECT DECISION CONFERENCES

In 1986 a very severe nuclear accident occurred at Chernobyl in the U.S.S.R. A nuclear reactor was completely destroyed, releasing radionuclides and radioactive debris for about 10 days. Four years later the International Chernobyl Project was undertaken at the request of the Soviet Authorities. Seven international agencies partici- pated, co-ordinated by the International Atomic Energy Authority (IAEA). The Soviet request was for

‘. . . an international experts’ assessment of the concept which the USSR has evolved to enable the population to live safely in areas affected by radioactive contamination following the Chernobyl accident, and an evaluation of the effectiveness of the steps taken in these areas to safeguard the health of the population’.

The Project had to assess the steps taken by the Soviet Authorities to enable the affected populations to live safely. It was appreciated that it would not be adequate to concentrate on the radiological protection aspects alone. It was clear from a number of sources that social and political factors, inter a h , were also affecting the decision making. Accordingly, the Project description, as approved by the International Advisory Committee, included the use of multi-attribute decision-aiding techniques to investigate and capture these factors (International Chernobyl Project Advisory Committee, 1991).

There are many decision-aiding techniques and a decision had to be made on which to use and in what manner they should be applied. The Project required a form suited to group decision making, since many ministers and scientists had provided important inputs. Decision conferencing, supported by fairly simple (additive value) decision models, seemed to offer the best way forward. Decision conferencing is very effective at stimulat-

ing discussion and eliciting issues. Also, decision conferences are short, intensive events which fitted well with the Project’s tight timescales.

Briefly, a decision conference is a 2 day event at which a group of people who are responsible for formulating and implementing policy meet to discuss all the major issues and concerns that relate to a current problem and to choose a way forward. To help them in their task, they are assisted by a facilitator and an analyst who attend to the process and decision modelling, leaving the group free to concentrate on the content of their problem. The facilitator and the analyst assist by keeping the decision focused, by building decision analytic models and by ensuring that all present understand the varied perspectives brought by their colleagues. In this way a shared understanding of both the problem and the way forward can be arrived at more quickly than through unaided discussion. The decision models are projected on a large screen for all the group to see the results. Typically a sequence of models is built, each a revision or development of the previous, which keep pace with the group’s evolving view of the problem. For further details see Eden and Radford (1990) and French (1989).

Initially it was decided to hold four decision conferences, one each in the Republics of the Ukraine and Byelorussia and the Russian Federation respectively and one at All-Union level. The purposes of these decision conferences were (1) to enable some of the decision problems

related to the Chernobyl accident to be structured efficiently and thus clarify and elucidate issues,

(2) to summarize for the International Chernobyl Project the key socioeconomic and political factors that together with the physical, radiological and medical evidence influence the relocation and protective measures taken in the Republics,

(3) to illustrate the use and potential benefits of formal decision analysis methods and the techniques of decision conferencing for the resolution of complex issues.

Subsequently a fifth decision conference was held at which representatives from the earlier conferences met to build a summary model that represented the consensus view of main issues and concerns elicited at the earlier events. This section describes the principal conclusions drawn

MULTI-ATTRIBUTE DECISION SUPPORT FOR NUCLEAR ACCIDENTS 41

Normal Living

Effects - Health Public

Acceptability

Radi-1 Related I I * I 4-l I

Fatal Hereditary Stress Affected Region Rest of Resources Cancers Related USSR

Figure 1. The attribute hierarchy used in the fifth decision conference

from this decision conference. Further details may be obtained in International Chernobyl Project Advisory Committee (1991), French et al. (1992) and Lochard et al. (1992). It should be noted that, strictly, the Project did not organize five decision conferences, but rather five events with the structure of decision conferences. There was no intention to guide - only to understand - the decision making of the various authorities in the - Soviet Union.

Generally, the countermeasure strategies considered at these conferences took a common form. Of the population living in the affected region, about 700 000 people: some, who would otherwise receive a high lifetime dose, would be permanently relocated; some, who would otherwise receive a lower lifetime dose, would remain in their homes but be protected by Countermeasures such as decontamination of buildings and changes in agricultural practice; and the remaining group, who would receive a very small lifetime dose, would be allowed to remain in their homes with no further countermeasures.

The attribute (or objective) hierarchy developed in the fifth summary decision conference is shown in Figure 1. The effect on Health provided by a countermeasure strategy was generally seen as

Table I. Some details on the strategies

having two components: the effect it had in reducing Fatal Cancers and Hereditary consequences and the effect it had in terms of increasing or decreasing Stress Related effects. The Radiation Related effects could be estimated from the dose averted in the protected and relocated populations. Precise data were not available in a form suitable to estimate quickly the dose saved by each strategy. Accordingly, approximations and judgement were used. The Stress Related effects were a subject of much debate in all the coriferences. The success of strategies in reducing such effects was judged subjectively, using the experience of medical personnel and officials who worked in the regional governments. Public Acceptability of the strategies was felt to be an important attribute. Many dimensions of this attribute were identified. However, two distinct factors stood out: the acceptability to the population in the Affected Region and to that in the Rest of USSR. These attributes were assessed entirely judgementally.

The different Resources required or costs were clearly an important factor in choosing between strategies. In the final conference this attribute was expressed simply as the cost in billions of roubles. Other measures related to the difficulty of implementing the countermeasures had been investigated at the earlier conferences but had led to much the same conclusions.

After exploring many strategies in the earlier conferences, the final conference considered four representative strategies. Table I provides some details on these strategies. The b in the labelling of the strategies SL2-b reflects the predicted future lifetime dose value used to separate those relocated from those protected by other means.

There is not space here to describe in full the multi-attribute value analysis, including the elici- tation of scores and weights: details can be found in the references cited above. It is sufficient to look at Figure 2, which shows a sequence of plots of

Number protected Estimated number Estimated number Number relocated by other means of fatal cancers of hereditary effects Cost (billions of

Strategy ( 1000s) ( 1000s) averted averted roubles)

SL2-2 706 SL2-10 160 SL2-20 20 SL2-40 3

0 546 686 703

3200 1700 650 380

500 260 100 60

28 17 15 14

42 S. FRENCH

SL2-2

SL2-10

SL2-20

SL2-40

Trade-off line - corresponding

10 alpha value

Resources

Figure 2a: Plot of radiation related effects againrr resources

0 25 50 75 100

Resources

Figure 2b Plot of health effects against resources

0 2 5 50 75 100

Resources

Plot of effects against resources Figure 2c:

Figure 2. Plots of different effects against Resources: (a) Radiation Related effects; (b) Health effects; (c) Eflects. These plots were produced using the HIVIEW (Barclay,

1984) software package

effects against the resources required (costs) of the strategies.

In Figure 2(a) the Radiation Related effects are plotted against Resources. Note first that the Resources scale is a preference scale, as indeed are all the other scales in the model. This means that greater scores correspond to increasing

preference and, in this case, decreasing cost. Moreover, conventionally all scales are renormalized to have a range 0-100. Thus on the Resources scale, here the horizontal axis, strategy SL2-40 has a score of 100 since it was the cheapest, whereas SL2-2 has a score of 0 since it was the most expensive. The vertical axis gives the scores for the strategies on the Radiation Related effects, again renormalized to a 0-100 scale with the most preferred strategy SL2-2 scoring 100.

Figure 2(a) corresponds to a cost effectiveness plot such as might be used in a cost benefit analysis. Such analyses commonly provide a basis for radiation protection decisions (Stoke11 et al., 1991) and, as they are used, take account only of medical factors directly related to the radiation and economic costs. Within the radiation protection community the trade-off between economic costs and radiation-related health effects is known as the alpha value, a. In 1990 all reasonable* values for a were such that SL2-40 was the ‘optimal’ strategy.

Figure 2(b) gives a similar plot in which the vertical axis now represents the overall Health effects, given by combining the Radiation Related and Stress Related scales. Note that although the Health scale is normalized so that the maximum and minimum possible scores are 100 and 0 respectively, because no strategy was either best in terms of both Radiation Related and Stress Related scales or worst in terms of both, the full range of the Health scale was not used.

Figure 2(c) again differs in the vertical scale. In this case the attribute Effects which arises from combining the Radiation Related, Stress Related and Public Acceptability scales is plotted. Again, because no single strategy was best or worst, on all three component scales the full &lo0 range of Effects was not used.

In each of the figures the trade-off between the two attributes used as axes is shown by a dotted line. In Figure 2(a) this corresponded directly to the U-value. In Figures 2(b) and 2(c) it corresponded to combining the a-value with judgemental weights on the further attributes which had been introduced.

When Stress Related effects and Public Acceptability were included as in Figures 2(b)

*i.e. all values for a which had been proposed by various national or international bodies or which had been used in other radiation protection decisions.


and 2(c), it may be seen that SL2-20 moved towards the top right-hand comer, i.e. in the direction of increasing preference on both horizontal and vertical scales.

From Figure 2(c), which in a sense represents a synthesis of all the preference information, it was clear that SL2-2 and SL2-I0 could never be optimal: they were dominated by SL2-20 on both the Effects and Resources scales. SL2-40 could be optimal, since it beat SL2-20 on the Resources scale: it was cheaper. However, it could only beat SL2-20 overall if the weight on Resources was sufficiently high, so that the poorer performance on the Effects scale was more than compensated by the financial savings. In fact, no member of the conference felt that it was appropriate to place as high a value on the weight for this to occur. Thus the analysis pointed strongly to the choice of

Without venturing further into the analysis, it can be seen from the sequence of plots in Figure 2 that the wider issues of Stress Related health effects and Public Acceptability had a considerable effect on the analysis. This finding has been confirmed in a number of other studies, e.g. French et al. (1993).

Thus decision analyses and decision support systems designed to guide decision making after a nuclear accident must address such issues, ideally in a clear and explicit fashion. Conventional cost benefit analyses (Stoke11 et al., 1991) will not be sufficient because of their inability to include intangible attributes satisfactorily. It should also be noted that the decision-conferencing format for supporting groups of decision makers proved very successful in the International Chernobyl Project. This influences heavily the approach proposed in Section 5.

SL2-20.

3. DECISION SUPPORT SYSTEMS FOR COUNTERMEASURES IN THE EVENT OF A

NUCLEAR ACCIDENT: RODOS

To understand how multi-attribute decision analyses can support decision making during and after a serious accidental release of radioactivity, we need to discuss briefly the chronology of events that are likely to occur. Figure 3 outlines a (very!) simplified view of this.

During the building and running of nuclear plants, many plans and preparations are made to deal with potential emergencies. Indeed, no plant

would be licensed for normal operation unless such preparations were made. Databases of demographic, agricultural, economic and geographic data are established so that all relevant information will be to hand in the event of an accident. Evacuation routes and procedures are planned for a number of accident scenarios. Emergency exercises are held regularly and involve the operators, local police, fire and ambulance services, regional officials and perhaps national officials. No accident ever goes ‘as planned’, however. Moreover, the public are seldom, if ever, exercised. Thus exercises should be seen as an education so that those involved in deciding on the most suitable measures to deal with an accident will be sensitive to issues that may arise rather than programmed to follow some predeter- mined rules.

In addition to the emergency planning specific to the plant, much national and international guidance for each class of countermeasure (e.g. sheltering, evacuation or food bans) is given in the form of intervention levels. Generally these are lower and upper levels on the predicted public and worker dose. Below the lower level the advice is that intervention is unnecessary, whereas above the upper level it is required. Between the lower and upper levels the action to be taken is left to the discretion of the authorities concerned in the light of the particular circumstances of the accident.

When there was a serious and imminent risk of an accident, a number of actions would be taken. Firstly and most obviously, plant oficials and engineers would take appropriate engineering actions to avoid or reduce the risk of a release. The decision making involved in this is not discussed here. We are concerned with decisions on countermeasures which, by and large, affect the public. The first of these is whether to take any precautionary measures such as (1) warn the public, (2) distribute stable iodine tablets, (3) begin evacuation of any areas.

Support for such decision making is discussed briefly in Section 6 . We defer even brief discussion of it until then, because one of the factors which must be considered is the probability that a release will actually occur. It is easier to discuss the methodology for doing t h s after we have considered the multi-attribute analyses which deal with the circumstances that arise once a release has occurred.

44 S . FRENCH

Normal working

I \I/

Possible accident imminent

Emergency planning; maintenance of local and geographic databases.

Engineering measures to control accident. Gather information on meteorological conditions. Ask experts for best judgements on scale of release and its likelihood. Forecasting of possible plume spread and dose. Decide on whether to implement precautionary countermeasures.

Accident?

Engineering measures to safe plus public announcement and calming measures

Release

1 Immediate

post-release

1 Long-term

I I

w

Engineering measures to stop Continuous forecasting of spread and dose, updated with monitoring data. Decide on immediate countermeasures.

Continuous forecasting of spread and dose, updated with monitoring data. Decide on shortlmedium term countermeasures.

Decision making on long- countemeasures, including relocation.

Figure 3. Simplified chronology of decision making on countermeasures during an accident

Immediately a release occurs, decisions are (1) food bans, needed on countermeasures such as

(1) (issue and) uptake of stable iodine, ( 2 ) sheltering, (3) evacuation.

Other examples of countermeasures which will Possible methodologies and techniques for need to be considered in the days immediately supporting such decisions are discussed in after a release are

(2) decontamination of livestock and agricultural

(3) decontamination of properties, (4) restriction on activities, (5) restriction on access to the region.

produce,

Section 4.


It is appropriate to note at this point that while decisions during the early phases of the accident will need to consider issues such as the psycho- logical stress and the political acceptability of the actions to the public, the driving consideration in all decision making during this phase is to avert dose. Moreover, the decisions are likely to be the responsibility of local plant management, regional emergency planning officers and senior officers in the emergency services, who are not in a position to express societal value judgements with authority. In the later phases, however, the issues of stress and public acceptability are likely to become much more important and perhaps the driving issues, as they were found to be in the Chernobyl study. Thus in decision making for the long term, when it may be necessary to consider countermeasures such as permanent relocation and permanent changes to agricultural practice and local industry, it is likely that regional and national politicians will be much more closely involved. There will be a need to address value trade-offs between intangible attributes and ‘harder’ attributes such as cost and averted dose (radiation-related health effects saved). In Section 5 I argue that support for these longer-term decisions may be provided most effectively in a decision-conferencing format similar to that used in the Chernobyl study.

Within the Framework research programme the CEC is supporting the development of an inte- grated and comprehensive DSS, named RODOS, for nuclear emergencies, which will be applicable from the early phase through to late phases of an accident. It is intended that its geographical scope will extend from the vicinity of the accident to far distant regions. RODOS is being developed through the co-operation of many European institutions and several Russian, Byelorussian and Ukrainian laboratories. While much of its architecture is agreed, it should be noted that development is continuing and the following should not be taken as a definitive statement of its structure. For a fuller description of its current state than the sketch given here, see Ehrhardt et al. (1993). Discussion of how the design of RODOS may be interpreted with a Bayesian framework may be found in French et al. (1995).

Essentially there are three types of subsystem within the conceptual architecture* of RODOS.

*i.e. not the formal software architecture which includes database systems, graphical interfaces, etc.

Analysing subsystems (ASY). These maintain a continually updated forecast of the spread and distribution of any contamination. Countermeasure subsystems (CSY). These identify possible countermeasures (mixes of sheltering and evacuation for the local population, food bans, etc.) and quantify the various costs and benefits (e.g. averted dose). Evaluation subsystems (ESY). These support evaluation of the different countermeasure strategies and produce a ranked shortlist.

The operational concept of RODOS assumes that it will operate in a semi-automatic mode in the early phases of any emergency, prompting and suggesting actions to local management, regional emergency planning officers and senior officers in the emergency services. It will not require any value judgements as input in this mode of operation, only monitoring data and expert judgement on the prevailing meteorological conditions and the likely spectrum and scale of release of the source term. The prime consideration in the early phases is to avert dose. In later phases, when it may be necessary to take into account intangibles such as stress and public acceptability and when more senior politicians and officials are on hand, the mode of operation will be more interactive and reflective, seeking value judgements from the decision makers and indicating the consequences of these.

The discussion has considered only accidents at nuclear plants, be they power or reprocessing plants. There are other possible accident scenarios, such as might arise from an accident during the transportation of nuclear waste. The RODOS project does not address such scenarios and they are not discussed in this paper. None the less, many of the underlying ideas, principles and methods in this and later sections would transfer to other circumstances with some modification.

4. DECISION ANALYSIS RELATING TO

COUNTERMEASURES IMMEDIATE AND MEDIUM-TERM

Typically the region around a nuclear plant is divided into a number of areas for emergency planning purposes; see Figure 4. Detailed plans will made in advance of any accident as to how the emergency services should implement each possible countermeasure in each area. It is assumed that

46 S. FRENCH

., . -

Figure 4. Division of region around a plant for emergency planning

any countermeasure will either be applied or not be applied to an entire area. Consider, for instance, the early countermeasures of sheltering, evacuation and uptake of stable iodine. A possible strategy for early countermeasures might be as follows.

(1) Advise sheltering in areas Al, A2, A3, C1, C2 and C3.

(2) Evacuate areas B1, B2 and B3. (3) Advise uptake of stable iodine in areas Al,

A2, A3, B1, B2, B3, C1, C2 and C3. (4) No immediate action in other areas.

The strategy above is but one of many that will need to be considered and evaluated in the event of an accident. In Figure 4 there are 8 x 4 = 32 planning areas. In each, four actions are possible. Thus there are 324 possible strategies. There are three complicating factors, however. The first increases the number of possible strategies considerably. There are more than four countermeasures that might be taken within an area. For instance, the advice to shelter is not given in an unqualified form. Rather the advice would be 'Shelter for the next n hours', where n might be 4, 12, 24 or whatever. Moreover, there are compound actions: 'Shelter for 4 hours while the plume passes overhead, then evacuate'. The second complicating factor actually simplifies the problem. Clearly, if a strong wind is blowing to

the north-west (top left) in Figure 4, one is unlikely to consider evacuation to the south-east. Thus the number of actions which would be considered in some of the planning areas may be just one (do nothing) or two (do nothing or advise sheltering).

The third complicating factor again reduces the number of strategies available considerably. One would never evacuate part of a small village; the public would never accept or understand such an action. A more specific example might occur in the areas immediately adjacent to the plant. Suppose the meteorological conditions are such that the plume rises steeply in the vicinity of the plant and does not contaminate the areas close to the plant. Some distance away, however, it returns to ground level. Then the strategy which would suggest itself on purely radiation protection issues might well be to shelter near the plant and evacuate further away. However, the public are unlikely to understand or accept this advice. To be acceptable, the strategy would need to evacuate a region extending from the plant until far enough downwind that the expected dose was sufficiently small to be negli- gible. Thus public acceptability is likely to reduce further the number of possible strategies to be considered. The strategies need to exhibit a continuity of treatment.

We shall return to these three points later in this section. In the meantime we will work through an example using a multi-attribute value-based resource allocation method.

Note that the example strategy given above could have been described by listing not the areas to which each action is applied but the actions applied to each area.

A1 . Advise sheltering, uptake of stable iodine A2. Advise sheltering, uptake of stable iodine A3. Advise sheltering, uptake of stable iodine B1. Evacuate B2. Evacuate ... ... We shall use this second format in the following.

The example concerns medium-term food control actions. An accident has occurred and affected a region, which has been divided into three areas. (we use three areas, not the 32 in Figure 4, to simplify the presentation.) The population in each area by age group is given in Table 11.

The two radiation-related health effects that are of concern are Collective Dose and Individual


Table 11. Demography of affected areas

Infants Children Adults ( G l years)

Area 1 50 1000 4000 Area 2 250 5000 20 000 Area 3 3000 60 000 240 000

Thyroid Dose. We shall consider strategies for reducing these doses to each age group in each area. We shall make certain assumptions.

1. Consideration of equity between groups in the population is deemed to suggest that the worth of averting dose is the same in each area. However, the worth of the averted dose is not necessarily the same for children and adults. Nor is the worth of averting dose necessarily the same for collective and individual dose.

2.

3.

In each area we need to consider countermeasures in respect of three agricultural produces: milk, meat and vegetables. Certain actions are possible for each.

Milk None: no action Decontamination: treat the milk to decontaminate Transformation to most of contamination left in butter: skimmed milk

Meat None: no action Decontamination: add bentonite to animals’ feed

before slaughter Vegetables None: no action Destruction: destroy and deep bury Preservation: preserve until radioactive

products decay

Within each area, treatments must be applied uniformly. Tables 111-V give details of the costs and averted doses given by the possible countermeasures in the three areas. Because of assumption 1, the averted collective doses and the averted thyroid doses have been accumulated over the three areas.

We will use a multi-attribute value resource allocation package, EQUITY (Barclay, 1989, to see what may be learned.

Essentially there are nine variables of ‘pots’ into which we may allocate resources.

Pot 1. Area 1: milk countermeasures Pot 2. Area 1: meat countermeasures Pot 3. Area 1: vegetable countermeasures Pot 4. Area 2: milk countermeasures Pot 5. Area 2: meat countermeasures Pot 6. Area 2: vegetable countermeasures Pot 7. Area 3: milk countermeasures Pot 8. Area 3: meat countermeasures Pot 9. Area 3: vegetable countermeasures

Given Tables 111-V, a countermeasure strategy may be described by the level of resource assigned to each pot. For example, one strategy might be as follows.

cost (ECUS)

Pot 1 Area 1: milk-transform to butter Pot 2 Area 1: meat-decontaminate Pot 3 Area 1: vegetable-preservation Pot 4 Area 2: milk-decontaminate Pot 5 Area 2: meat-none Pot 6 Area 2: vegetable-destruction Pot 7 Area 3: milk-decontaminate Pot 8 Area 3: meat-none Pot 9 Area 3: vegetable-preservation

1030 5 200

10 OOO 31 500

0 20 000 14 500

0 75 000

Total cost: 157230

A little thought shows that there are 33 x 23 x 33 = 5832 possible strategies. EQUITY explores these strategies by using a greedy algorithm which allocates resources to gain the greatest benefit; see Everett (1963). So that the algorithm is able to do this, it is necessary to define an overall measure of benefit.

Initially assume that the attributes of concern are as in Figure 5, i.e. assume that, apart from cost, the strategies are to be evaluated in terms of the averted collective dose and their acceptability to the public. We will discuss the inclusion of thyroid dose averted shortly. The inclusion of an attribute to represent the effect on stress of the strategies is straightfor- ward and can be achieved in the same manner as the inclusion of public acceptability. In any case, intangibles such as stress and public acceptability might not be included in the analysis for short-term countermeasures.

In analysing this model, some simplifying assumptions have been made about how decision

48 S. FRENCH

Table 111. Costs and averted doses of countermeasures in Area 1

Collective dose (ManSv) Thyroid dose (ManSv)

Countermeasure Cost (ECUs) Infant Child Adult Infant Child Adult -

Milk None Decontamination Transformation to butter

Meat None Decontamination

Vegetables None Destruction Preservation

0 685

1030

0.00 0 0 0.0 0.04 400 11 0.4 0.04 630 36 0.4

0 5050 5280

0 166 186

0 5200

0.00 0 0 0.0 0.14 30 39 0.5

0 400

0 26

0 6000

loo00

0.00 0 0 0.0 0.04 80 6 0.0 0.04 40 1 0.0

0 630 320

0 166 86

Table IV. Costs and averted doses of countermeasures in Area 2


Countermeasure Cost (ECUs) Infant Child Adult Infant Child Adult

Milk None 0 0 0 0 0.0 0 0 Decontamination 3150 0 260 58 0.4 6500 266 Transformation to butter 4720 0 880 58 0.4 6800 276

Meat None 0 0 0 0 0.0 0 0 Decontamination 26 200 0 20 48 0.4 400 0


0 0 0 0 0.0 0 0 20 000 0 110 11 0.0 200 66 33 OOO 0 95 1 0.0 100 31

Table V. Costs and averted doses of countermeasures in Area 3


Countermeasure Cost (ECUs) Infant Child Adult Infant Child Adult __._




0 0.00 0 0 0.0 14 500 0.04 700 38 0.4 21 780 0.04 810 38 0.4

0 5900 6200

0 186 186

0 0.00 0 0 0.0 63 700 0.04 10 8 0.4

0 400

0 0

0 0.00 0 0 0.0 45 000 0.04 380 26 0.0 75 000 0.04 190 11 0.0

0 3020 1510

0 506 256


Overall

Benefit

Collective dose

Public Infant Child Adult Acceptability cost

Figure 5 . Initial attribute hierarchy for example

makers would assess the public acceptability of the different countermeasures.

Doing nothing is the least acceptable option in respect of all three foodstuffs. Because milk is perceived as a food taken particularly by infants and children, it is assumed that it is most important in the public’s eyes that this is decontaminated. Thus the different countermeasures were scored as in Table VI. The overall benefit score was assumed to be a weighted sum of the three collective dose scales and the public acceptability scale:

benefit = 5 x infant collective dose averted+ 2 x child collective dose averted+ 1 x adult collective dose averted+ 4 x public acceptability score

Table VI. Public acceptability scores. Greater values =+ increasing preference

Public Countermeasure acceptability ~~




0 300 240

0 100

0 100 20

Benefit 100

80

60

40

20

0

o 48130 96250 144380 in500 240630 c o s t

Figure 6 . Cost-benefit frontier. Benefit scale: percentage of maximum achievable. N.B. The direction of the

cost scale is reversed from Figure 2

The weights (5:2:1:4) were chosen in this example arbitrarily, but in practice would be assessed by swing weighting techniques (Keen- ey and Raiffa, 1976). Note, however, that the different weights on infant, child and adult collective dose are consistent with assumption 2 above.

Figure 6 shows the cost benefit frontier generated by EQUITY. Note that the vertical axis shows the benefit on a scale of 0-100. Think of this as the percentage of the maximum benefit attainable if resources were unlimited. EQUITY provides a means of finding points on the cost- benefit boundary (efficient frontier or Pareto boundary) that correspond to strategies which achieve a given percentage of the maximum benefit or which demand at most a given level of resource. By exploring the problem in this way, the strategy, labelled B in Figure 7, was found. In practice, this exploration would be driven by consideration of reasonable a- values.

However, this strategy breaks some of the continuity-of-treatment constraints that might be imposed. Milk is transformed to butter in two areas and decontaminated in one. Will the public be happy to accept a policy in which milk is treated non-uniformly? Consider the strategy, labelled P in

50 S. FRENCH

Strategy B Cost (ECUs) Benefit

Pot 1 Area 1: milk-transform to butter Pot 2 Area 1: meat-decontaminate Pot 3 Area 1: vegetable-destruction Pot 4 Area 2: milk- transform to butter Pot 5 Area 2: meat-none Pot 6 Area 2: vegetable-destruction Pot 7 Area 3: milk-decontaminate Pot 8 Area 3: meat-none Pot 9 Area 3: vegetable-destruction

Total (

1030 5 200 6 000 4 720

0 20 000 14 500

0 45 000

:ost: 96450 -

Strategy P Cost (ECUs)

Pot 1 Area 1: milk-transform to butter Pot 2 Area 1: meat - none Pot 3 Area 1: vegetable-destruction Pot 4 Area 2: milk-transform to butter Pot 5 Area 2: meat - none Pot 6 Area 2: vegetable - destruction Pot 7 Area 3: milk-transform to butter Pot 8 Area 3: meat-none Pot 9 Area 3: vegetable-destruction

Total

1030 0

6000 4 720

0 20 OOO 21 780

0 45 OOO

cost: 98530 ~

Benefit

100

so

60

40

20

0 0 48130 156250234380312500360630

c o s t

Figure 7. Cost-benefit plot showing positions of strategies B and P

0 48130 96250 144380 192500 240630 cost

Figure 8. Cost-benefit plot of individual thyroid dose against cost in ECUs

Figure 7, in which we have modified B by forcing it to fit the continuity constraint.

Strategy B provides 93% of the maximum benefit and strategy P provides 89% of the maximum benefit; see Figure 7. As can be seen, the strategies are very close. Thus the decision makers might be well persuaded to adopt strategy P.

So far we have not considered individual thyroid doses. Figure 8 shows a cost-benefit plot in which the benefit is a weighted sum of the individual thyroid doses in the ratio infant:child:adult=5:2: 1. Neither collective dose nor public acceptability was included in this evaluation. Again it can be seen that strategy P is close to the cost-benefit frontier. There is a slightly more expensive but more effective strategy*, labelled B, which lies on the cost-benefit frontier and a cheaper but less effective one, labelled C , again lying on the frontier. Thus strategy P again looks a reasonable strategy to adopt. In practice, perhaps one would explore the effectiveness of P against the individual dose to each of the age groups rather than to a weighted average across the groups. However, the approach should be clear.

*In fact, strategy B here is not the same strategy as in Figure 7, but that need not detain us.

MULTI-AITRIBUTE DECISION SUPPORT FOR NUCLEAR ACCIDENTS 51

Coarse MAVIUT Expert Ranking

Filler Slrategies + System ---D of --D

Though the direction we are taking here looks promising, there are a number of problems that should be noted. Firstly, national and international bodies have set intervention levels for the majority of countermeasures. These introduce further constraints. In the example above, we did not consider the possibility that intervention levels might require intervention, say, to reduce the dose arising from meat. Secondly, we have suggested that the need to ensure continuity of treatment between the areas may constrain the choice of strategy. It was entirely fortuitous that we were able to find a strategy which satisfied a (fairly arbitrary) constraint of this type. Thirdly, there may be practicability constraints; for instance, it is no good planning to transform all the milk to butter if there is not sufficient capacity to do so at the dairies. Thus, while the development above may be pedagogically informative, it may not point us directly to the solution of realistic problems.

In RODOS we are tackling the problems another way. The CSY and ESY subsystems (see above) will interact as follows. The CSY subsystem will identify and quantify the costs and benefits of possible countermeasures. In effect, it will construct tables of the form of Tables 111-V, but with many more (possibly) relevant summary statistics. The ESY subsystem will have to perform the functions shown in Figure 9.

First, a very simple expert system will be used to discard strategies which are incompatible with the requirements of intervention levels, which do not give continuity of treatment or which fail very coarse practicability rules. This coarse expert system is being implemented using constraint management techniques; see e.g. Tsang (1993) and Van Hentenryck (1989). This technology is in one sense as old as combinatorial programming, for it does nothing other than identify objects that satisfy a set of constraints. However, in another sense it is very new in drawing upon modern tree

Fine Expert System * Filter

ESY

Figure 9. The conceptual structure of an ESY module

search and list manipulation algorithms implemented with AI languages. We are currently building such a coarse expert system. Early experiments show that constraint satisfaction technology is well able to cope with the combinatorial problems we face here.

The strategies satisfying the constraints imposed by the coarse expert system will be passed to a multi-attribute value (MAV) ranking module which will identify the top 10 to 20 ranking strategies. The operator will be able to use interactive sensitivity analysis, similar to that in the VISA software package (Belton and Vickers, 1993), to confirm that these strategies are worthy of careful consideration. A prototype of the MAV ranking module, HERESY, has been written at Leeds. Figure 10 shows a screen dump of this.

HERESY’S purpose is to identify the top few, say 10, ranking strategies and check the sensitivity of these to the choice of weights. The screen is divided into three areas. At the top the attribute hierarchy is shown. In the middle is a histogram showing the overall scores (values) of the top 10 strategies. At the bottom is a histogram showing the current weights wi on (some of) the attributes. Not all wi need be shown at the same time. There may be cognitive advantages in concentrating attention on particular branches within the attribute hierarchy. All bars on the histogram are labelled appropriately. The user selects a weight with a mouse by clicking on the appropriate bar in the bottom histogram and then increases or decreases the weight either by the keyboard or by pulling with a mouse. As the weight is changed, the middle histogram changes accordingly. When a change in the ranking of the top 10 strategies occurs (or when one drops out of the top 10 and another enters), the histogram rearranges itself. The machine also beeps and informs the user of the change in a text window. This means that the user can identify the sensitivity of the ranking to the default weights in the model.

The computational speed of the prototype confirms that the identification of the top 50 ranking strategies from about 10 000 countermeasure strategies and the associated sensitivity analysis can be performed almost instantly on a Silicon Graphics Indigo R3000, a less powerful machine than the Hewlett-Packard workstation planned as the support machine for RODOS.

These strategies would then be passed on to an expert system with a much finer and more sophisticated system of rules, each of which

52 S. FRENCH

Figure 10. Screen dump from HERESY

could be applied to each of the candidate strategies. The small number of strategies would allow a full set of explanations to be developed, which would give a critique ofeach of the strategies. Thus the output of RODOS will be a shortlist of strategies, each of which satisfies the constraints implied by intervention levels, practicability, etc., together with a detailed commentary on each strategy explaining its strengths and weaknesses. Klein (1994) discusses a similar combination of expert system technology with MAV/MAU ideas to

provide decision makers with explanatory remarks on the ranking of strategies.

5. DECISION ANALYSIS RELATING TO LONG-TERM COUNTERMEASURES

Once short-term and medium-term countermeasures have been decided upon and implementation is proceeding, decision-making activity will need to address the longer term. Analysts will have


formulated various possible strategies for countermeasures such as permanent relocation and permanent changes in agricultural and industrial practices. Moreover, they will have investigated these in terms of costs and the collective and individual dose, together with its distribution, which would be averted. These strategies will have been developed according to the principles and practice in, say, the ALARA procedures (Stoke11 et al., 1991). There will now be a need to select a single strategy from the range offered in the light both of the information provided and also of wider non-radiation-related health, socio- political and economic issues. As has been indicated, this decision making is likely to be the responsibility of senior regional and national politicians. It is here that decision conferencing can help.

Firstly, the possible strategies have been developed by ‘back-room’ analysts and advisers. The lists of possible countermeasures will come accompanied by the tables of data associated with the cost-benefit analysis. The decision makers will ‘own’ neither the strategies nor the supporting analyses. Given the possible consequences of their decision, there will be a great temptation to hide behind the data, to decide by choosing a value of c1

the monetary value per unit dose saved and to leave the ‘decision’ to the cost-benefit analysis. More subtly, they may hide explicit analysis of the wider issues in their choice of a. Thus it will be necessary ‘to put the decision back in their laps’ (French, 1992). Decision conferencing, with its attention to group processes and its ability to focus discussion, can do this. Moreover, its use of decision models, particularly multi-attribute ones, provides the support the decision makers need to include the wider issues in the analysis.

The decision conference held within the International Chernobyl Project pointed the way towards this form of decision support and demonstrated its potential effectiveness. More recently the BER-3 group of the Nordic Co- operation Organisation investigated its potential further in an exercise (French et al., 1993). The primary objectives of this exercise were

to achieve a common understanding between decision makers and local government officials on the one hand and the radiation protection community on the other of the issues that arise in decisions in the aftermath of a major nuclear accident,

(2) to identify issues which need to be considered

(3) to explore the use of decision conferencing as a

To achieve these objectives, some 30 people drawn from the radiation protection communities and local and regional governments of five Nordic countries (Denmark, Finland, Iceland, Norway and Sweden) took part in a simulated decision conference. The participants were invited to consider a scenario of a hypothetical radiation accident. The scenario assumed that appropriate early protective actions (sheltering, issuing of iodine tablets, etc.) had been taken and that the conference was meeting some 8 days into the accident to consider medium- and longer-term protective actions, particularly the need for relocation of certain areas. By the end of the conference, considerable consensus on the general form of the strategy had emerged. Moreover, there was a better understanding of the evaluation criteria against which such a strategy needed to be developed.

It is interesting that the attribute hierarchy which was developed showed considerable similarity to that which was used in the final Chernobyl decision conference: compare Figures 1 and 11. The radiation-related effects were broken down into collective and individual doses rather than expected number of fatal cancers and hereditary effects. Moreover, many participants felt that it was important to retain flexibility in the strategy of protective actions, even if this increased the uncertainty for the affected population, who would not know exactly what would be done for several months. None the less, the similarity between the two hierarchies is remarkable; and I believe not solely due to my role as facilitator at both events, although that common factor must have had some effect.

One difference was that while the health effects due to stress and public acceptability of the strategies were important attributes, they were not given so high a weight in the BER-3 conference as they were in the Chernobyl ones.

In evaluating the decision-conferencing format, several points were made during the concluding discussion.

1. All participants felt that having many varied perspectives present in the meeting had been useful. It had contributed to a fuller and shared understanding of the problems likely to

in preparing guidance on intervention levels,

format for major decision making.

54 S. FRENCH

Overall

2.

3.

4.

5 .

6.

Flexibility Acceptability Dose Dose cost

Figure 1 1. Hierarchy of evaluation criteria used in the BER-3 study

be faced in the event of a major nuclear accident. Most felt that the software (HIVIEW and VISA) had been useful. Its graphical, visual display of sensitivity analyses had helped focus discussion. There was a suggestion that the meeting would have progressed faster if the evaluation attributes had been defined more fully earlier. However, this may be a comment made with hindsight. It is commonly found in decision conferences that one repeatedly revisits the definition of the attributes as understanding of the issues evolves. Such an iterative, evolu- tionary process seems almost inevitable. Ab initio definition of criteria is very difficult. Because of the nature of the exercise, the set of strategies was kept more or less fixed during the conference. If the conference had been for real, the set of strategies would undoubtedly have evolved as understanding of the issues, cost and effects grew. Much more technical support would have been available in a real conference. For instance, when a new strategy was suggested, there would have been radiation protection analysts available to estimate its costs and to predict its effect in terms of averted dose and maximum individual dose. This would have meant that discussion might have developed faster and in a more focused manner than it did at the meeting. There was a feeling that the meeting was too large at nearly 30 participants. Much of the reason for its size was to ensure adequate representation of the five Nordic countries.

Certainly in real circumstances a decision conference would be smaller and the grouping more tightly focused on the issues deriving from real circumstances. One participant suggested that whether or not decision conferencing would be a useful tool in the event of a real emergency, it was clearly a useful tool in stimulating discussion in planning and emergency preparedness, as it had been at this conference.

Thus the idea of using decision conferencing to support the final stages of decision making shown in Figure 3 seems promising. The design of RODOS is such that it can provide a further focus to such meetings by offering quick ‘what if analyses of the effects of particular countermeasures on public and worker doses and on costs. Where more development is clearly needed is in judging the effects of certain countermeasures on stress, public acceptability and other intangibles. The current methodology requires that the participants at the decision conference make these judgements directly. It is possible that the many studies into stress and public opinion being undertaken in the former Soviet Union will provide more structured assessment techniques.

7.

6 . DECISION ANALYSIS WHEN THERE IS A RISK OF AN IMMINENT ACCIDENT

The discussion so far has ignored uncertainty. The multi-attribute value approach taken here belongs to the Bayesian school of decision analysis (French, 1986). Yet we have not considered using


utility functions to model both multi-attribute preferences and attitudes to risk, nor have we used subjective probabilities to model uncertainty. Predicted doses are just that: predictions. All predictions have uncertainty and these are no exception. In any accident we would be uncertain about many things that affect the predicted dose: the source term; the quantities of the different radionuclides released; their atmospheric dispersion and deposition; actual public compliance with advice; transport through food chains; and so on. Moreover, the connection between dose and effects is uncertain. As yet we have no models, inaccurate or otherwise, to predict quantitative measures of stress and public acceptability. There is uncertainty in all aspects of the problems discussed here. The Bayesian paradigm would have us model these uncertainties probabilistically, our preferences by multi-attribute utilities, and then would guide our choice between strategies by expected utilities. Why have we not done this?

While undoubtedly part of the reason is that we wish to progress from deterministic to probabilistic analysis one step at a time, there is a technical reason which suggests that our approach using multi-attribute values is a good approximation to that using expected utilities. Stewart (1994) has recently shown that using multi-attribute values evaluated at the expectations of the attributes is often a good approximation to a full expected utility analysis; see also Von Winterfeldt and Edwards (1986). However, note the need to use expectations of the attributes. Therefore subjective probabilities do enter the analysis in constructing the predictions. Thus within RODOS the ASY subsystems are being developed to include fully Bayesian forecasting methods (French et al.,

Stewart (1995) identifies some circumstances in which the approximation of multi-attribute utility analyses by multi-attribute value analyses is valid. In our case intuition justifies the approximation on the grounds that none of the possible outcomes differs significantly in qualitative ways: there will be some fatal cancers, some hereditary effects, some stress effects and so on. The uncertainty is in matters of number and degree.

However, there is one decision noted in Figure 3 in which some of the possible outcomes do differ significantly in qualitative ways, namely whether to initiate precautionary measures when there is a risk of an imminent accident. Here the outcomes differ considerably in the case that there is an

1995).

accident from the case that there is not. The situation is easy to model using decision

trees (or influence diagrams). However, there are two complications to the analysis.

1.

2.

Consideration needs to be given to the assessment of appropriate risk tolerances, which are the parameters in utility function that encode attitudes to risk. The ability to perform sensitivity analyses on these parameters may lessen the difficulty of this task, but at present it is one not being addressed within the RODOS projects. In radiation protection decisions concerning countermeasures after an accident, it is conventional to use averted dose as an attribute and not consider the absolute dose. Indeed, it is considered good practice to do so. However, in these decisions there is always some dose to avert; moreover, the dose if no countermeasures are applied is fixed (although it may only be known within certain bounds). Thus all averted doses are calculated from the same reference point. When there is a risk of a release, there is a possibility that no release will occur and therefore there will be no dose to avert. Furthermore, the scale of the release is unknown. If averted dose is used as an attribute, then the values will be calculated from different reference points depending on the scale of the accident. In particular, all strategies will score 0 in the case of no accident, which may be an inappropriate value to input to the analysis as it provides no discrimination between the strategies. Deci- sion analysts will recognize some of the issues here which make theories of regret so difficult to formulate satisfactorily.

Further work is being undertaken within the RODOS project to overcome these problems. We are running exercises with groups of decision makers in Belgium, France, Germany and the U.K. These are designed, among other purposes, to identify the needs of decision makers in deciding upon precautionary measures. Since we are working with politically sensitive issues, preliminary results are still subject to confidentiality restric- tions. However, we are confident that we shall be able to report on these exercises and our developing approach to support decisions on precautionary measures in a future paper.

56 S. FRENCH

7. CONCLUDING REMARKS

This paper has surveyed several related applica- tions of multi-attribute value analyses to the support of radiation protection decision making following an accidental release of radioactivity. Considerable progress has been made in its application over the past 4 or 5 years. In particular, the decision-conferencing format has proved successful in helping senior decision makers understand and deal with the issues that arise in considering long-term countermeasures. Progress is being made within the RODOS project in developing multi-attribute value analytic support for short-term and medium-term countermeasures. However, much remains to be done, particularly in relation to the incorporation of uncertainty when there is a risk of an imminent accident.

ACKNOWLEDGEMENTS

Much of the above research was supported by grants from the Commission of the European Communities

and by funding from the BER-3 group of the Nordic Co-operation Organisation. The ideas in this paper were developed in close co-operation with Main Despres, Joachim Ehrhardt, David Ranyard, Lisa Simpson and Daniel Vanderpooten. I am also grateful for many helpful discussions to V. E. Belton, R. M. Cooke, G. Fraser, G. N. Kelly, J. Lochard, M. Morrey, L. D. Phillips, D. Rios Insua, A. Salo, K. Sinkko, J. Q. Smith, T. J. Stewart and 0. Walmod-Larsen. The example in Section 4 was simplified from one prepared by Alain Despres. Shimon Young programmed the prototype of the HERESY decision analysis software.

(B17-0060-GB, F13P-CT92-0036, F13P-CT92-013b)

REFERENCES

Barclay, S., HIVIEW Software Package, London School of Economics, 1984.

Barclay, S., EQUITY Software Package, London School of Economics, 1985.

Belton, V. and Vickers, S., VISA Software Package, Department of Management Science, University of Strathclyde, Glasgow, 1993.

Eden, C. and Radford, J. (eds), Tackling Strategic Problems, London: Sage, 1990.

Ehrhardt, J., Paler-Sauer, J., Schiile, O., Benz, G., Rafat, M. and Richter, J., ‘Development of RODOS, a comprehensive decision support system for nuclear emergencies in Europe - an overview’, Radiat. Pro- tect. Dosim., in press (1993).

Everett, H., ‘Generalised Lagrange multiplier method for solving problems of optimum allocation of resource’, Oper. Res., 11, 399-417 (1963).

French, S., Decision Theory: An Introduction to the Mathematics of Rationality, Chichester: Ellis Hor- wood, 1986.

French, S. (ed.), Readings in Decision Analysis, London: Chapman and Hall, 1989.

French, S., ‘The use of decision conferencing to determine intervention levels and countermeasures after a nuclear accident’, Proc. Int. Seminar on Intervention Levels and Countermeasures for Nuclear Accidents, Cadarache, 1991, Luxembourg: Commis- sion of the European Communities, 1992, EUR 14469, pp. 586-594.

French, S., Kelly, G. N. and Morrey, M., ‘Decision conferencing and the International Chernobyl Pro- ject’, J . Radiol. Protect., 12, 17-28 (1992).

French, S., Ranyard, D. C. and Smith, J. Q., ‘Un- certainty in RODOS’, Res. Rep. 95-4, School of Computer Studies, University of Leeds, 1995.

French, S., Walmod-Larsen, 0. and Sinkko, K., ‘Decision conferencing on countermeasures after a large nuclear accident’, Rise-R-676(EN), Ris0 National Laboratory, Roskilde, 1993.

International Chernobyl Project Advisory Committee, The International Chernobyl Project: Technical Re- port, Vienna: International Atomic Energy Agency, 1991.

Keeney, R. L., Value-Focused Thinking: A Path to Creative Decision Making, Cambridge, MA: Harvard University Press, 1992.

Keeney, R. L. and Raiffa, H., Decisions with Multiple Objectives: Preferences and Value Trade-offs, Chi- Chester: Wiley, 1976.

Klein, D. A., Decision-Analytic Intelligent Systems: Automated Explanation and Knowledge Acquisition, NJ: Lawrence Erlbaum Associates, 1994.

Lochard, J., Schneider, T. and French, S., International Chernobyl Project - Input from the Commission of the European Communities to the Evaluation of the Relocation Policy Adopted by the Former Soviet Union, Luxembourg: Commission of the European Cornmu- nities, 1992, EUR 14543 EN.

Smith, J. Q. and French, S., ‘Bayesian updating of atmospheric dispersion models for use after an accidental release of radiation’, The Statistician, 42,

Smith, J. Q., French, S. and Ranyard, D. C., ‘An efficient graphical algorithm for updating esti- mates of the dispersal of gaseous waste after an accidental release’, in Gammerman, A. (ed.), Probabilistic Reasoning and Bayesian Belief Networks, Henley-on-Thames: Alfred Waller, 1994,

Stewart, T. J. ‘Simplified approaches for multi-criteria decision making under uncertainty’, .J. Multi-Crit. Decis. Anal., 4, 246-258 (1994).

501-51 1 (1993).

pp. 125-144.


Stokell, P. J., Croft, J. R., Lochard, J. and Lombard, J. , Van Hentenryck, P., Constraint Satisfaction in Logic Programming, Cambridge, MA: MIT Press, 1989.

Von Winterfeldt, D. and Edwards, W., Decision Analysis and Behavioural Research, Cambridge: Cambridge University Press, 1986.

ALARA: From Theory Toward Practice, Luxem- bourg: Commission of the European Communities, 1991, EUR 13796.

Tsang, E., Foundations of Constraint Satisfaction, New York: Academic, 1993.

Multi-attribute decision support in the event of a nuclear accident

Documents

Transcript of Multi-attribute decision support in the event of a nuclear accident