Incomplete Holistic Comparisons in Value Tree Analysis

Helsinki University of Technology Systems Analysis Laboratory

Antti Punkka and Ahti Salo

Systems Analysis Laboratory

Helsinki University of Technology

P.O. Box 1100, 02015 HUT, Finland

http://www.sal.hut.fi/

Incomplete Holistic Comparisons inIncomplete Holistic Comparisons inValue Tree AnalysisValue Tree Analysis


2

Overall goal (a0)

Attribute 1(a1)

)()()( 15

1

15

1

1

iixvwxvxV N

ii ii i

Value tree analysis Value tree analysis

Alternative 1 (x1)

Attribute 5(a5)

Attribute 4(a4)

Attribute 3(a3)

Attribute 2(a2)

Alternative 3 (x3)

Alternative 2 (x2)


3

m alternatives, X={x1,…,xm} , n attributes, A={a1,…,an} Additive value function

Least and most preferred achievement levels – all attributes relevant–

– attribute weight wi represents the improvement in overall value when an alternative’s achievement with regard to attribute ai changes from the least to the most preferred level

Value tree analysisValue tree analysis

n

i

jii

j xvxV1

)()(

*0 , ii xx

Xxxvxvxv ki

kii iii )()()(0 *0

n

i

ji

Nii

n

i ii

jii

iij xvxv

xv

xvxvxV

i1

*

1*

* )()()(

)()()(

]1,0[ iii wxvxvii

)()( 0*

n

iiw

1

1


4

Weight elicitation Weight elicitation

Complete information– captured by point estimates– e.g., SMART (Edwards 1977)

Incomplete information– weight and weight ratio intervals

» e.g.,

– e.g., PAIRS (Salo and Hämäläinen 1992), PRIME (Salo and Hämäläinen 2001)

Ordinal information– ask the DM to rank the attributes in terms of importance– e.g., rank sum weights (Stillwell et al. 1981)– incomplete ordinal information (RICH; Salo and Punkka 2003)

2/5.1 21 ww


5

Incomplete informationIncomplete information

Complete information may be hard to acquire – alternatives and their impacts?– relative importance of attributes?

» e.g.,

Alternatives’ overall values can be represented as intervals– e.g., the smallest and the largest

possible value can be solved through LP

n

i

ji

Ni

Swxvw

i1

)(maxmin/ where S is the feasible region for the attribute weights based on the DM’s preference statements

3w

2w

1w

S23 ww

)0,1,0(

)1,0,0(

)0,0,1(

23 3ww

13 2ww

13 4ww

9

4,

9

4,

9

1

5

2,

5

2,

5

1

11

6,

11

2,

11

3

19

12,

19

4,

19

3

21 5.1 ww 21 25.0 ww

2/1/4/1,1/3/1 3132 wwww


6

Pairwise dominancePairwise dominance

Alternative xk dominates xj in the sense of pairwise dominance– dominated alternative is non-optimal

whenever the DM’s preference statements are fulfilled => it can be discarded

– e.g., a problem with two attributes,

Alternatives may remain non-dominated, however– decision rules assist the DM in selection

of the most preferred one

0)]()([min

lk

SZ

xVxV

where S is the feasible region for scores and weights

V

w1 0.4 0.7w20.6 0.3

x1 dominates x2

)( 1xV

)( 2xV

7.04.0 1 w


7

Decision rulesDecision rules

Maximax– alternative with greatest maximum

overall value

Maximin– alternative with greatest minimum

overall value

Minimax regret– alternative for which the greatest

possible loss of value against some other alternative is the smallest

Central values– alternative with greatest sum of

maximum and minimum overall value

0.22 0.25 0.23

0.60

0.41

0.53

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Alternative 1 Alternative 2 Alternative 3

Alternatives

Val

ue


8

Use of ordinal preference statementsUse of ordinal preference statements

Complete ordinal information – ask the DM to rank the attributes in terms of importance– derive a representative weight vector from the ranking

» rank sum weights (Stillwell et al. 1981)

» rank reciprocal (Stillwell et al. 1981)

» rank-order centroid weights (Barron 1992)

Incomplete ordinal preference information– the DM may be unable to rank the attributes

» ”which is more important - economy or environmental impacts”

ii Rw /1~

1~ ii Rnw

n

Rji

i

jw )/1(~


9

Rank Inclusion in Criteria Hierarchies (RICH)Rank Inclusion in Criteria Hierarchies (RICH)

Associate possible rankings with sets of attributes – e.g., ”economy and environmental

impacts are among the three most important attributes”

– presumably easier and faster to give than numerical statements

– easy to understand– statements define possibly non-

convex feasible regions

Supported by the decision support tool RICH Decisions http://www.rich.hut.fi

“The most important of the three attributes is either attribute 1 or 2”

3w

2w

1w

})({ 11 aS

})({ 21 aS

)0,0,1(

)0,1,0(

)1,0,0(

23 ww


10

Ordinal information in evaluation of the alternativesOrdinal information in evaluation of the alternatives

Numerical evaluation may be difficult– may lead to erroneous approximations on alternatives’ properties (Payne

et al. 1993)– allow the DM to use incomplete ordinal information

Score elicitation– associate sets of rankings with sets of alternatives

» e.g., alternatives 1 and 2 are the two least preferred with regard to environmental impacts

» e.g., alternatives 3 and 4 are the two most preferred with regard to environmental impacts and cost together

– rank two alternatives in relative terms» e.g., alternative 1 is better than alternative 2 with regard to environmental

impacts– can be subjected to

» all attributes (holistic comparisons)» a (sub)set of attributes or a single attribute


11

Incomplete holistic comparisonsIncomplete holistic comparisons

Evaluate some alternatives without decomposition into “subproblems”– comparisons interpreted as (pairwise) dominance relations – e.g., alternative x1 is better than alternative x2

– e.g., alternative x4 is not the most preferred one

Constraints on the feasible region– e.g., normalized scores known– three attributes– alternative x1 is preferred to alternative x2

3w

2w

1w

)()( 21 xVxV

)0,0,1(

)0,1,0(

)1,0,0(

)(59.036.022.0

18.063.072.0)(2

321

3211

xVwww

wwwxV

)55.0,00.0,45.0(

)40.0,60.0,00.0(

S

1 2 3x1's scores 0.72 0.63 0.18x2's scores 0.22 0.36 0.59. . . .. . . .. . . .

attribute


12

(Incomplete) ordinalinformation about the

importance of attributes(RICH)

(Incomplete) ordinalinformation about alternatives,

score informationin form of intervals

(Incomplete) holisticcomparisons

3},,...,1{3 jnjww j 5.0)(4.0),()()( 244

344

244

144 xvxvxvxv N

0)()( 21 xVxV

LPs for 1) overall valueintervals and2) pairwise

dominance relations

Constraints on thefeasible region

Decisionrecommendations

Different forms of incomplete ordinal informationDifferent forms of incomplete ordinal information


13

Rank the alternatives subject to their properties– the most preferred alternative has the ranking one, etc.– e.g., alternatives x1, x2 and x3 ranked with regard to cost: r=(r(x1), r(x2),

r(x3))=(1,2,3)» alternative x1 is the preferred to x2 which is preferred to x3

– the alternative with a smaller rank with regard to some attributes has greater sum of scores with regard to these attributes

– mathematically rA’X’ is a bijection from X’X onto {1,…,m’}, |X’|=m’

Compatible rank-orderings– IX’ is a set of alternatives, J{1,…,m’} a set of rankings

» if |I|<|J|, the rankings of alternatives in I are in J

» if |I||J| , the rankings in J are attained by alternatives in I

» many compatible rank-orderings

» e.g., if m=3, I={x1}, J={1} for A’={a1, a2}, then compatible rank-orderings are

rA’X=(1,2,3) and (1,3,2).

Rank-orderings (1/2)Rank-orderings (1/2)


14

Rank-orderings (2/2)Rank-orderings (2/2)

Feasible region associated with a rank-ordering rA’X’ convex

– can be used as an elementary set

R(I,J) contains the rank-orderings that are compatible with the sets I and J– feasible regions defined by R(I,J) not necessarily convex

Express statements as pairs of Ii, Ji, i=1,…,k

– feasible region is the intersection of the corresponding S(Ii,Ji):s

)(),( ''

),(''

XA

JIRrrSJIS

XA

ki

ii JISS,...1

),(

)}()(whenever)()(|{)( ''

''

''

''

jXA

kXA

Aa

jii

Aa

kii

XA xrxrxvxvZrS

ii


15

Efficiency of preference statementsEfficiency of preference statements

Monte Carlo study– randomly generated problem instances (e.g., Barron and Barret 1994)

» statements are based on them e.g., weight vector e.g., weight vector w=(0.32, 0.60, 0.08)w=(0.32, 0.60, 0.08) approximated through the rank-ordering approximated through the rank-ordering

r=(2,1,3)r=(2,1,3)

» “correct choice”, xC(i) at round i, (i.e., the alternative with the highest overall value) can be obtained

» xe(i) is the alternative recommended by a decision rule at round i

Measures

– expected loss of value (ELV)

– percentage of correct choices (PCC)

– average number of non-dominated alternatives

sn

i

eCs

ixVixVn 1

))](())(([1

ELV

s

C

n

nPCC nC is the number of problems where xe(i)= xC(i)

ns is the number of simulation rounds


16

Efficiency of holistic comparisonsEfficiency of holistic comparisons

Questions– how effective are holistic comparisons?– differences between strategies in choosing the compared alternatives

Randomly generated problems– n=5,7,10 attributes; m=5,7,10,15,50 alternatives– each weight vector has the same probability– scores completely known, randomly generated

» uniform distribution, Uni[0,1]» triangular distribution, Tri(0,1/2,1)

Three strategies for choosing the alternatives for pairwise comparisons– each applied in two different ways

» “disconnected comparisons”, x1 vs. x2, x3 vs. x4, etc.» “chained comparisons”, x1 vs. x2, x2 vs. x3, etc.


17

Simulation layoutSimulation layout

Elicitation strategies– A. arrange the alternatives in a descending order by the sum of the

scores (strategies SoS1 (disconnected) and SoS2 (chained))– B. arrange the alternatives in a descending order by the score of the most

important attribute (strategies MIA1 and MIA2)– C. arrange the alternatives randomly (strategies Rnd1 and Rnd2)

ELV and PCC was studied using central values, maximax, maximin and minimax regret decision rules

100 problem instances (simulation rounds)– several linear programs are needed– results indicative– parameter variation (m,n and the number of comparisons) leads to 114

combinations, experiments


18

Simulation results (1/2)Simulation results (1/2)

Sum of Scores is the best strategy– SoS1 outperforms MIA1 in 113 of 114 experiments in terms of ELV

» in 82 of these the difference in loss of value significant risk level at 2.5% for a 1-tailed t-testrisk level at 2.5% for a 1-tailed t-test

– SoS1 outperforms Rnd1 in every of the experiments in terms of ELV» in 92 of these the difference in loss of value significant

– no clear difference between MIA1 and Rnd1

Chained comparisons are better than disconnected comparisons in terms of ELV and percentage of correct choices


19

Simulation results (2/2)Simulation results (2/2)

Holistic comparisons reduce the number of non-dominated alternatives efficiently– e.g., m=50, n=5, the average number of non-dominated alternatives was

between 3.92 and 9.17 with 10 comparisons, depending on the strategy– e.g., m=50, n=5, with only one comparisonone comparison the average number of non-

dominated alternatives was between 20.57 and 23.69» by discarding one alternative, an average of almost 30 were eliminated

Triangularity assumption increases efficiency


20

ConclusionConclusion

Incomplete ordinal information enhances possibilities in preference elicitation– presumably easier and faster to give than numerical statements– easy to understand

Screening of alternatives– holistic comparisons efficient in discarding non-optimal alternatives– useful especially in problems with many alternatives

» consequences of alternatives may be time-consuming to obtain» constraints on the feasible region

Further research directions– efficient computational procedures– simulation study on the efficiency of incomplete holistic comparisons– implementation of a decision support system– case studies


21

Related referencesRelated referencesBarron, F. H., “Selecting a Best Multiattribute Alternative with Partial Information about Attribute Weights”, Acta

Psychologica 80 (1992) 91-103.

Barron, F. H. and Barron, B. E., “Decision Quality using Ranked Attribute Weights”, Management Science 42

(1996) 1515-1523.

Edwards, W., “How to Use Multiattribute Utility Measurement for Social Decision Making”, IEEE Transactions on

Systems, Man, and Cybernetics 7 (1977) 326-340.

Payne, J. W., Bettman, J. R. and Johnson, E. J., “The Adaptive Decision Maker”, Cambridge University Press,

New York (1993).

Salo, A. ja R. P. Hämäläinen, "Preference Assessment by Imprecise Ratio Statements”, Operations Research 40

(1992) 1053-1061.

Salo, A. and Hämäläinen, R. P., “Preference Ratios in Multiattribute Evaluation (PRIME) - Elicitation and

Decision Procedures under Incomplete Information”, IEEE Transactions on Systems, Man, and Cybernetics

31 (2001) 533-545.

Salo, A. and Punkka, A., “Rank Inclusion in Criteria Hierarchies”, (submitted manuscript; 2003).

Stillwell, W. G., Seaver, D. A. and Edwards, W., “A Comparison of Weight Approximation Techniques in

Multiattribute Utility Decision Making”, Organizational Behavior and Human Performance 28 (1981) 62-77.

Incomplete Holistic Comparisons in Value Tree Analysis

Documents

Transcript of Incomplete Holistic Comparisons in Value Tree Analysis