Can We Avoid Biases in Environmental Decision Analysis ?
description
Transcript of Can We Avoid Biases in Environmental Decision Analysis ?
![Page 1: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/1.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Can We Avoid Biases in Environmental
Decision Analysis ?
Raimo P. Hämäläinen
Helsinki University of Technology
Systems Analysis Laboratory
www.paijanne.hut.fi
![Page 2: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/2.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Structure of the presentation
• Background & decision analysis interviews
• Goals of the study
• Case: Regulation of Lake Päijänne
• Splitting bias & swapping of levels
• Description of the experiment
• Results of the experiment
• Conclusions ?
![Page 3: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/3.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Environmental decision analysis
• Parliamentary nuclear power decision(Hämäläinen et. al)
• Decision analysis interviews(Marttunen & Hämäläinen)
• Spontaneous decision conferencing in nuclear emergency management(Hämäläinen & Sinkko)
![Page 4: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/4.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Cognitive biases
• Splitting bias– attribute receives more weight if it is split
– origins: subjects give rank information only (Pöyhönen & Hämäläinen)
– Not observable in hierarchical weighting
![Page 5: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/5.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Decision analysis interviews
• Opinions of large groups of people traditionally collected through questionnaires
• Decision analysis interviews may provide a more reliable way to collect these opinions
• Idea:– one value tree for all = common terminology
– emphasis on finding the viewpoints of different stakeholder groups
– interactive, computer supported
![Page 6: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/6.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Research interest
• Existence of biases in a real case
• Can biases can be avoided through training and proper instructing ?
• Identify what can go wrong in the Lake Päijänne case
• Compare the well trained university students’ and spontaneous stakeholders’ responses
![Page 7: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/7.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
The Lake Päijänne case
• Regulation started 1964
• Main aims were to improve hydroelectricity production and to reduce damages caused by flooding
• Environmental values & increase in free time
• need for an improved regulation policy
![Page 8: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/8.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Splitting bias
• When an attribute is split, the weight it receives increases
Attribute 1
Attribute 2
Attribute 3
Attribute 1
Attribute 2
Attribute 3
subattribute 1b
Subattribute 1a
0.4
0.3
0.3
0.3
0.3
0.1
0.3
0.4
![Page 9: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/9.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Swapping of levels
• Does the order of the levels affect the resulting weights?
• Important question in environmental decision analysis:– stakeholder groups may vary regionally
• Not studied before
![Page 10: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/10.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Example of swapping of levels
Lake Päijänne
River Kymijoki
Attribute 3
Attribute 3
Attribute 2
Attribute 1
Attribute 2
Attribute 1
Attribute 3
Attribute 2
Attribute 1
River Kymijoki
Lake Päijänne
River Kymijoki
Lake Päijänne
River Kymijoki
Lake Päijänne
![Page 11: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/11.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Earlier experiments on biases
• Structure of the decision model affects the results
• Previous experiments typically:– subjects: university students
– problems: artificial
– results: taken from group averages
• Lake Päijänne-case: a real problem with real stakeholders
![Page 12: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/12.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Important new features
• Realistic case
• Decision analysis interviews instead of passive decision support or survey
• Interactive computer support (resulting weights shown immediately)
• Instructions and training before the weighting
![Page 13: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/13.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Subjects:
• University students attending a course on decision analysis (N = 30)– held during a tutorial session, not mandatory
• Habitants of Asikkala (N = 40)– 3 groups of students
– 1 group of adults (volunteers)
• 3 experts from the Finnish Environment Institute & 2 summer residence owners
![Page 14: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/14.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Experimental setting
• Weighting done with the SWING method using a tailored Excel interface
• Subjects entered the numbers themselves, two assistants were present to help
• Resulting weights shown as bars
• Order of value trees partly randomized
![Page 15: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/15.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Sessions
• A short introduction to: – Lake Päijänne case
– value trees & weighting
– different structures of the value tree
• In HUT the avoidance of biases was emphasized more
• Duration: 60 - 90 minutes
![Page 16: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/16.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
SWING method
• Easy to use
• Attribute ranges clearly presented
• Idea:– choose the attribute you would first like to
move to its best level
– assign it 100 points
– assign other attributes points less than 100 in respect to the first attribute
![Page 17: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/17.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Flat-weighting
Muu talous ???
VesivoimaVesivoima
Muu talous
Ympäristö
Talous
Rantojen käytettävyys
Virkistyskalastus
Kalojen lisääntyminen
Rantakasvillisuus
Lahtienumpeenkasvu
Virkistys
Luonto
Tulvat, maatalous jateollisuus
Tulvat, loma-asutus
Vesiliikenne
Ammattikalastus
![Page 18: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/18.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Upper level weights:
Muu talous ???
VesivoimaVesivoima
Muu talous
Ympäristö
Talous
Rantojen käytettävyys
Virkistyskalastus
Kalojen lisääntyminen
Rantakasvillisuus
Lahtienumpeenkasvu
Virkistys
Luonto
Tulvat, maatalous jateollisuus
Tulvat, loma-asutus
Vesiliikenne
Ammattikalastus
![Page 19: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/19.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
ENV5-tree:
Luonto
Virkistys
Ympäristö
Talous
Rantojen käytettävyys
Virkistyskalastus
Kalojen lisääntyminen
Rantakasvillisuus
Lahtienumpeenkasvu
![Page 20: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/20.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
ENV2-tree:
Luonto
Virkistys
Ympäristö
Talous
![Page 21: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/21.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
EC5-tree:
Muu talous ???
VesivoimaVesivoima
Muu talous
Ympäristö
TalousTulvat, maatalous ja
teollisuus
Tulvat, loma-asutus
Vesiliikenne
Ammattikalastus
![Page 22: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/22.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
EC2-tree:
Muu talous ???
Vesivoima
Muu talous
TalousMuu talous ???
Vesivoima
Muu talous
Ympäristö
Talous
![Page 23: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/23.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Swapping of levels:
Muu talous ???
Kymijoki ja muut
Päijänne
Rantakasvillisuus
Tulvavahingot
Kymijoki ja muut
Päijänne
Muu talous ???
Rantakasvillisuus
Tulvavahingot
Kymijoki ja muut
Päijänne
Rantakasvillisuus
Tulvavahingot
![Page 24: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/24.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Flat weights vs. upper level weights
• Both in group averages and in results of individuals the total weights for the environment and economy were similar with both methods
• One explanation: symmetric value tree
![Page 25: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/25.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Splitting bias0
.00
.30
.50
.81
.0
ASIKKALA HUT
Means of Wenv
GROUP
Wenv
TREE
EC2EC5ENV2ENV5
![Page 26: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/26.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
A typical resident in Asikkala
0
0.2
0.4
0.6
0.8
1ENVIRONMENT ECONOMY
5 1 5 2 1 1 5 1 1 1 5 2
![Page 27: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/27.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Example from HUT(one of the best ones)
0
0.2
0.4
0.6
0.8
1ENVIRONMENT ECONOMY
5 1 5 2 1 1 5 1 1 1 5 2
![Page 28: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/28.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Why even weights ?
• Some students: none of the attributes seemed to be important
• Asikkala: all of the attributes were important
even weights for all attributes
![Page 29: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/29.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
What caused the bias ?
• Similar points for
all attributes in one branch
regardless of the structure
of the value tree
100 80
80100 7070 70 50
100 80 70
![Page 30: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/30.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Effect of instructions
• Students had good instructions– only some had bias in their results
• In the spontaneous stakeholders’ sessions the information load was too high and thus the instructions were not adopted as well– nearly all had systematically consistent bias
![Page 31: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/31.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
n
nenv
econ
n
nenv
econ
w
wenv
econ
w
wenv
econ
STUDENTS STAKEHOLDERS
Adjusted / not adjusted weights
![Page 32: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/32.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Examples
01234567
0 1 2 3 4 5
01234567
0 1 2 3 4 5
STUDENTS STAKEHOLDERS
w
w
env
econ
w
w
env
econ
n
nenv
econ
n
nenv
econ
![Page 33: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/33.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Observation
• The students and the experts from FEI could nearly avoid the splitting bias– good background education + instructions did
reduce the bias
• What did the students think? - Arithmetics or real avoidance of biases
![Page 34: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/34.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Avoiding the splitting bias ?
• Good instruction can eliminate it
• When the economical attributes were split, the magnitude of the bias was slightly larger
• Graphical feedback did not eliminate
• Hierarchical weighting
![Page 35: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/35.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Swapping of attribute levels
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
If the order of the levels would not affect the weigts, the pairs of weights should be equal (as in the first picture)
![Page 36: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/36.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Conclusions about swapping of levels ?
• Only a few had clearly differing weights with the two trees
• No systematic pattern was found
• Less differences residents of Asikkala and students than with the splitting bias
• A simple scale lead to similar weights with both trees (100, 70 for example)
• Neither tree gained clear support
![Page 37: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/37.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Solutions to reduce biases ?
• Hierarchical weighting
• Models should be tested on real decision makers
• Interactiveness of weighting (= possibility to return to change the points given earlier )
• Well balanced trees
![Page 38: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/38.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Other observations in Asikkala
• Concept of weight seemed to be difficult for most subjects in Asikkala
• Information load was high
• Facilitators role becomes important when the DM’s are uncertain
![Page 39: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/39.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Problems related to the Lake Päijänne case
• Current regulation policy cannot be improved very significantly – no big differences between the alternatives
– unrealistic hopes and false information are probably larger problems than the regulation itself
• ‘money is not money’– strong feelings against the power companies
and regulation (shape of value function ?)
![Page 40: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/40.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
Suggestions for future research
• Hierarchical weighting
• Encouragement to reconsider and readjust the statements iterate
• Decision Analyst must supervise!
![Page 41: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/41.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
R.P. Hämäläinen, E. Kettunen, M. Marttunen and H. Ehtamo: Evaluating a framework for multi-stakeholder decision support in water resources management, Group Decision and Negotiation, 2001. (to appear)
M. Pöyhönen, Hans C.J. Vrolijk and R.P. Hämäläinen: Behavioral and procedural consequences of structural variation in value trees. European Journal of Operational Research, 2001. (to appear)
M. Pöyhönen and R.P. Hämäläinen: There is hope in attribute weighting, Journal of Information Systems and Operational Research (INFOR), vol. 38, no. 3, Aug. 2000, pp. 272-282. Abstract
R.P. Hämäläinen, M. Lindstedt and K. Sinkko: Multi-attribute risk analysis in nuclear emergency management, Risk Analysis, Vol. 20, No 4, 2000, pp. 455-467.
M. Pöyhönen and R.P. Hämäläinen: Notes on the weighting biases in value trees, Journal of Behavioral Decision Making, Vol. 11, 1998, pp. 139-150.
Susanna Alaja: Structuring effects in environmental decision models, Helsinki University of Technology, Systems Analysis Laboratory, Theses, 1998.
References
![Page 42: Can We Avoid Biases in Environmental Decision Analysis ?](https://reader036.fdocuments.in/reader036/viewer/2022081511/56813bf5550346895da53f80/html5/thumbnails/42.jpg)
S ystemsAnalysis LaboratoryHelsinki University of Technology
M. Pöyhönen, R.P. Hämäläinen and A. A. Salo: An experiment on the numerical modeling of verbal ratio statements, Journal of Multi-Criteria Decision Analysis, Vol. 6, 1997, pp. 1-10.
R.P. Hämäläinen and M. Pöyhönen: On-line group decision support by preference programming intraffic planning, Group Decision and Negotiation, Vol. 5, 1996, pp.485-50.
M. Marttunen and R.P. Hämäläinen: Decision analysis interviews in environmental impact assessment, European Journal of Operational Research, Vol. 87, No. 3, 1995, pp. 551-563.
R.P. Hämäläinen, A.A. Salo and K. Pöysti: Observations about consensus seeking in a multiple criteria environment, in: Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences, Vol. IV, 1991, IEEE Computer Society Press, Hawaii, pp. 190-198.
R.P. Hämäläinen: Computer assisted energy policy analysis in the parliament of Finland, Interfaces, Vol. 18, No. 4, 1988, pp. 12-23. Also in: Case and Readings in Management Science, 2nd edition, M. Render, R.M. Stair Jr. and I. Greenberg (eds.), Allyn & Bacon, Massachusetts 1990 pp. 278-288.