D 3.5.5 STAKEHOLDER ANALYSIS AND KNOWLEDGE ELICITATION ... · Title STAKEHOLDER ANALYSIS AND...

D 3.5.5 STAKEHOLDER ANALYSIS AND KNOWLEDGE ELICITATION TRAINING WORKSHOP

25-26 MARCH, 2008 - USHGOROD Report of the NeWater project - New Approache s to Adaptive Water Management under Uncertainty www.newater.info

Title STAKEHOLDER ANALYSIS AND KNOWLEDGE ELICITATION TRAINING WORKSHOP

25-26 MARCH, 2008 - USHGOROD

Purpose Documentation of the workshop including evaluation

Filename D 3.5.5

Authors Martine Poolman, Sukaina Bharwani, Dagmar Haase (Facilitators), Michael Fischer (KnETs software developer)

Document history

Current version. Final

Changes to previous version.

Date

Status C

Target readership WB 3 partners

General readership Readers interested in Knowledge Elicitation Tools in the context of Adaptive and Integrated Water Resources Management

Correct reference Poolman, M. Bharwani, S. Haase, D and Fischer, M (2008): Stakeholder Analysis and Knowledge Elicitation Training Workshop. Documentation of the workshop in Ushgrod, 25-26 March 2008. NeWater Deliverable 3.5.5

Dagmar Haase, Case Study Leader UFZ Centre for Environmental Research Leipzig-Halle

April 2008

Prepared under contract from the European Commission

Contract no 511179 (GOCE) Integrated Project in PRIORITY 6.3 Global Change and Ecosystems in the 6th EU framework programme

Deliverable title: Report on Review of IWRM Concepts and success in

transferring them into practice Deliverable no. : D 3.5.5 Due date of deliverable: Month 28 Actual submis sion date: 02.04.2008 Start of the project: 01.01.2005 Duration: 4 years

iii

Policy Summary

Part I: Participants learned more about stakeholder processes and left the workshop with more experience, practice and confidence in how stakeholder-issue analyses. Stakeholder processes are of importance when setting up river basin management plans and to introduce the concept of Adaptive Management. The workshop helped participants to learn how to identify important stakeholders, how to carry out an inventory of perceptions, goals, interests and resources. Further, an introduction was given into prioritizing stakeholders, assess dependencies and assess network of stakeholders. Last not least tools for stakeholder analysis were part of the first day: SWOTs, Role Play Games, and Influence Network Mapping etc.

Part II: Identifying decision criteria and getting closer to human behaviour is fundamental for the understanding of decision-making processes and actions that shape our landscape and its environmental resources. Often we are confronted with multiple reasons for land use decisions of multiple actors that have a stake on one resource (e.g. land, water resources, soil fertility etc.). Participatory methods such as Knowledge Elicitation Tools (KnETs) represent a new and reproducible way to formalise this knowledge using computational techniques and, what is more, to implement scenario techniques within the interviews. KnETs can be understood as an amplified methodology of classical social science empirical tools such as interviews and questionnaire surveys. It produces input data for the logic of agent-based models (ABM), decision trees or decision ranking. KnETs link qualitative and quantitative representations of stakeholder knowledge. The fusion of both was realised in form of an iterative process that incorporates a set-up phase of the game conducting interviews, a formalisation phase of the game, an empirical data collection phase of “gaming” with respective respondent groups of interest and a resulting decision tree creation and interpretation phase. Finally, another game round was conducted with a non-involved respondents’ group to verify the results of the models and to assess our gained knowledge on decision making.

iv

AGENDA

Day 1, 10:00am – 18:00pm

“Enhanced Stakeholder-Issue Analysis Training”

Martine Poolmann, Delft University of Technology, the Netherlands

9:45 Introduction

- Expectations of training

10:30 AWM introduction

- IWRM and AWRM compared

- How Stakeholder-Issue Analysis fits in

11:15 - Coffee/Tea Break

11:30 Exercise: AWRM and IWRM

- Questions & Discussion

12:15 Lunch

13:15 Exercise part 2 Stakeholder Analysis

- Introduction to Stakeholder-Issue Analysis

14:30 Coffee/Tea Break

14:45 Examination of SI-analysis steps (interactive)

- Exploration of tools/instruments available to use

15:45 Coffee/Tea Break

16:00 Discussion and conclusions

17:00 Closure of Day 1

v

Day 2, 9:00am – 14:00pm

Designing 'the process' for the Tisza River Basin – Session 1 in handout

Sukaina Bharwani, SEI Oxford

with support by Dagmar Haase, UFZ and Svetlana Kuptsova, Water Board Ushgorod

9:00 – 10:00

- Planning KnETs application and designing the 'research process' - Show examples of what has been done (Svetlana) - Split into the 2 groups. What is the research question? Brainstorm around this question using

cards. - Choosing a case municipality or community in both parts of the basin (Hungary, Ukraine), - Which stakeholder groups will be sampled? - Will they be split further into groups of varying attributes (e.g. based on gender, age, ethnicity,

wealth etc)? - Are we clear about the questions that we are trying to explore with the game for each

stakeholder group? - What questions/rules will be useful to compare within groups and/or between them?

Break

10:00 – 12:00

• Learning the method: conceptual design

Eliciting knowledge for representation in the game (identifying salient domains and drivers from Tisza case study based on your identified research question and using discussions with experts (from the group) working in pairs to interview each other). E.g.: What are the frameworks for a farmer to decide to a) turn to alternative water sources or b) to alternative cultivation or c) foresting?

• Spend some time filling out the table

Session 2 - Practical design of the game (some work on computers)

Session 3 - Initial testing of the game with your expert

12:00-14:00

• Session 4 - Playing the game with your expert

• Creating rules and heuristics

Lunch Break

15:00 – 16:00

vi

Session 5 – Verification - Going back to pen and paper!

Refining decision trees using those produced by the game and your own knowledge of the domain.

Break

16:30 – 18:00

• Session 6 – Validation - Testing rules and accessing tacit knowledge

• What worked and what didn't?

Running through games designed by each group and discussing pros and cons of individual approaches.

Open time for discussion of issues, concerns, etc.

Review and next steps.

Evaluation and wrap up.

18:00 Closure

NeWater

Participants Training for trainers workshop, Ushgorod, 25-26 March 2008

Name Institution Email

Bharwani, Sukaina SEI Oxford [email protected]@sei.se

Poolman, Martine TU Delft [email protected] Haase, Dagmar UFZ [email protected] Rebryk, Svetlana Transcarpathian Water

Management (TWM) [email protected]

Kuptsova, Svetlana TWM [email protected] Skral, Marina TWM [email protected] Durkot, Victor TWM [email protected] Nabyvanets, Yuri Ukraine Research

Hydrometeorology Institute (HydroMet)

[email protected]

Sheregiy, Michael Transcarpathian HydroMet [email protected] Monivchuk, Vasyl Transcarpathian HydroMet [email protected]

Iarochevitch, Alexej Ukrainian Water Consulting [email protected] Monych, Ivan TWM Osiysky, Eduard TWM

mailto:[email protected]












Evaluation forms of TtT on Stakeholder Issue Analysis

Training Uzgorod, Tisza Basin GENERAL RESPONSE

Ukraine, March 25, 2008

Part 1: Course Content

Subject Relevance

Is this knowledge applicable

for you?

If yes, for what

will you use this?

Introduction and outline of the course 4,9 6: 5 "yes", 1 "no

Demonstration of Stakeholder Issue Analysis 4,8 5: 5 "yes"

Using Stakeholder Issue Analysis 4,8 5: 5 "yes"

Introduction Adaptive Water Management 4,7 5: 3 "yes", 2 "no"

Linking Adaptive Water Management to theme/tools 4,6 5: 4 "yes", 1 "no"

Further explanation on Stakeholder Issue Analysis 4,6 5: 5 "yes"

Steps & requirements for use of models in Tisza 2: 1 "yes", 1 "no"

Part 2: Course Presentation

Subject

Personal

interest Presentation

Introduction and outline of the course 4,7 4,7

Demonstration of Stakeholder Issue Analysis 4,9 5,0

Using Stakeholder Issue Analysis 4,6 5,0

Introduction Adaptive Water Management 4,7 4,9

Linking Adaptive Water Management to theme/tools 4,6 4,7

Further explanation on Stakeholder Issue Analysis 4,8 5,0

Steps & requirements for use of models in Tisza

Part 3: General

1. Was the goal of the course achieved? 4,5

2. Was the balance between theory and practical application

ok? 4,3

3. Was the theme of integrated assessment – in relation to

adaptive management and Stakeholder Issue Analysis as a tool

explained sufficiently? 4,0

4. Was the workload of the course ok? 3,5

5. Were the hand-outs ok? 4,8

6. Logistical support provided to the participants 4,8

7. Were the food and lodging arrangements ok? 4,8

Part 4: Personal Reflections

How do you expect to apply this knowledge in the future?

In presentations, public seminars and trainings

Try to use it when public information concerning water management, explaining water issues

Which lesson learnt in the course contributes most to your

professional development?

0

base of work with stakeholders in water management

Process of stakeholder issue analysis

Information collection

Stakeholder analysis and adaptive water Resources management

1 = not satisfactory, 5 = very satisfactory



in activities planning process



1 = too general, 5 = too detailed

1 = not enough, 5 = too much

thank you!

0

How do you handle uncertainty in water management in your

work?

I think it can, but need time to answer what way! May be analisizing different stakeholders

involvement in water management

Yes, directly in descision making process, in implementation of IWRM approachesDo you think the Stakeholder Issue Analysis can help you

achieving adaptive water management? In what way?

very good!

try to find out more information about the problem

If there is anything you would like to say, remark or comment,

please feel free to do so

.. Very needed

course was quite interesting and usefulMy overall opinion about the course is …

very good and necessary

more games!

Thank you very much!

try to find out more information about the problem

Try to study them, to improve traditional methods(monitoring, forecasting) and not traditional

(public and self governments) collaboration facilitation.

Stakeholder-Issue Analysis Training, 24 January, 2008Hoogheemraadschap de Stichtse Rijnlanden, Houten, NL

Enhanced Stakeholder-Issue Analysis Training

24 January 2008Hoogheemraadschap de Stichtse Rijnlanden

Houten, the Netherlands

Martine Poolman [email protected] student, Section of Water Resources Management Faculty of Civil Engineering and GeosciencesDelft University of Technology, the Netherlands


Morning Programme

9:45 - Introduction - Expectations of training

10:30 - AWM introduction- IWRM and AWRM compared- How Stakeholder-Issue Analysis fits in


11:30 - Exercise: AWRM and IWRM- Questions & Discussion

12:15 Lunch


Participants will leave today with more experience, practice and confidence in how stakeholder-issue analyses:

• Can be carried out• Fit in with the concept of Adaptive Management• Are of importance when setting up management plans• Have advantages and disadvantages (which you can deal with)


Introduction Round


NeWater Project

• New Approaches to Adaptive Water Management under Uncertainty

• EU research project with 43 project partners

• Goal: Develop new robust strategies for adaptive water management that are implementable in the real world

• Case studies in Rhine, Elbe, Guadiana, Tisza, Amudarya, Nile and Orange


This training

• Part of NeWater Project• In cooperation with the NeWater Case Study Basins

• Upon request theme of choice

• To spend a day discussing and practicing

• Help answer some questions about theme

• To provide you with (access to) sufficient information to disseminate today’s training to colleagues


Martine Poolman

• PhD researcher @ Water Resources Management, Civil Engineering

• MSc. in Policy Analysis from faculty of Technology, Policy and Management, TUDelft

• Field work:– carrying out stakeholder and institutional analyses

– Main focus now on Ghana, to understand more about maintenance issues around small reservoirs used for irrigation, livestock drinking and fishing.


Expectations of Training

• ?


Questions already posed• When do you involve stakeholders?

• Too little, too much communication?Under/Overdosis of information vs. Waking sleeping dogs

• How to “win” stakeholders over?– those who have real stake– those who don’t have stake but can prevent work

• How to deal with difficult stakeholders?

• Which information do you share?

• How to balance interests of stakeholdersEnvironmental groups vs. Agrarians

agrarians vs. agrarians


Introduction to Integrated and Adaptive Water Resources

Management


Challenges for water management

• Uncertainty due to Global ChangeClimate change, population growth, economic growth, etc. (in)directly influence demand and supply of water

• Uncertainty in management Uncertainty of monitoring, data collection, interpretations, effects of new pollutants

• Uncertainty due to ComplexityLinks with other sectors, with up- and downstream water-users, with institutions, at various scales and time frames


Traditional water management

• Relied on predict and control• Sectoral approach

• Focused more on controlling local hydrological problems through, for example,– Dikes to protect towns against floods– Reservoirs for irrigation water– etc.


Dublin Principles (1992)

• Freshwater is a finite and vulnerable resource,

• Water development and management should be based on a participatory approach at all levels;

• Women play a central part in the provision, management, and safeguarding of water;

• Water should be recognised as an economic good.


Integrated Water Resources Management


IWRM is not an end in itself, but a means of achieving:

• Efficiency to make (water) resources go as far as possible;

• Equity, in the allocation of water across different social and economic groups;

• Environmental sustainability, to protect the water resources base and associated eco-systems.

GWP (2000)


However,

• A lot of discussion about implementation (how to)• Site specifications make prescriptions difficult• Have to deal with various laws and regulations

So, there are a lot of uncertainties and variations to deal with


Adaptive WRM

• Acknowledges explicitly uncertainties and complexity of the systems to be managed,

• Is a systematic process to improve management approaches by learning from the consequences of implemented strategies,

• Active involvement of stakeholders in the process of developing, implementing and monitoring of river basin management plans is key.


“Adaptive management is learning to manage by managing to learn” (Bormann et al, 1993)

Managing to learn from:– The past

– Present similar actions

– Other people doing similar things

– Stakeholders who will be affected by (lack of) changes


Traditional AWRM

Command and Control

Prediction, quantification of risk

Centralised governance

Negotiate acceptable solutions

Scenario planning

Polycentric governance, shared responsibilities, participation

Cross-sector policy integration

Decentralised Infrastructure

Multi-functional landscapes

Mix: technical, social, ecological problem solving approach

Sectoral management

Central infrastructure

Single function landscapes

Pure technical approach to problem solving

Whose scenario?With whom?


Processes in adaptive water management• Involvement of stakeholders

– Exchange information – Integrate knowledge & requirements set by others – Build commitment– Receive and give feedbackStrive for: Balanced representation and access to information

• Decision making processes– open to information from diverse sources– allow for changes in rules and structures– exchange information and work across spatial scales and sectors


Decision-making process


Stakeholder-Issue Analysis fits in …

• Identifying key stakeholders• Assess their interests• Assess how interests (may) affect project• How best to accommodate stakeholders (strategy)

• Assessment of socio-economic context• Examination of dynamics of that context

~ people change, ideas change, situations change

stakeholderissue


Formulate the topicIdentify who the stakeholders are

Make an inventory of the stakeholders’ perception, goals and their interests (ISSUES) towards the topic

Prioritize stakeholders according to interests in proposed project work

Determine the dynamics of the network of stakeholders Identify potentials for collaboration

Draw up potential strategies for obtaining support or reducing obstacles

~ Stakeholder(Issue) Analysis Steps


Formulate the topic

Identify who the stakeholders are

Make an inventory of the stakeholders’ perception, goals and their interests

towards the topic

Prioritize stakeholders according to interests in proposed project work

Determine the dynamics of the network of stakeholders

Identify potentials for collaboration

Draw up potential strategies for obtaining support or reducing obstacles

Evaluate outcomes

Decision-Making Process Steps Stakeholder-Issue Analysis Steps


Coffee and Tea break


Exercise part 1

• Drawing train station• Listing elements of own drawing• Listing elements of some one else’s drawing• Comparing element lists• Plenary discussion


Lunch


Afternoon Programme

13:15 - Exercise part 2 Stakeholder Analysis- Introduction to Stakeholder-Issue Analysis


14:45 - Examination of SI-analysis steps (interactive)- Exploration of tools/instruments available to use



Exercise Stakeholder-Issue Analysis (part 2)Step a

• In groups list the people you will find near or in your station

Step b

Station is considered unsafe and will need to be altered

- Determine why the station is unsafe

- Determine what will need to be altered


Formulating the topic

• How did you determine• What would you have done in the “real world”?

• Context in which issues are to be resolved• Clarify the objectives• Identify constraints

• Interviews• Visit to site (community mapping, transect walks, etc)• Background study (literature or talks about similar proj’s)


Identify Stakeholders

- Build up on initial list (step a)

How to:- Brainstorm within team

- Literature review/ past experience

- Semi-structured interviews with “experts”

What to ask?


Identify Stakeholders (2)

What to ask?- Can they contribute to decision-making?

- Are they needed for implementation?

- Can they block decision-making and implementation?

- Are they affected by or have interest in the issues at stake?

~ from HarmoniCOP Handbook, 2005


Identify Stakeholders (3)Also realize:

- Who are the “voiceless”?

- Who are the representatives?

- Who is responsible for what is intended?

- Who is likely to mobilize for or against project?

- Who can make project more effective through their participation?

- Who can make it less effective by their non-participation or opposition?

- Who can contribute financial and technical resources?

~ World Back Sourcebook, 1996


Stakeholder Analysis tablesStakeholder Involvement in

issueInterest in issue

Influence/power

Position Impact of issue on stakeholder

1.

2.

Stakeholder Interests Potential project impact (+ or -)

Relative priorities of interests (scale 1 to 5)

1.

2.

3.

Group/stakeholder Group’s/ stakeholder’s Interests in Issue

Resources Resource Mobilization Capacity

Position on Issue

1.

2.


Inventory of perceptions, goals, interests, resources (1)

• Stakeholders’ position on topic• Level of influence they hold• Level of interest they have• Group to which they belong or can be associated with



A. Quick and Dirty- Would want to reach what?- Why would want to reach these goals?- Which (dis)advantages will arise for stakeholder during

project?- Which resources do they have that are needed?



B. Goal Action Diagram

Goal: obtaining a new house

Search real-estate websitesPlace advertisements

Walk through desired neighbourhood

Read advertisements Inform/ask contacts (friends, family, colleagues,etc)

- Real-estate agent- Family, etc.


Prioritizing Stakeholders

Stakeholder classification grid:– Power vs interest– Influence vs influence


Assess Dependencies


Assess Network of Stakeholders


Taking dynamics into account

• Situations change continuously– Ideas change

– Stakeholders change

– Relations between stakeholders change

– Policy changes

• In grids: mark possibilities for change– Foresee-able changes

– Possible changes

In order to identify potentials for collaboration


Existing Tools (others)

• Participatory Approaches

• SWOTs

• Network-Influence mapping

• Role Play Games


Participatory Approaches

• For introductions

• Creating a good atmosphere

• Seeing is also learning

• Who does what in community

• When/where do things take place

• How satisfied are people

• Past experiences

• Future dreams

• Capacities of stakeholders


Example of seeing is learning


Weaknesses- What could be improved?- What is done badly?- What should be avoided?

SWOTsStrengths

- What advantages does the approach have?- What is done well?- What relevant resources do people in the approach have access to?- What do others see as strengths?

Opportunities- Where are the good opportunities that those setting up the approach are faced with?- What are the interesting trends the members are aware of?

Threats- What obstacles are being faced?- Are relevant situations changing?- Are there cultural conflicts?


Influence Network Mapping


Black = command Red = money Green = support/advice Blue = information Yellow = future links


Cut-point

Feedback and network learning

Black = command Red = money Green = support/advice Blue = information Yellow = future links


Role Play Games• Placing SHs in someone else’s “shoes”

• Experience a real-life situation without having to take the risks in real-life

• Learn more about others’ situation

• To practice and examine:– How to solve problems

– Willingness to take risks, make trade-offs, change behaviour

See also: http://serc.carleton.edu/introgeo/roleplaying/reasons.html


Role Play Games

Example of our students: African Village Water Supply• Roles:

– Missionaries

– 5 villagers (chief, woman, man-with-a-plan, owner of land, man)

• Water Supply– Sand dams

– 3 locations

• Issues?


Potentials for collaboration

• We have:– the long-list

– the categorisation

– the prioritisation … of stakeholders.

– An idea of possible changes to take into account

– An idea of dynamics we may need to take into account

• So we can draw up strategies


Potential Strategies

• Based on:– Perceptions, goals, interests

– Power and influence on goals and each other

– Criticality and dedication

– (dynamic) position within the network of stakeholders

• Participation Spectrum


Participation Spectrum


But how do we evaluate whether what we’ve done (so far) is sufficient, satisfying (for all parties?) and useful?


Outcome Mapping

• A way to measure successful development of a programme/project

• Focus on behaviour of ALL stakeholders– Includes the project owner

– Based on expectations/ideas of project owners

– Requires feedback from stakeholders

• Helps to be specific about the targeted stakeholders, the changes you expect or would like to see and the strategies you employ


Outcome Mapping

• Results = focused on change in processes and outcomes, therefore:

• Results = changes in behaviour, actions or relationships that can be influence by project

• Enhances understanding of change processes

• Focus on M & E activities to obtain feedback from stakeholders


End of Day

• Evaluations

Knowledge elicitation tools (KnETs) workshop Facilitator: Sukaina Bharwani, Stockholm Environment Institute, Oxford, UK. KnETs software developer: Michael Fischer, University of Kent, UK. Introduction The need to understand the multiple stresses which interact to form complex vulnerabilities has led to the design of tools for knowledge elicitation (KnETs). This interdisciplinary approach integrates methods used in ethnographic fieldwork with classical knowledge engineering techniques from computer science in order to alleviate weaknesses inherent in both methods. This provides a participatory and robust process, from knowledge elicitation to knowledge representation, providing a greater clarity of the ethnographic data and thus possibly a greater understanding of social vulnerability and adaptive behaviour. The fusion of these techniques has resulted in a 4-stage process (Figure 1) which incorporates consistent verification and validation on the data as it is collected. The application of this innovative methodology is successful precisely due to the mutual benefits that each discipline provides by addressing current bottlenecks in both areas of research. This process has been significantly influenced by the work of Christina Gladwin (Gladwin 1989) and Fergus Sinclair (Sinclair 1993; Dixon 2005) and the importance they attach to emic and tacit knowledge.

Figure 1 Stages within the knowledge elicitation process.

Session 1 – Stages 1 and 2 in Figure 1 * Aim: Eliciting knowledge for representation in a questionnaire game – conceptual design of the game Begin to identify salient domains and drivers from the case study situation based on discussions with experts from the group, using cards and pens provided. In reality, this may evolve in a series of stages during fieldwork as shown in Table 1.

Sukaina Bharwani Page 1 KnETs workshop

Table 1 Example of stages of interaction with stakeholders before using knowledge elicitation game (Stage 1 in Figure 1).

Aim Possible tools and questions 1. Identify baseline current vulnerability

a. Resource map b. Brainstorm using cards (for later use in the problem tree1) (20 mins)

What are the social and environmental stresses (to the livelihood of the stakeholder group, for example, e.g. displacement, drought, flooding)?

2. Identify intermediate linkages

a. Problem prioritization (10 mins) from brainstorm (e.g. using stones, leaves, sweets etc). What is the biggest problem facing you? (from all those identified in 1b)

b. Problem tree (focusing on one problem from the brainstorm looking at causes and solutions of that single issue). What are the causes and what do you see as possible solutions?

(Kate Gant pers comm.)

3. Identify adaptation strategies

a. Discussion based on problem tree – causes, stresses and solutions (15-20 mins). How do you cope with these problems? (identified in 2a and 2b). NB: We are interested in coping, adaptive capacity and resilience. This is similar to 2b, but here we are critically interested in why people do what they do, why they adapt in one way rather than another. This context-specific, often ‘socio-cultural’ information is essential to be able to understand decision-making processes accurately, and from the perspective of the expert.

b. Institutional Chapatti diagram (20 mins) – influence vs. impact, to establish power relationships and weak and/or under-represented groups.

c. Influence vs. impact matrix (identifies which are the important stakeholders to target, including ‘hard to reach’ groups and those ‘without a voice’.

4. Move to individual interviews with KnETs The idea of the process described in Table 1 is to get more and more focused on one particular stress or issue ((Stage 1 in Figure 1)). This is core to designing a game for knowledge elicitation (Stage 2 in Figure 1). There are many ways to get this information and it is personal preference as to the technique used. Table 1 is simply one example of what is possible. 1 Use as much input that is meaningful to informants as possible, e.g. symbols, pictures etc.


Tip: Identifying issues that are important to stakeholders may involve focused exercises such as a simple brainstorm or a Problem Tree. However, most importantly it should include a period of exploratory fieldwork and a series of interviews over some time to gain a good understanding of the issues involved. This should happen before attempting to design a game for knowledge elicitation which is a more focused and structured stage.

To help your discussion to identify and explore key issues there are several things we need to know which will help us to design to game.

1. What is the fundamental question we are interested in answering? E.g. this may be related to the problem that is given highest priority in the problem tree exercise (2b in Table 1).

2. Given our question, which are the important stakeholder groups to interview further? (3c Table 1)

3. Again given stakeholder groups which are identified as being important, re-think what is the fundamental question we are interested in exploring, in relation to this particular stakeholder group?

Tip: The fundamental question should be specific and focused rather than vague and abstract! 4. Related to this, how should we stratify our sample from the population when

playing the game e.g. according to wealth, gender or ethnicity for example. When you have decided this, it is useful to split the group further into a ‘training’ set and a ‘testing’ set. The ‘training’ group is the one that we will refer to most here since they will play the game and help to refine the rules. The ‘testing’ group is introduced at the end of the process to test how well the rules do predict the behaviour of the group generally.

Continue thinking about your own case-study, but to help you understand what a game might look like some examples might help. There are some hypothetical examples provided in the accompanying files from a floating fishing village in a floodplain in Cambodia and from an agricultural community of small-scale farmers in South Africa. The different versions show some of the different layouts of game you can have (see relevant folders). Go to: \KnETs Training Documents (public)\Cambodia Double-click on the jar file in each folder and when the window showing the game appears, click on the ‘Interview’ tab. Think of yourself as the target group and choose the option you would undertake given the conditions shown. Also choose any strategies that would be necessary to support your choice under the given conditions. Your choices will be recorded in the log area. When you click on the ‘Next Run’ button a new set of conditions will be presented to you. A minimum of 10 runs is required to produce any rules and these will not be very useful or surprising! Create rules (Stage 3 in Figure 1) by clicking on 'Eval' from the Menu and J48. Usually more interesting rules appear after 30-50 runs, but the crucial point is that this all depends on how well the game is designed, which in turn depends on how good your


understanding of the domain is, so far. This is why it is important to spend time on Stage 1 in Figure 1, and Session 1 in this training – thinking about the research design and the research question (though this may change with time) is very important. Example 1: Cambodian Floating Fishing Community

Figure 2 Example of using KnETs with floating fishermen.

Once you have experimented a little with this example you can try loading some data (File – Load Questionnaire) that has already been done for you (Goals.wbq) and continue playing the game to see how the rules change.


However, this example illustrates a common error in decision tree modeling which is thrules are created, which may predict behaviour but not for the ri

at ght reasons. We require more

formation on the reasons why choices are made and these are more likely to be socio-

knowledge elicitation game? If the game doesn’t work very

ell a common reason is because more exploratory fieldwork is needed before this more

incultural in nature and very specific to the context of our study. Tip: Does the game work well? How much time have you spent trying to understand thedomain prior to designing the wfocused and structured stage. Figure 2 shows strategies in response to stress, but not why those particular strategies are chosen. This is why it is so important that a good amount of fieldwork is undertaken beforthis approach is used. Otherwise the process becomes more cyclical than it needs to be athe game constantly needs to be updated with the new information that arises, when over time, informants may explain not just what they do, but why they do what they do. T

e nd

he llowing example is an improved version of the original game with some of the reasons why

be loaded from:

\KnETs Training Documents (public)\Cambodia\improved version

fothe floating fishermen were choosing particular strategies. This can


Figure 3 Cambodian Floating Fishing Community – improved version

ve

e er

d by real people (Gladwin 1989). A test of a decision

ion (Goals.wbq and Goals2.wbq) from the ore responses and running J48 again

Example 2: South African small-scale farmers

Emic vs. Etic Decision Criteria The improved model illustrates that your understanding of the emic categories which dridecision-making is paramount. These could also be described as the socio-cultural or context-specific influences. This refers to units of meaning drawn from the society and culture of interest. This is in contrast to etic categories, which may have meaning for outsidobservers/analysts of a culture but need not have meaning for the people of the culture undstudy (Harris 1979). It is the emic categories that make a decision model both logical anpredictive of choices made in real lifemodel is whether it predicts most informants’ decisions based on what they claim their decision criteria are (Gladwin 1989). That is, we must be aware of the common error in creating decision rules which look very simple and appear to predict the decisions of informants to a very high degree of accuracy. While very simple rules may be able to correctly predict choices made by informants, if they are based on very general criteria and not what informants claim their decision criteria to be, then the actual logical processes that are taking place are not being captured (Gladwin 1989). Another example from a community garden project in Limpopo, South Africa will illustrate this point further (Figure 4). Go to: \KnETs Training Documents (public)\South Africa Emic categories relating to taste, nutritional requirements, and tradition are included here. You can load the data captured from an earlier sessfolder and run J48 to create the rules. By providing myou can see how the rules change with more data.


Figure 4 A game for South African small-scale farmers.

Figure 5 shows a decision tree that could result from such a game. It is important to draw out the decision trees from the rules that are suggested. You will end up with several trees and many rules within each tree. It is useful to practice drawing out possible decision trees according to the output of the game and your own knowledge of the domain.


Priority – crops in daily home

use

Grow Soya

Grow Rapeseed (i.e. high market

demand)Grow

Sunflower

Low/medium market demand

Y

N

N

Forecast –normal rainfall

Y

Forecast –above normal

N

rainfall

Forecast –below normal

rainfall

Rain comes in

September

Y

N

Grow Corn

N

N

Grow Rapeseed

eholder ry and fill out the table attached

st page of handout) according to the given categories. This will help to frame how these r game.

(i.e. rain comes in January)

Figure 5 An example of a decision tree which includes some emic (cultural) criteria

Designing your own game Once you have understood some of the significant issues that are important to the stakgroup in your case-study, and written them on the cards, t(lacan be represented in you


Session 2 * Practical design of the game - Introduction of xml file (including TextPad).

you do need to update the jar file (to include new images, for example), use the following

Simply altering the modelParameters.xml file and saving it, will make most of the necessary changes you will require in the game. NB: Most people can ignore this bit! Ifcommand: jar -uf jar_file_name.jar

Now you can practice putting the issues you identified in the previous session into the xml

le to alter the ‘game’ and to make it specific to your situation. There are the beginnings of

me xtent subjective, but it is important to get as close as possible to how the informant

fian example for the Tisza River Basin in the relevant folders. You can change those aspects which you do not feel are relevant. Tip: How the game looks is not as important as how you categorise drivers, strategies, goals and reasons in the table from the previous session. Of course, these are to soeviews his/her constraints and options in decision-making. Their decision-making framework is important as are emic/cultural criteria which will make it unique. You can also play the same game using the cards from the previous session and it wdepend on the stakeholder group you are talking to as to whether you use the computer to play the game or have an additional person to represent the ‘game’ in card form

ill

. The omputer aided game is still required to provide scenarios and to record the responses of the

rs. In order to try this, please write any outstanding drivers, strategies, goals and

he cards. Ideally they will be arried over from The Problem Tree exercise in Stage 1.

cstakeholdereasons you have listed in your table on the cards provided. Tip: When using cards for the game, symbols/pictures could also be used, and as far as possible the informants should be involved in creating tc Current vulnerability At this point it is useful to consider what the current scenario which faces informants appears

be. That is, what are their current constraints? When you have identified these, the

lready

n reality be most likely to hange and affect the informant. These should be ordered correctly in the XML file since

these variables will iterate first, thus presenting the informant, initially with something similar to the current scenario, and then something that represents a short-term future scenario and lastly, something that is a possible long-term prospect, but may not seem very probable at present. This allows informants time to gradually adjust their thinking to a possible ‘future’ situation and to consider longer term pathways for adaptation.

to“default” value in the xml files should be set to “true” for each domain where applicable. That is, when the game begins the informant is first presented with a scenario that is afamiliar to them – the ‘business as usual’ scenario. Additionally, it is also valuable to identify which domains would ic


Session 3 * Testing the variables in the game and refining them Probe your expert further, presenting some scenarios from the game. The resulting discussimay require new strategies, drivers, or even new goals to be added to the game. Always be prepared for something that you may have overlooked as being important in the decision-making and amend the xml file as many times as necessary.

on

As the rule induction algorithm works on the number of consistent patterns in the responses, the more variables in the game, the more responses are required to produce enough patterns to create rules. A good understanding of the domain will allow strategies to be merged and grouped together. This is beneficial since fewer variables considerably shorten the length of the game and thus it is less likely to be a great burden or imposition to the expert. Bonus Tip: It is useful to have an ‘Other’ box for goals, strategies and reasons because often new things come up as the game proceeds. Even when you have come to the point where you think that you will make no more changes to the xml file and can begin the game there may still be something unexpected that emerges as an answer from the informant. In this case you can use the ‘Other’ option and just make a note of the meaning you want to assign to it when analysing the rules later! Session 4 * And finally, playing the game with your expert! (Assign roles to individuals in the group where one person is responsible for running the game on the computer, one is responsible for representing the game in card form and one is the expert. Please rotate these roles so that you have an understanding of the difficulties of each one!) It is problematic to show decision trees as they are created or as discussion is ongoing as this may introduce bias from the informant in his/her answers. But this can be done at the next stage. Again, a minimum of 10 runs is required to produce any rules and these may not be very useful. Usually more interesting rules appear after 30-50 runs. With more runs, more interesting rules should emerge which include socio-cultural drivers, preferences or priorities. Start recording these rules on paper in the form of decision trees.

ain should you begin laying the game. Accept that you will need to restart the game several time since there

will always be things you have missed or were unaware of when you started!

Tip: Only when you feel you have good knowledge of the domp


Session 5 - Verification * Going back to pen and paper! (refer to South African and Cambodian example) One symptom of decision trees being too simple to represent emic criteria (that which is specific to the informant) is that rules can often look like ‘vines’ rather than trees. That is,

ey are quite linear in nature and can just as easily be represented in check-list form because

stage o 1989) and although the decision predicted may

as it ha

ththey are simple straight-line ‘elimination by aspects’ models. This is usually only the first

f a real-life decision process (Gladwinbe correct, much contextual information will be missing. Figure 6 is an example of a ‘vine’

s few branch points and includes very general criteria.

Y N

N

Y N

YY

YY

N

Figure 6 An example of a vine


What can be done to turn a vine into a tree?

ve

1. Generalize some of the criteria, i.e. put similar criteria together to eliminate

mic be obscured (Gladwin 1989).

ter)

If you recognize a vine rather than a tree resulting from your game there are several things you can do. This requires some thinking and re-shuffling of the decision trees/rules you haso far on paper, or even using your cards if that is helpful.

redundancy (Gladwin 1989). 2. Cluster the decision criteria logically in an order that is consistent with the

decision-making of the stakeholder e.g. first criteria that ‘enable’ the decision, and then the ordering aspect that is ‘maximized’ in the decision. This means that all possible constraints are passed before the decision is taken. For example, in a tree that quickly proceeds to ‘fish’ or ‘don’t fish’, these ecriteria or constraints may

3. Identify and eliminate decision criteria that belong in a logically prior (or ladecision, and put them in another tree (Gladwin 1989). Vines often occur instead of trees because the modeler does not realize that s/he has a series of trees to model not just one (Gladwin 1989).

Y

Y

Y

Y

Y

NN

Figure 7 An example of turning a vine into a tree


Figure 7 shows that the vine has begun to look more like a tree with the grouping of some criteria ordered for maximization (e.g. market-related) and the logical ordering of others, such as ‘enabling’ criteria (e.g. health, availability of equipment), as suggested by Gladwin (1989). This is a verification stage that is reliant on your own understanding of the domain.

Figure 5 is also a good example of a tree with many branches, as a result of the grouping certain criteria and re-ordering others as the impact of different drivers/conditions becomes apparent. Clearly, as your understanding of a domain improves, such trees can be continually refined, creating more branches, and perhaps more trees, rather than linear, checklist type criteria, which is usually only the very first stage of actual decision-making (the ‘pre-attentive’ phase (see (Murtaugh 1980)). It is useful to note the particular features of the rule induction algorithm being used in this version of KnETs. As you will see from the menu option when you run the program there will be other algorithms available in the future to potentially be able to compare the rules that are created. Session 6 * Validation of rules with experts and accessing tacit knowledge – Testing our understanding of the domain expert’s decision-making processes. We must keep in mind that our rules do not necessarily represent the decision-making process, but simply our understanding (as non-experts!) of it. Therefore, we should be open to the probability that there will be gaps or inconsistencies in our knowledge as well as inaccuracies. This stage of the knowledge elicitation process is the opportunity to rectify these issues as far as possible.

rds from previous exercises. But using the ing’

e you not only present the scenario, but also e decision suggested from the decision trees, asking the stakeholder if the conclusion is

correct or not. Again, as you may not get this far with your own games today, it may be beneficial to look at the following hypothetical examples from our floating fishermen. Let us assume, Figure 8 relates to the category of ‘very poor fishermen’ who have little excess capital and high debts.

Again this stage can be carried out using cacomputer is beneficial in recording new rules and refinements. This is done using a ‘learnprogram which is currently being developed as a plug-in to the current game interface. If you have rules from your own game, it is at this stage that you would return to your takeholders and try to validate them. This tims

th


Y

YY

Y

N

Y

Y

N

Figure 8 Decision tree presented to stakeholder

The fishermen are asked whether they believe the decisions to continue fishing or begin as farming, given the conditions shown are correct. They indicate that even if fish catch w

declining and access to fishing grounds was restricted, they would not start farming, but they would continue fishing as shown in Figure 9.


Figure 9 Stakeholders response to incorrect decision tree.

It may be the case that this decision tree is now correct and another tree can be tested. However, it is at this point that we may be able to probe further and access tacit knowledge – that which is often common-sense or very obvious to the expert informant (but not to the on-expert!) and therefore, not mentioned by him/her!

istinguishing feature between hy

n The way in which to access this information is to identify the dthe incorrect decision and correct decision i.e. between farming and fishing (e.g. asking ‘wwould you continue fishing rather than farming?’). When a reason is identified one needs toformulate a statement/condition which when satisfied (i.e. when answered ‘yes’) produces the correct decision as shown in Figure 10.


Y

Y

Y

YY

N

Y

Figure 10 ‘Learning’ and tree refinement with stakeholder feedback.

men said that they would continue fishing despite the declining fish catch and and ‘it is what they have always done’. Their

aditional livelihood is very important to them and is a part of their identity, but they did not ecessarily make this explicit during interviews when talking about adaptation to different resses. It was assumed that we should already know this or that it was quite obvious.

this case, with the extra capital available they would simply buy better fishing gear to try reduce their vulnerability to these negative conditions, but continue with traditional

ctivities because they value them.

Here, fisherrestricted access since ‘that is what they do’ trnst Intoa


Another example of accessing ‘tacit’ knowledge using stakeholder feedback

igure 11 relates to the hypothetical category of ‘slightly better-off’ fishermen who have ore access to capital to begin with and already have good fishing equipment.

Fm

ital ey would not migrate to the town to find work, but that they would continue fishing as

Figure 11 Decision tree presented to stakeholder

The fishermen are asked whether they believe the decision to migrate to the town to find work under the given the conditions is correct. They indicate that even if had extra capthshown in

Figure 12.


Figure 12 Stakeholders response to incorrect decision tree

wer as shown in

One may assume that the reason for this is their access to good fishing gear, given what we know about the value placed on maintaining the traditional livelihood activity. However, repeating the same process as before may reveal other, tacit criteria for this decision. Again they are asked whether there is distinguishing condition between fishing and migrating. And in actual fact, the fishermen asserted that the difference between migrating for work and continuing their traditional livelihood was really due to a lack of skills. In identifying this one eeds to formulate a condition which when satisfied produces the new ansn

Figure 13.


Y

YN

NY

Figure 13 ‘Learning’ and tree refinement with stakeholder feedback in an iterative process.

Therefore, even though these fishermen have access to more capital and equipment, there is still a huge shortage of skills in the village which constrains the adaptation options of the villagers further. This was not something which was immediately obvious as a constraining factor and revealed that simply creating more income-generating activities or jobs in local towns would also require much training and education due to the very low literacy/skill rates in this village. The very remote location of this particular community on the lake, and distance from the mainland, was also a unique constraint which increased their vulnerability and limited their adaptation options further. Once the rules have been validated to your satisfaction with your stakeholder group (the ‘training’ group), you should find other stakeholders of a similar profile who have not been involved in the process (e.g. other ‘very poor’ fishermen in the same village) and validate the rules with them (the ‘testing’ group) to see how well the rules predict the decision-making behaviour of the group – ‘very poor fishermen’ – in this village generally.


References Dixon, H. J., Doores, J.W., Jos 5). Agroforesthi, L., Sinclair, F.L. (200 ry Knowledge Toolkit

for Windows: Methodo relogical Guidelines, Computer Softwa and Manual for AKT5., School of Agr

Gladwin, C. H. (1989). ig ee Modellingcultural and Forest Sciences, University of Wraphic Decision Tr

ales, Bangor. Ethno . Beverly Hills, CA, Sage

Materialism: The Struggle for a Science of CulturePublications.

Harris, M. (1979). Cultural . New York,

(1 ess model for forecasting i imp cations." Special

Random House. Murtaugh, M., Gladwin, H.

automobile type - cho980). “A hierarchical decision-proc

ce automobile choice and its energy- liIssue of Transportation Research.

Joshi, L., Ambrose, B., and Thapa, B.Sinclair, F. L., Walker, D. H., (1993). Use of a Knowledge-Based System Approach in the Improvement of Tree Fodder Resources on Farmland in the Eastern Hills of Nepal., School of Agricultural and Forest

s, University ofscience

Wales, Bangor.



ELEMENTS KEY QUESTIONS DRIVERS IDENTIFIED

Groups – a range of groups may need to be identified from the population. They should have different properties (e.g. gender, education, occupation, location, religion, ethnic group) and when a group is selected due to its importance to the issue, several key informants within the group need to be selected for further interviews.

What are the different groups that have been identified? How are they different? How do their differences affect their decision-making?

NB- One table may be required for each group that is considered important to the research question.

Goals - a range of preferable outcomes, including most preferable and least preferable (e.g. maintain income, farming, fishing, etc.)

What are the goals of the different groups that have been identified? What are they trying to achieve on a day to day and long-term basis?

Drivers - variables which support/constrain decision-making (e.g. rainfall, market demand, soil quality, availability of capital, flooding, access to irrigation, declining fish harvest, drought, etc)

What are the factors that support/constrain each group in achieving these goals?

Strategies – short-term actions that must be taken to achieve the goals (e.g. using fertilizer, using irrigation, selling to traders)

What immediate actions are necessary taken to achieve the desired goals?

Adaptive options – long-term actions that may be taken to make an outcome more acceptable (e.g. switching to drought resistant crops, buying better fishing gear, clearing flooded forest)

When it is difficult to achieve these goals on a short-term basis, what long-term actions are taken?

Reasons why – these may be more specific to the group and therefore cultural, often relating to the social context in which the decision is being made.

Why do you make the decisions you make? e.g. it is tradition, we have always done it this way, we don’t know any other way, no other skills, no institutional support, etc

D 3.5.5 STAKEHOLDER ANALYSIS AND KNOWLEDGE ELICITATION ... · Title STAKEHOLDER ANALYSIS AND...

Documents

Transcript of D 3.5.5 STAKEHOLDER ANALYSIS AND KNOWLEDGE ELICITATION ... · Title STAKEHOLDER ANALYSIS AND...