Formulating Indicators for Assessing Scenarios

MRC-MDBC STRATEGIC LIAISON PROGRAM

BASIN DEVELOPMENT PLANNING

TRAINING MODULE 3

SCENARIO-BASED PLANNING

for the MEKONG BASIN

Napakuang, Lao PDR

8-11 December 2003

Formulating Indicators Formulating Indicators

for Assessing Scenariosfor Assessing Scenarios

DEVELOPING INDICATORS

A process for this part of basin planningA process for this part of basin planningAssemble inventory of water-

related information through sub-area analysis

Estimate waterdemands of

potentialdevelopment

Identify devel’tobjectives andvalued assets

to be sustained

Estimate socio-economic benefits

Tabulate scenariosand outcomes

Experts/ plannersrefine assessment

criteria and/orindicators

Planners + expertsevaluate trade-offs

between benefits andimpacts of each scenario

MODEL RUN Planners/ modellerscooperate to get

best help from DSF

Assemble adevelopment

scenario(a set of future water

demands anddevelopments)

Get agreement onassessment criteriaand indicators for

objectives and assetimpacts


Summary of steps needed to prepare for Summary of steps needed to prepare for assessment of scenariosassessment of scenarios

Derive BDP Development

Objectives

(derived from 95 Agreement)

Select Assessment

Criteria(Tests of howwell scenarios

will meetobjectives)

SelectIndicators

(Measurementsto be used inassessment

tests)


Step 1 - Defining development objectivesStep 1 - Defining development objectives

• First and foremost: Objectives must be derived from the Basin Agreement. This is the mandate of the Mekong River Commission.

• But the objectives in the Agreement are broad and not easy to quantify.

• An agreed set of more precise and/or detailed BDP development objectives needs to be developed through consultative processes.

• These derived development objectives should cover all aspects of each of the objectives (whether explicit or implicit) of the Agreement.


Example of BDP Development ObjectiveExample of BDP Development Objective

Start point: Chap III of Agreement - Objectives and Principles of Cooperation. For example:

Article 3. Protection of the Environment and Ecological Balance

Suggested BDP Development Objectives derived from Article 3 could include:

• “To achieve and maintain acceptable water quality in the Mekong River”

• “To maintain fisheries production.”

and so on...


Step 2 - Select Assessment CriteriaStep 2 - Select Assessment Criteria• Assessment criteria are the tests (for achievement

of objectives) to be applied to the anticipated outcome of development scenarios.

• At least one such criterion (or test) is needed per BDP Development Objective.

• The tests (criteria) should preferably be capable of being measured in a quantifiable manner and linked to water flow regime.

• Selection of the criteria should be done in a consultative way - especially with relevant experts (MRC programs + expert line agencies)


Example of Assessment CriteriaExample of Assessment Criteria

For the objective “To maintain fisheries production” some criteria could be:

• Changes in dry season flow regime• Changes in wet season flood flows• Changes in selected flooded areas• Changes in connectivity for fish migration

These are examples of factors that support or affect fisheries production.

Note that they have all been related back to flow. Expert advice is needed for this.


Step 3 - Selection of IndicatorsStep 3 - Selection of Indicators

• Indicators are the specific measurements to be used as part of the assessment tests.

• At least one indicator is needed per criterion.• To the maximum extent possible the indicators

should be capable of being generated by the DSF - either as direct numerical output or as spatial data for interpretation.

• Again, the indicators should be selected in a consultative way - especially with relevant experts (MRC programs + expert line agencies) - and with modellers.


Examples of IndicatorsExamples of Indicators

For the fisheries maintenance assessment criterion “Changes in wet season flood flows”, possible indicators are:

• Annual flood frequency (of selected flood sizes)

• Rate of rise of wet season flood

• Rate of fall of wet season flood

Note these are all numerical indicators and can be computed through DSF simulations.


Assessment - It is done by TEAMWORKAssessment - It is done by TEAMWORK

Derive BDP Development

Objectives

(derived from 95 Agreement)

Select Assessment

Criteria(Tests of howwell scenarios

will meetobjectives)

SelectIndicators

(Measurementsto be used inassessment

tests)

BDP Team

(MRCS + NMCs)

BDP Team

+ Experts

BDP Team

+ Experts

+ Modellers


Some ideas for shortcutsSome ideas for shortcuts

• MRC has done a lot of work already. • Capitalise on it! For example:• Transboundary issues are thoroughly identified

and listed. Good start point to derive indicators.

• WUP has already prepared a comprehensive list of suggested indicators. See DFR 640 Technical Reference Report: Impact Analysis Tools. Start with these.

• Vital to take a structured approach, with all steps documented.


Building on work already doneBuilding on work already done

1995 Agreement

Transb’d’y Issues

WUP suggested indicators Anything

else you can find

Agreed Indicators


A Structured FormatA Structured Format

Objectives ofMekong

Agreement

DevelopmentObjectives

AssessmentCriteria

Indicators

A A.1 A.1.1 A.1.1.1

A.1.1.2

A.1.1.3

A.2 A.2.1 A.2.1.1

A.2.1.2

B B.1 B.1.1 B.1.1.1

B.2 etc

Any format will do. Important thing is to have one.


Finding out if the indicators workFinding out if the indicators work• The only method is to actually apply the

indicators to some scenario outcomes generated by the DSF, or otherwise. Revise them and repeat. An adaptive, iterative process.

Ask the questions:• Do we have the minimum set of indicators?• Are any redundant?• Are any not sensitive enough to show variations?

- even when we ‘know’ there must be change.• Have we missed any vital indicators? (Go back

to experts with results and check that they are satisfied their issues are being assessed.)

Is the project or program basin-wide?

Yes/NoArt 2 – Projects, Programs andPlanning

To prefer and emphasise joint and/orbasin-wide development projectsand basin programs.

Are there expected transboundaryeffects?

Yes/No

Phosphorus concentration

Nitrogen concentration

Change of eutrophic status ofselected water bodies

Dissolved oxygen

Change of sedimentation in criticalreaches

Turbidity (NTU)

Changes in microbial pollut ion E Coli (cells/ml)

To achieve and maintain acceptablewater quality in the Mekong River

etc

Elevation of dry season f low (meandaily f low in April)

Reduction of dry season f low (meandaily f low in April)

Changes in dry season f low regime

Smoothing of dry season flow(percentile distribution)

Annual f lood frequency

Rate of rise

Changes in w et season f lood f lows

Rate of fall

Frequency of overbank f lows

Flooded area

Changes in selected f looded areas

Area by f lood duration

To maintain f isheries productivity

Changes in connectivity for f ishmigration/dispersal

Longitudinal migration length (km)

Art 3 – Protection of theEnvironment and Ecological Balance

Etc

Minimum flow sTo protect the acceptable minimummonthly f low during the dry season

Changes in minimum flow regime

Duration of minimum flow s

Mean peak w et season f low (Kratie)

Date of f low reversal (Prek Kdam)

Art 6 – Maintenance of Flows on theMainstream

To protect the natural f low reversalof the Tonle Sap

Changes in f low reversal regime

Peak reverse f loe (Prek Kdam)

MRC-MDBC STRATEGIC LIAISON PROGRAM

BASIN DEVELOPMENT PLANNING

TRAINING MODULE 3

SCENARIO-BASED PLANNING

for the MEKONG BASIN

Napakuang, Lao PDR

8-11 December 2003

Multi-criteria Analysis forMulti-criteria Analysis forRanking Development ScenariosRanking Development Scenarios

Assemble inventory of water-related information through sub-

area analysis

Estimate waterdemands of

potentialdevelopment

Identify devel’tobjectives andvalued assets

to be sustained

Estimate socio-economic benefits

Tabulate scenariosand outcomes

Experts/ plannersrefine assessment

criteria and/orindicators

Planners + expertsevaluate trade-offs

between benefits andimpacts of each scenario

MODEL RUN Planners/ modellerscooperate to get

best help from DSF

Assemble adevelopment

scenario(a set of future water

demands anddevelopments)

Get agreement onassessment criteriaand indicators for

objectives and assetimpacts

MULTI-CRITERIA ANALYSIS

Scenario-Based Planning - How do we do this bit?Scenario-Based Planning - How do we do this bit?

MULTI-CRITERIA ANALYSIS The need for MCAThe need for MCA

• The ideal situation is to be able to measure the outcomes of alternative developments in the same units such as dollars.

• Comparison is then a simple numerical exercise.• And in such a situation it doe not matter that for

any particular scenario it may meet some criteria extremely well, and others extremely poorly.

• The alternative developments can simply be optimised to maximise net dollar outcomes.

• However, work already done on outcome indicators shows that this ideal will rarely occur in practice.

MULTI-CRITERIA ANALYSIS Example of simulation resultsExample of simulation results

Which scenario is best?Which scenario is best?

Computed value of each indicator

Scenario A Scenario B Scenario C Scenario D

Indicator 1

(benefits $’000)7,800 9,000 6,500 8,800

Indicator 2

(area ha)75,000 60,000 94,000 94,000

Indicator 3

(flow m3/s)560 590 610 550

Indicator 4

(concentrationmg/l)

0.33 0.67 1.75 0.02

Indicator 5

(rate of risem3/s per day)

1,200 1,260 1,190 1,190

MULTI-CRITERIA ANALYSIS Some questionsSome questions

• Are all criteria of equal importance?• Who says so? Who are the decision makers?• Should the evaluation be passed over to the

community or kept for ‘professionals’ to do?• Some indicators do not show much change

between scenarios - should they be dropped from the analysis?

• Is there some other indicator(s) that might be of more help in the scenario evaluation?

• etc

MULTI-CRITERIA ANALYSIS Some answersSome answers

• Experience shows that stakeholders need to be involved if the evaluation is to have credibility. (At basin level this can be high level and broad)

• A structured process is the only successful approach to dealing with complexity.

• At the same time, every effort should be made to have the absolute minimum complexity.

• There are a number of recognised methodologies and proprietary MCA software available.

• However, it is important to remember that the decisions will be socio-political, not technical.

MULTI-CRITERIA ANALYSIS Some Structured ApproachesSome Structured Approaches

• Criteria weighting. This requires joint discussions by stakeholders and ‘experts’ to agree on weighting. Numerical techniques can then be used to score the scenarios.

• DELPHI - a way of combining expert/informed opinions (not especially useful in scenario evaluation, but can help develop weightings)

• AHP (Analytical Hierarchy Process) - one of many pair-wise preference processes - plenty of software available.

• etc

MULTI-CRITERIA ANALYSIS Which is the best approach?Which is the best approach?

• In basin planning, MDBC experience is that the outcomes of any unclear or partially invisible process will frequently be rejected by the stakeholders (and community).

• MDBC and its member States tend not to use MCA software - both for the above reason, and because none seems to work very well for the complex systems of basin natural resources. (Research into MCA continues however.)

• Instead, the common approach is to use stakeholder reference panels and an independent facilitator to work towards consensus.

Evaluating Development ChoicesEvaluating Development ChoicesScenarios, Indicators , MCA and all that.Scenarios, Indicators , MCA and all that.

Thank you

GOOD LUCK!GOOD LUCK!

Questions?

Formulating Indicators for Assessing Scenarios

Documents

Transcript of Formulating Indicators for Assessing Scenarios