1

Choosing from alternative approaches for assessing management effectiveness

Prepared by Graeme Worboys*1, Dan Salzer*2, Marc Hockings*3 and Richard Margoluis*4

*1: Griffith University, Gold Coast, Queensland Australia: IUCN WCPA

*2: The Nature Conservancy, Oregon, USA: Conservation Measures Partnership

*3:The University of Queensland, Gatton, Queensland Australia: IUCN WCPA

*4: Foundations of Success, Maryland, USA: Conservation Measures Partnership

2

This paper….. Discusses management effectiveness evaluation

needs and methods Discusses evaluation needs of protected area

stakeholders and some methods available Describes some attributes of methods Describes opportunities for further research Introduces the work of the Conservation

Measures Partnership to this topic

3

Benefits of management effectiveness evaluations

Effectiveness evaluation can: Improve planning Demonstrate accountability Provide a pivotal role in adaptive management

But: in undertaking evaluation, it is critical that: It provides the right information to the right people at

the right time The evaluation methodology is the right one for the

intended purpose

4

What are some protected area management effectiveness evaluation needs?

Design Needs

Evaluating whether the design of the protected area site or system is appropriate to the values it seeks to maintain.

Management Process Needs Evaluating whether the management systems and processes are adequate and appropriate for the protected area site or the protected area system

Conservation Outcome NeedsIs the site or system effective in maintaining biodiversity, abating threats and achieving other management objectives?

5

What triggers an effectiveness evaluation?

Political requirementsLegal requirementsManagerial requirements Environmental requirementsSocial considerations Economic/financial requirements

6

Current status of management effectiveness approaches Many different approaches used by different

agencies/organizations

Little collaboration on what approaches to use

Lack of knowledge on what works, what doesn’t

No consensus on framework on which to compare and contrast different approaches

7

Harmonization of approaches can help protected area management

Enhances the comparability of data across systems

Improves communication through a “common” language

Reinforces a core group of attributes to use when evaluating management effectiveness

8

What are some attributes for PA management effectiveness evaluation approaches?

Principal evaluation subject: Protected Area/System Design Management processes Conservation condition/status of natural and cultural heritage Mixed subjects

Internal or External (to a management organization)

Low cost or High cost

9

What are some attributes for PA management effectiveness evaluation approaches?

Duration; frequency; time frame for evaluations (e.g., short duration, single evaluation event)

Focus on specific element(s) of an identified Evaluation Framework(e.g., WCPA framework)

Context Planning Input Process Output Outcome

10

What are some attributes for PA management effectiveness evaluation approaches? Type of Data analysis/compilation

Scorecards based on perception-based monitoring Analysis derived from protected area monitoring program data

A diversity of subject areas evaluated within elements

Descriptions of how the evaluation works

11

If we look at some evaluation approaches, what are their broad characteristics?

A preliminary appraisal of 30 evaluation approaches illustrated that: There were a large number of attributes (more than

300)

External organizations played a key role (16 were external)

The majority of evaluations were short in duration (22)

The majority used questionnaires (17)

Outcomes (23), inputs (18) and processes (16) were important WCPA elements

12

If we look at the 30 evaluation approaches, what are the most common attributes that have been used?

Boundaries (5)

Natural heritage condition (6)

Supportive Legislation (10)

Plan of Management existence (14)

Key threats (issue identification) (4)

Sufficient staff (12), $ (8), equipment resources (4)

Education (4)

13

If we just look at evaluations conducted by the Protected Area agencies(9) within the 30 approaches reviewed, we observe the following characteristics:

Many were:

Site based (5)

Long term (7)

Outcome focused (9)

Included Qualitative and Quantitative data (7)

14

Important questions to answer before selecting the effectiveness evaluation approach:

What is needed?

Who needs it?

Why is it needed?

Who does it?

When is it needed by?

At what cost?

15

We suggest you obtain guidance from existing evaluation approaches that have similar attributes to your evaluation needs:

Short term; low cost; questionnaire based; mixed elements; local area; external evaluator

Long term; high cost; internal; output/outcome focused; on site survey work; local area.

Eg: Some generic attributes that you may need:

Example of an existing evaluation approach with similar attributes (for your background research)

Tasmania World Heritage Area Monitoring and Evaluation Framework (Jones 2000)

WWF/World Bank Forest Alliance: Reporting Progress at Protected Area Sites (Stolton et al 2003)

16

Choosing between alternative evaluation approaches……. Needs a clear understanding of what is

required Needs an understanding of the alternative

approaches available Needs an awareness that there are pre-

existing methods that can help Needs further research to facilitate the

process between needs and approaches

17

Opportunities for research and action…..

Develop an approach to clearly identifying PA management Effectiveness Evaluation needs

Determine generic attributes of Effectiveness Evaluation methods

Clarify conditions under which Effectiveness Evaluation methods work to address specific needs.

Develop a decision tree

18

Acknowledgements: Appreciation is extended to Dr Jamison (Jamie) Ervin for the

opportunity to use unpublished material for this presentation.

19

Harmonizing Management Effectiveness Approaches

The Conservation Measures Partnership

Dan Salzer, TNC

World Parks Congress Durban, South AfricaSeptember 12, 2003

20

Core Members African Wildlife Foundation Conservation International The Nature Conservancy Wildlife Conservation Society World Wildlife Fund/World Wide Fund for Nature

Collaborating Members Enterprise Works Worldwide Foundations of Success (CMP – facilitator) World Commission on Protected Areas Cambridge Conservation Forum

Who is the Conservation Measures Partnership (CMP)?

21

Why did CMP form? Capitalize on collective experience Reduce duplication of effort Promote rapid diffusion of successful tools

and strategies Adopt best-practices from ongoing projects Make conservation more efficient

22

CMP’s Vision

Transform the practice of conservation by developing, testing and sharing tools to credibly assess and improve the effectiveness of conservation investments

23

Focus of CMP’s current work Consensus set of standards for the practice

of conservation Conservation “Rosetta Stone” Peer-review conservation audit standards Activity-based cost accounting Global status indicators Management effectiveness indicators

24

CMP’s role in comparing alternative management effectiveness approaches Members contribute alternative approaches

to include in comparisons

Develop a CMP conservation audit process derived from our agreed upon conservation practice standards

Help with analysis of alternatives & guidance for selecting alternative approaches

25

For more information ….

www.ConservationMeasures.org