Performance Measures

Enhanced Forest Management Pilot ProjectPerformance Measures

April 1997 (Revised September, 1997)(Updated September, 1998)(Updated November, 1999)

E F M P P

Ministry of Forests

Enhanced Forest Management ProjectPerformance Measures

Table of Contents

Page

1. Introduction..........................................................................................................................

2. Methodology.........................................................................................................................

3. Performance Measures Framework.....................................................................................3.1 Characteristics of Performance Measures.....................................................................

3.2 The Steps to Developing Performance Measures..........................................................

4. EFM Project Objectives and Performance Measures...........................................................4.1 Overall Project Measures................................................................................................

4.2 Linkage of Objectives and Measures.............................................................................

4.3 Committee Objectives and Measures............................................................................

Appendices

A Pilot Project Evaluation Framework

B Interviews - Participants and Interview Guide

Enhanced Forest Management Pilot ProjectPerformance Measures1 Page 1

1. Introduction

The objective of this project is to clearly define the EFM project objectives and establish a preliminary set of performance measures and an evaluation framework by which to set targets and track overall project progress and outcomes.

2. Methodology

Minutes from meetings and documentation on the EFM project were reviewed. A draft summary of objectives with some examples of performance measures was compiled. A set of interview questions relating to project objectives and expectations was faxed to several individuals associated with the EFM project prior to a half hour telephone interview with each (see Appendix B). The purpose of the interviews was to:

clarify the EFM project objectives;

develop a few key performance measures for the various aspects of the EFM project; and

obtain comments on the EFM project committees and process

Comments from the interviewees are incorporated into the objectives and performance measures presented in Section 4.0

3. Performance Measures Framework

3.1 Characteristics of Performance Measures

Effective performance measures will often have the following characteristics:

they support the project strategy

they show trends

they include targets that promote continuous improvement

they contain both leading and lagging measures (i.e. performance drivers vs. outcomes)

they have impacts that are ‘forecastable’ and/or measurable

they are balanced (not all economic, or environmental etc.)

they are easily measured with current systems

Each measure should have its own specification document which contains the following information:

Measurement name: an easy to understand name that describes the activity or result being measured and can be recognized as supporting a critical strategy or objective

Definition: the meaning of the measurement

Computation: the precise equation including the specific data components and data sources


Champion: the person who is knowledgeable about the measurement area and is responsible for measurements, results and implementation

Frequency of Collection/Reporting: how often the measurement will be collected and reported

Purpose: the reason for using the measurement, including its link to strategies

Reporting levels: who is going to get reports on the measurement and in what format

3.2 The Steps to Developing Performance Measures

1. Define project objectives and desired outcomes.

2. Once the objectives are defined, develop the performance measures. Ask questions such as : how will you know when objective x is achieved?

Measures can focus on:

inputs (what is consumed by the program) These are also referred to as “leading” performance measures or “performance drivers”

outputs (what is produced by the program) outcomes (results or impacts of the program activities)

Both outputs and outcomes can be referred to as “lagging” performance measures that result from the performance drivers (see Section 4.2).

Minimize the number of performance measures listed. Ask questions like:

is this measure relevant to the strategies or objectives of the program? who will use it and how? who will be accountable for the measure? how easily can the measure be understood and communicated? how easy will it be to gather the data for the measure?

3. Develop the measurement strategy. For each measure, outline how the information will be collected, by whom and how often.

4. Record all of the above in a table to demonstrate the linkage between strategy or desired outcome and the measures. This ensures that the measures are linked to the strategy and are not proposed simply because they would be nice to know.


4. EFM Project Objectives and Performance Measures

4.1 Overall Project Measures

There are two overall objectives for the EFM project which bridge all of the elements of sustainability (economic, environmental and social). These are:

Maximize forest productivity; and, Operationalize (implement) local forest management strategies within existing land use plans and

zoning initiatives

Underlying these overall project objectives are specific project objectives which fall under one of the elements of sustainable forestry:

Economic Optimize economic efficiency Empower local forest industry managers

Environmental Optimize biological response Accomplish objectives within the parameters of the Forest Practices Code and higher level plans

Social Optimize a positive impact on the community Involve the public in the development of a local forest management strategy

This set of objectives will need to be balanced in order for the project outcomes to be sustainable. A balanced set of objectives and appropriate performance indicators will then form the decision framework against which FRBC funding priorities can be developed and benefits assessed. Figure 1 illustrates the interrelationship.

Figure 1 : Enhanced Forest Management Project Objectives : Sustainability Analysis

ENVIRONMENT(ecosystem)

Optimize the Biological Response to Management Input

Empower Local Managers Optimize Economic Efficiency

OptimizePositive Impact on the Community Involve Public

ECONOMIC(industry)

SOCIAL(community)

•

•

Maximize Forest

Management Strategy

Productivity Successfully Implement

within existingLand Use Plan

Ojectives

PositivePositive

Figure 2 EFM Project Objectives and Performance Measures Framework

Overall Project Objectives

Optimize Forest Productivity Increase pilot site harvestable MAI Increase productive land base Decrease area lost to unsalvageable losses

Successfully Implement Enhanced Forest Management Strategy Develop and implement local forest

management strategies in each pilot area within existing Land Use Plan Objectives

EFM Strategy focusing FRBC land based programs funding

Optimize Positive Impact on Community Increase economic activity

generated per m3 harvested

Promote education on enhanced forest management

Optimize the Biological Response to Forest Management Input Resource inventories demonstrate that the non-timber

resource land use objectives are not compromised by enhanced forest management activities

Optimize biological effectiveness of forest management activities

Optimize Economic Efficiency Decrease investment needed

to increase the harvestable Revenue/ha Cost/Benefit per stand, forest

and landscape levels

Empower Local Managers EFM strategy incorporated

into forest development plans Local forestry manager has

clear accountability for forest management and results on the ground

Involve Public Adequate stakeholder

representation on advisory groups

Stakeholders either support EFM Strategy or recognize and accept the rationale for the Strategy

Pilot Project and Working Groups (WG’s)

Increase Forest Productivity in Pilot Sites Measured at Five Year IntervalsLeading (Performance Driver) % of area planted with genetically improved

seedlings % area planted Area of incremental silv treatment i.e.,

fertilizing Disturbance rates and area by type (fire,

insects, disease) Interim measures leading towards increased

productivityLagging (Outcomes) Average regen delay (years) Years to reach free growing status % increase in MAI (m3/ha) % area of previously uneconomic or inoperable

timber now accessed % area of productive land lost through

harvesting (i.e., roads and landings)

Develop Framework for Implementation of Plans and EFM Objectives Forest resource database, resource

targets, performance measures, and comm plan developed within one year of pilot commencing

Build & Implement EFM Strategy Forest mgmt strategy and

recommendations (including Action Plan for implementation) are developed & delivered as per the Pilot Project Plan

Complete EFM Strategy and commence implementation within 3 years

Direct Project Benefits to Community Leading (Performance Driver) Local expenditures ($/yr)Lagging (Outcome) Direct and indirect

contribution to local economy ($’s, jobs)

develop curriculum package suitable for grade schools

Improve Forest Resource InventoriesLeading (Performance Driver) Forest resource inventories initiated, updated or

recommendedLagging (Outcome) Types of resource inventories digitized and available for

local resource planning

Increase Biological Effectiveness Carry out field trials, initiate research and make

recommendations on the timing location and intensity of silviculture practices

Maintain Forest Biodiversity of Pilot SiteLeading (Performance Driver) maintenance of species diversity and seral stage

distribution as per land use objectivesLagging (Outcome) Forest cover distribution at the landscape level (stand type;

pattern (geographic); stage of development) Birds & mammals - presence/absence, distribution,

Maximize Annual Return on Investment at stand, forest Level Leading (Performance Driver) Decrease expenditures on

treatments/ha land baseLagging (Outcome) Decrease in treatment costs

and increase in revenue per ha

Local Forest Managers and Local Stakeholders Develop Pilot Project EFM StrategyLeading (Performance Driver) Stakeholder representation, turnout, and input at local advisory

group meetings Number and frequency of meetings Local managers sign-off on EFM strategyLagging (Outcome) Public satisfaction with management plan (survey, media articles,

number of complaints) Annual report of implementation and effectiveness of EFM

strategy to SC by local forestry manager

Promote the Enhanced Forest Management Pilot Project and Communicate Project Learnings Number of notifications, bulletins, articles re: EFM strategy Develop and Implement Communication Plan

Steering Committee(SC)

Monitor Results SC performs annual review of the EFM

strategies performance measures as implemented by the Pilot Projects

Review interim measures of pilots for phases leading up to an EFM strategy

Facilitate/Guide Local Strategies WG’s report to SC on a bimonthly basis

re: pilot project performance Working Groups receive timely feedback

from SC re: issues raised

Review Evaluation Criteria Evaluates the biological, socio-economic, and community impact performance measures on an annual basis to ensure that they

are appropriate and are an adequate measures of progress towards the project objectives

Empower Local Managers

Provide WG’s with guidance & resources to enable the development of EFM strategy for the Pilots

Monitor Stakeholder Involvement Endorse pilot project WG

participants, public involvement process, and advisory group structure

Promote the Enhanced Forest Management Pilot Project and Communicate Project Learnings Number of policy recommendations put forward to and acted on by government Ensure technical reports and strategy documents developed through the project are adequately distributed to government and industry decision makers Provincial communications plan developed and implemented as per Terms of Reference Ensure Pilot communication plans are developed and implemented as per Pilot Project Plans

Technical Advisory Committee (TAC)

Initiate Research to Support Implementation of Strategies Number of research needs identified by

WG’s and reviewed by the TAC

Assist with Selection and Application of Timber Supply Models and Socio Economic Analysis Tools Ensures that the WG’s modelling approach is adequate to address the pilot project objectives Ensure socio-economic analyses for each pilot are appropriate for meeting their objectives

Ensure Proper Use of ModelsProvide ongoing advice and training where feasible, to pilot WG members to facilitate efficient use of various forest management models and to understand the modelling results.

Provide Technical Advice to Pilot Project Local Working Groups Advice given on time as requested by WG’s

Enhanced Forest Management Pilot ProjectPerformance Measures 5

4.2 Linkage of Objectives and Measures

The overall project objectives and performance measures for the overall project, the pilot projects and each committee are illustrated in Figure 2. A separate column has been developed for each major objective. The linkages between the overall project objectives and the committee and pilot project objectives are shown in each column.

In order to track cause and effect, the pilot project measures are presented in terms of leading and lagging performance measures. Leading measures are performance drivers - they relate to the cause. Lagging measures are the outcome as a result of the performance driver. The specific links between the two sets of measures will need to be established as the pilot projects are implemented.

4.3 Committee Objectives and Measures

The following section provides more detailed information with respect to the work plan activities of the Steering Committee and the Technical Advisory committee and the linkage of performance measures to these activities.

4.3.1 Steering Committee

Table 1 outlines Steering Committee objectives and the activities associated with these objectives that should be included in the Steering Committee work plan. The achievement of these activities as per the work plan schedule could constitute the performance measure. The target would be the anticipated date for completion of the activity.

4.3.2 Technical Advisory Committee

Similar to the Steering Committee work plan, Table 2 outlines the objectives of the Technical Advisory Committee and the activities associated with these objectives that should be included in the Committee work plan. The achievement of these activities as per the work plan schedule could constitute the performance measure. The target would constitute the anticipated date of completion.

4.3.3 Pilot Projects

Appendix A is a draft format for evaluating pilot project performance measures. The project objectives or goals are fairly well established and a sample of corresponding performance measures have been included. Each pilot site should select or add measures that will most accurately represent the pilots progress towards the objectives. Once established this table should become a reporting tool and ultimately be incorporated into the pilot management strategy document.

Enhanced Forest Management Pilot ProjectPerformance Measures Page 6

Table 1 - Steering Committee Objectives and Measures

Objective Activity Performance Measure Output and/or Target Date

(goal) (Process) TAC Invermere TFL 39 Burns Lake

Monitor Develop project timelines 1. Review and approve pilot project timelines N/A Approved April 1996 Approved Nov 1996 Approved July 1997

Progress and Results

Conduct bimonthly committee meetings Request reports from WGs and TAC on

key performance measures at bimonthly meetings

2. Meetings held as per schedule3. WGs and TAC report to Steering Committee on

schedule4. Perform annual review of overall project progress

against performance measures and timelines

Meetings held on the second Wednesday bimonthlyRecord reports in minutes at bimonthly meeting

Hold reviews in January of each year

Empower Local Endorse project work plan 1. Workplan approved Submitted March ‘97 Approved April 1996 Approved Nov. 1996 Approved Sept. 1997

Managers Provide resources to support strategy development

2. Each pilot site and TAC has sufficient resources to carry out workplan

Committee formed with annual budget

Annual budget plus 3 FTEs

Annual budget plus contractors

Annual budget plus coordinator

Promote the Project

Make presentations and provide project updates to FRBC and FSSC

Develop back grounder and update as necessary

Develop generic presentation for project

1. Number of presentations and level of positive response

2. Provincial communications plan developed and implemented as per Terms of Reference

3. Generic presentation prepared on schedule

Van Isl. IAMC in Oct. 1996; FRBC Env. Comm. in May 1997; FRBC Board in Sept. 1997 and Committee members promote project at other associations and committees they attend Provincial backgounder released June 1996. Update due for release in December 1997

Generic presentation produced in July 1996 and distributed to all committee membersCommunicate Project

Review and approve pilot communication plans

1. Pilot project communication plans approved as per the work plan schedule

N/A Approved July 1996 Partially approved Nov 1996

Approved Sept 1998

Learnings Technical reports and strategy documents developed by WGs and reviewed

Develop process for meeting with gov’t officials to recommend policy changes

2. Technical reports and strategy documents adequately distributed to government and industry decision makers

3. Number of policy recommendations put forward to and acted on by government

None to Date 7 technical reports reviewed and published between Sept ’97 and Feb ‘98

2 technical reports reviewed and released (distribution?)

None to Date

Monitor Stakeholder Involvement

Endorse local working group participants Review public involvement/advisory

Group process

1. Multidisciplinary working group2. Public involvement process implemented and with

information transfer on a regular basis

N/A Advisory group established; 2 news paper supplements

Public has been approached. Met with enviro groups

Process will be underway by Sept 1998

Facilitate/Guide Local Strategy Development

Request reports from WGs and TAC Provide advice to WGs re: analysis and

implementation of strategies

1. Working Groups report to Steering Committee on a bimonthly basis

2. Working Groups receive timely feedback from Steering Committee re: issues raised

Record reports in minutes at bimonthly meeting

Track issues and resource requests in minutes for timely feedback

Assist with data and resource acquisition

3. Working group strategies presented to Steering Committee on schedule

N/A June 1998 March 1999 March 2000

Review Evaluation Criteria

Review criteria in terms of impacts on timber supply, environment and employment

Review resource scenarios as available Review socio-economic indicators and

financial evaluation principles

1. Biological and socio-economic performance measures are appropriate and adequate measures of progress towards the project objectives

2. Resource scenarios reviewed as per pilot timeline3. Socio-economic impact of enhanced forest

practices are quantified

TAC developing soci-economic indicators

SEA completed; employing Woodstock silv optimization model

SEA underway; employing Woodstock silv optimization model

SEA to commence Fall 1998

Enhanced Forest Management Pilot ProjectPerformance Measures Page 7

Table 2 - Technical Advisory Committee Objectives and Measures

Objective Activities Performance Measures Targets (date)

Selection and application of timber supply and socio-economic analysis models

Ensure that WG approach is adequate to address pilot site objectives

Ensure socio-economic analysis for each pilot area is appropriate for meeting objectives

1. Feedback to Working Groups on time as per pilot project schedule

2. Economic analysis updates and results provided on time as per pilot project schedule

Ongoing and reviewed each year

Ensure proper use of models

Provide ongoing training and advice as feasible

1. Positive feedback from pilot sites on service requested


Provide technical advice Pilot Working Groups as requested

Provide advice to WGs re: data requirements

Assist with defining alternative forest management scenarios to support local objectives

1. Prompt response and assistance as requested with positive feedback from pilot sites


Initiate researchto support implementation of strategies

Determine research needs 1. Number of research needs identified by Working Groups and reviewed by TAC


Share Learnings with other areas of the province

Meet with government and industry officials in other areas of the province

1. # Meetings conducted with government and industry as per TAC schedule


Enhance Forest Management Pilot ProjectPerformance Measures Appendix A-1

Table 3 - Pilot Project Evaluation Framework

Project Objectives(Goal)

Strategy (How to accomplish

goal)

Performance Measures (How to track

achievement of goal)

Target & Acceptable Range

Data Source/ Definition

Frequency of Reporting

Increase forest productivity in pilot site (measured at 5 year intervals)

Leading (Performance Driver) % of trees planted that are genetically improved % area naturally regenerated % of area meeting free to grow requirements

annually Area of stands treated (fertilizing, spacing,

brushing, etc.) Area harvested by type of silviculture system

(clearcut, selection, partial retention) Disturbance rates and area by type (fire, insects,

disease, logging, etc.)Lagging (Outcomes) Average regen delay (years) Years to reach free growing status % increase in MAI (m3/ha) m3 per ha harvested % of fibre that meets utilization standards NSR area as % of annual harvest area % area of previously uneconomic or inoperable

timber brought into production % area of productive land lost through

harvesting (i.e., roads and landings)

Develop framework for Implementation of Plans and EFM Objectives

Forest info system, resource targets and performance measures developed within one year of pilot commencing

Build EFM Strategy Action plan for implementation developed on schedule

Database of implementation progress developed on schedule including forest productivityemployment forest practices

Recommendations/reports presented to steering committee on schedule

Implement EFM Strategy (within 3

Stakeholder buy-in, and Advisory Group and Steering Committee approval





goal)






years of commencing pilot)

Direct some project investments toward the community

Leading (Performance Driver) Local expenditures ($/yr) Decrease cutting permit approval process timeLagging (Outcome) Direct and indirect contribution to local economy

($’s, jobs) Number of local jobs created in short term and

estimate of jobs over the longer term Increase in local average % of year employed secure 5 yrs of clear cutting permit approvals by

1999 develop curriculum package suitable for grade

schools

Improve forest resource inventories

Leading (Performance Driver) Forest resource inventories initiated, updated or

recommendedLagging (Outcome) Types of resource inventories digitized and

available for local resource planning Ability to adequately measure other resource

values to see if land use objectives are being maintained

Optimize biological effectiveness

Carry out field trials, initiate research and make recommendations on the timing location and intensity of silviculture practices

Maintain forest biodiversity of the pilot site

Leading (Performance Driver) maintenance of species diversity and seral stage

distribution as per land use objectivesLagging (Outcome) Forest cover distribution at the landscape level





goal)






(stand type; pattern (geographic); stage of development)

Birds and mammals - presence/absence, distribution, habitat supply

Maximize annual return on investment at stand/forest level

Leading (Performance Driver) Decrease expenditures on treatments/m3 cut Decrease expenditures on treatments/ha land

baseLagging (Outcome) Return on treatments

MAI log value

operating earnings/m3

Local forest managers and local stakeholders develop pilot project EFM strategy

Leading (Performance Driver) Stakeholder representation and turnout at local

advisory group meetings Stakeholder advisory committee input during

strategy development Number and frequency of meetings Local managers sign-off on EFM strategy Number of notifications, bulletins, articles re:

EFM strategyLagging (Outcome) Turnover rate and attrition of Working Group and

advisory committee over the planning period Public satisfaction with management plan

(survey, media articles, number of complaints)

Enhanced Forest Management Pilot ProjectPerformance Measures Appendix B

Participants and Interview Guide

The following members of the Steering Committee were contacted for their comments on the EFMP project objectives and suggested performance measures:

Rick Brand Gordon Baskerville Stan Coleman Greg Anderson Barry Phillips Bronwen Beedle Dana Bonnieux Dave Peterson Don Laishley Rick Jeffery

The following questions served as an interview guide:

Project Objectives

1. Five years from now, we will know that the EFM project has been a success if....

2. What is your main expectation from the project?

3. How will other forest operations in the province benefit from the EFM project?

4. How will the government benefit from the EFM project?

5. What is forest productivity and how will we know if it has been increased through EFM?

6. What is biological effectiveness and how is it important to this project?

7. How does economic efficiency relate to the EFM project?

8. What does “local empowerment” mean and how do you know if it has been encouraged or achieved?

9. If land-use planning and zoning initiatives are successfully “operationalized”, it will be apparent because...

The EFM Process

10. What are the key challenges of the Steering Committee and how will it know if it is doing an effective job?

11. What information does the Steering Committee need to review on a regular basis from the TAC and the Working Groups?

12. What are the key challenges of the TAC and how will it know if it is on track with achieving the EFM project objectives?

13. What information does the TAC need from the Working Groups?

14. What objectives and performance measures have been developed to track the success of the pilot site initiatives? (Examples…, discuss…)

Performance Measures

Business

Transcript of Performance Measures