Indicators: What are Developed for What?
Transcript of Indicators: What are Developed for What?
Ranping Song
World Resources Institute
Manila
Nov 2014
Indicators: What are Developed for What?
• Overview
• Indicator Framework to Track Progress of Goals
• Indicator Framework to Track Specific Policy, Action and Project
– Input Indicators
– Activities Indicators
– Intermediate Effect Indicators
– GHG and Non-GHG Effect Indicators
• Monitoring and Data
Outline
• A Working Group to summarize several organizations on work on MRV: CCAP; DECC; ECOFYS; GGGI; GIZ; UNDP; WRI
• Show where there is complementary work and where there are gaps
• A Spreadsheet Organized by:
– the type of activity (e.g. methodology, tool or metrics and indicators
– the level of MRV (e.g. bottom up (facility, corporate level) or top down; or project, policy or goal)
– the scope (sector specific, cross-sectoral, or initiative specific)
– the organization leading the work
– a description of the activity or project
• Emission, Emissions Reductions, Non-GHG metrics, Implementation, Finance
Status of MRV Indicators
Snapshot of Indicators and Metrics
MRV of Emissions
Type Level Scope Organization Activity/Product
Metrics and Indicators Top-down N/A GGGI Peru Forest: Development (with Govt of Peru) of National Forest Plan for Green Growth Example: Peru Forest: National GHG emissions inventory; regional and local inventories of GHG emissions in forest sector
Metrics and Indicators Top-down N/A DECC Example: KPI - Change in Greenhouse Gas (GHG) emissions as a result of ICF support. (Also considering gathering data at country level on emission intensity of GDP based on the methodology for GHG emission reduction.)
Metrics and Indicators Top-down N/A UNDP Foreseen standardized baseline work for the agriculture sector and rural electrification sector will contain a series of indicators and metrics to quantify baseline GHG emissions within a sector's boundary. The indicators and metrics enable the objective comparison of mitigations actions against a baseline.
Metrics and Indicators Top-down Cross-Sectoral WRI GHGP- Global Protocol for Community-Scale Greenhouse Gas Emissions (City Inventory) methodologies dictate indicators/methods/data that need to be collected. Examples: GHGP City Inventory: Emissions by Sources: Total GHG emissions (in tCO2e). For sources included in GPC 2012 BASIC; if quantification is not possible, Notation Keys should be used. The total number of occurrences of each Notation Key and relevant GPC reference number should be indicated. If GPC 2012 BASIC+ or EXPANDED is chosen, sources that are included should be clearly indicated. Emissions by Scopes: Indicate the scope of each emission source, and separate total emissions by scope 1, scope 2, and scope 3. It is noted that in reporting by ‘scopes’, complete Scope 1 emissions must be reported including emissions from Energy Generation (GPC I.3.1). Gases: Data for CO2, CH4, N2O, HFCs, PFCs, and SF6 in metric tons and in tons of CO2 equivalent should be reported. Data quality: High (H): localized emission factors and detailed activity data Medium (M): national emission factors or generic activity data Low (L): international/national emission factors and generic activity data Year: Year of inventory or emission data Quantification: Report source or sector-specific quantification methods used
Metrics and Indicators Bottom-up N/A DECC Key Performance Indicators
Metrics and Indicators Bottom-up N/A WRI GHGP- Corporate Accounting and Reporting Standards (Corporate Standard) methodologies dictate indicators/methods/data that need to be collected. Examples: Corporate Accounting: Emissions disaggregated by source types Scope 1: Direct Emissions from Owned/Controlled Operations a. Direct Emissions from Stationary Combustion b. Direct Emissions from Mobile Combustion c. Direct Emissions from Process Sources d. Direct Emissions from Fugitive Sources e. Direct Emissions from Agricultural Sources Scope 2: Indirect Emissions from the Use of Purchased Electricity, Steam, Heating and Cooling a. Indirect Emissions from Purchased/Acquired Electricity b. Indirect Emissions from Purchased/Acquired Steam c. Indirect Emissions from Purchased/Acquired Heating d. Indirect Emissions from Purchased/Acquired Cooling
MRV of GHG Emission Reductions
Type Level Scope Organization Activity/Product
Metrics and Indicators
Project Level N/A GGGI Peru Forest: Development (with Govt of Peru) of National Forest Plan for Green Growth Examples: Area of forest under protected status; Area covered by primary forest; forestation and deforestation rates (Ha / year); GHG emissions intensity of production (Ton GHG / unit of GDP); Energy intensity of production (Mwatts / unit of GDP);
Metrics and Indicators
Project Level N/A ECOFYS Indicators for monitoring GHG benefits of NAMAs (specific to each sector/ NAMA) under the Mitigation Momentum project but also in other countries (e.g. Tanzania, Ethiopia, etc.) For all countries, MRV approach is under development
Metrics and Indicators
Project Level N/A DECC Different indicators depending on the type of intervention around capacity, private sector involvement (e.g., number of loans)
Metrics and Indicators
Project Level N/A GIZ Has developed metrics and indicators for project level (Climate Results Sourcebook)
Metrics and Indicators
Project Level N/A WRI GHGP- Project Accounting Protocol and Guidelines dictate indicators/methods/data that need to be collected. Examples: http://www.ghgprotocol.org/files/ghgp/ghg_project_protocol.pdf
Metrics and Indicators
Policy Level N/A ECOFYS Indicators for monitoring GHG benefits of NAMAs (specific to each sector/ NAMA)
Metrics and Indicators
Policy Level N/A DECC Considering including energy savings in our list of KPIs
Metrics and Indicators
Policy Level N/A WRI GHGP- Mitigation Accounting Guidance - Policies and Actions Standard methodologies dictate indicators/methods/data that need to be collected.
Metrics and Indicators
Target Level N/A GIZ At outcome level: Change in GHG emissions reduced or avoided (quantitative)
Metrics and Indicators
Target Level N/A WRI GHGP- Mitigation Accounting Guidance- Mitigation Goals Standard methodologies dictate indicators/methods/data that need to be collected.
MRV of Non-GHG Metrics
Type Level Scope Organization Activity/Product Metrics and Indicators Project Level N/A GGGI Peru Forest: Development (with Govt of Peru) of National Forest Plan for Green Growth Examples: Economic: Output (Forest GDP) including timber and non-
timber production, tourism, payment for ecosystem services (incl. REDD), production of small-scale users. Social: Poverty rate, unemployment rate
Metrics and Indicators Project Level N/A ECOFYS Indicators for measuring impacts- co-benefits/ transformational change of NAMAs (specific to each sector/ NAMA)
Metrics and Indicators Project Level N/A DECC Project specific indicators. Examples: As part of the 15 KPIs: Level of installed capacity of clean energy as a result of ICF support; Number of low carbon technologies supported (units installed) through ICF support; Value of ecosystem services generated or protected as a result of ICF support;
Metrics and Indicators Project Level N/A GIZ Indicators for transformational change (NAMA Facility), based on LCD indicators of DFID • Carbon productivity of GDP • Emissions intensity of poverty reduction • Quality of the enabling and investment environment • Scale of technology deployment • Private sector investment levels • Growth and job creation • Private sector engagement and development
Metrics and Indicators Project Level Cross-Sectoral GIZ (with URC) Identification of success factors for transformational change processes, and quantification of sustainable development indicators.
Metrics and Indicators Policy Level N/A CCAP Voluntary MRV of progress, action and sustainable development indicators to improve policy performance and garner domestic support - MRV for NAMAs: Tracking progress while promoting sustainable development- MRV of NAMAs: Guidance for Selecting Sustainable Development Indicators -
Metrics and Indicators Policy Level N/A ECOFYS Indicators for measuring impacts- co-benefits/ transformational change of NAMAs (specific to each sector/ NAMA)
Metrics and Indicators Policy Level N/A DECC 15 key performance indicators for monitoring and evaluation of climate projects. Also, low carbon thematic indicators have been created but we plan to further develop some methodologies and gather data at country level for the following: % of population with access to electricity (with a breakdown off-grid/on-grid), marginal abatement cost of abatement measures at country level; % of national workforce in environmental goods and services sector; fuel mix for energy at country level, national imports of fossil fuels Examples: Numbers of people supported by ICF programmes to cope with the effects of climate change; Number of people with improved access to clean energy as a result of ICF programmes; Number of people with improved resilience as a result of ICF support
Metrics and Indicators Policy Level N/A GIZ At output level: • Number of low carbon technologies supported (quantitative) • Level of installed capacity of clean energy (quantitative) • Number of hectares where deforestation and degradation avoided (quantitative) • Number of forest dependent people with livelihood benefits protected (quantitative) • Number of direct jobs created (quantitative) • Volume of public finance mobilized for climate change purposes (quantitative) • Volume of private finance mobilized for climate change purposes (quantitative) • Level of institutional knowledge of climate change issues (qualitative) At outcome level: • Change in GHG emissions reduced or avoided (quantitative) • Number of people with improved access to clean energy (quantitative) • Value of ecosystem goods and services generated or protected (quantitative) • Level of integration of climate change into national planning (qualitative) • Extent to which NF interventions are likely to have a transformational impact (qualitative)
Metrics and Indicators Target Level N/A DECC As part of the 15 KPIs: Level of integration of climate change in national planning as a result of ICF support; Level of institutional knowledge of climate change issues as a result of ICF support; Extent to which ICF intervention is likely to have a transformational impact. Forestry Specific- As part of the KPIs: Number of hectares where deforestation and degradation have been avoided through ICF support; Number of forest dependent people with livelihoods benefits protected or improved as a result of ICF support.
MRV of Implementation
Type Level Scope Organization Activity/Product
Metrics and Indicators
Project Level N/A ECOFYS Indicators for NAMA implementation (specific to each NAMA)
Metrics and Indicators
Policy Level N/A CCAP Voluntary MRV of progress, action and sustainable development indicators to improve policy performance and garner domestic support - MRV for NAMAs: Tracking progress while promoting sustainable development- http://ccap.org/assets/MRV-for-NAMAs-Tracking-Progress-while-Promoting-Sustainable-Development_CCAP-November-2011.pdf
Metrics and Indicators
Policy Level N/A ECOFYS Indicators for NAMA implementation (specific to each NAMA)
Metrics and Indicators
Policy Level N/A DECC Plan: UK is interested in measuring the quality of national LCD strategy - ranking countries against NAMA registry and develop scorecards to compare countries performance
Metrics and Indicators
Policy Level N/A GIZ Indicators for assessing key barriers to implementation
Metrics and Indicators
Policy Level Cross-Sectoral WRI Methodologies in Policy Implementation tool dictate indicators/methods/data that need to be collected. The Policy Implementation Tool measures four key “functions” that are essential to policy implementation and five “principles” of good governance. Functions: Finance; Administration; Monitoring, Reporting and Revision; Compliance and Enforcement. Principles: Role and Responsibility; Transparency; Coordination; Stakeholder Participation; Institutional Capacity
MRV of Finance
Type Level Scope Organization Activity/Product
Metrics and Indicators
Project Level N/A GGGI Peru Forest: Development (with Govt of Peru) of National Forest Plan for Green Growth Examples: Volume of finance for forestation/ reforestation; area under concessions for sustainable forestry
Metrics and Indicators
Project Level N/A DECC As part of the 15 KPIs: Volume of public finance mobilised for climate change purposes as a result of ICF funding; Volume of private finance mobilised for climate change purposes as a result of ICF funding;
Metrics and Indicators
Project Level N/A GIZ Indicators for Private Climate Finance (for enhancing transparency for investors)
Metrics and Indicators
Policy Level N/A DECC Level of private/public finance leveraged. Considering measuring ratio and volume of MBD low carbon investment as a % of total MDB investment at country level
Metrics and Indicators
Target Level N/A GIZ Work on definition of Private Climate Finance
• Management Tool
– Develop Implementation Strategy
– Track Policy Implementation
– Improve Policy Design and Implementation
• Reporting Tool
– Understand Final Effect
– Communication
– Accountability
Purpose of Indicators
The Policy Making Continuum
Source: Climate Policy Implementation Tracking Framework
Goals
Broad policies/Plans
Policy instruments
Implementation of technologies process or practices
Guide
Concretize
Incentivize
LEV
EL O
F D
ETA
IL
HIG
H
LOW
ASPIRATIONAL CONCRETE
1. Track Overall Performance of Sustainable Development Goals
– Specific to the SD goal
– Broad in scope
– Primarily national data
– Need tailor to national circumstances/priorities
– Usually top-down
– Usually effect/outcome indicators
– Can not attribute to specific policy or action
2. Track Performance of Specific Policies, Actions and Projects
– Specific to the policy, action, or project in question
– Primarily at the same scale of the policy/action/project
– Top-down and bottom-up
– Include both process (input and activities) and effect/outcome indicators
– Can not reflect the overall picture
Indicators to Track and Understand What?
Indicators to Track Progress of Goals
• Leadership Council of the Sustainable Development Solutions Network (SDSN)
• Commissioned by UN Secretary-General and mobilize scientific and technical expertise from academia, civil society, and the private sector
• Based on An Action Agenda for Sustainable Development on 10 goals and 30 targets
• 100 Indicators tie to 30 Targets under 10 Goals
• Undergone extensive public consultation with several hundred sets of comments
Indicators for Overall Progress of Sustainable Development
Indicators for Climate and Energy
Source: Indicators of Sustainable Development Working Draft
Indicators for Climate and Energy
Source: Indicators of Sustainable Development Working Draft
Indicators to Track Performance of Policies and Action
GHG Protocol Policy and Action Standard
Objectives:
• Understand effect of policy and action before and after implementation
• Track progress of policy during implementation
Development Process
• Three year process with 30 members in Advisory Committees and 50 experts in working groups
• Piloted in 17 countries with 25 policies
• Received hundreds comments in two round of public consultation
• Asian Development Bank • Australia, Department of Climate
Change and Energy Efficiency • Brazil, Ministry of Environment • California Air Resources Board • CCAP • Chile, Ministry of Environment • China, NDRC • Colombia, Ministry of Environment and
Sustainable Development • Costa Rican Institute of Electricity • Ecofys • Ethiopia, EPA • European Commission • Godrej & Boyce Mfg Co. Ltd., India • India, BEE (TBC) • Japan, Ministry of Environment • Johnson Controls
Advisory Committee members
• Maersk Group • New York City, Mayor's Office • OECD • Siemens • South Africa, Department of
Environmental Affairs • State of Rio de Janeiro • Stockholm Environment Institute – US • Thailand Greenhouse Gas Management
Organization • Tsinghua University • UK DECC • United Nations Climate Change
Secretariat • UNDP • US EPA • WBCSD • World Bank
Objective:
• Develop basis for monitoring progress towards policy adoption and implementation in an applied policy context
Process:
• Piloted in four countries for five policies
Climate Policy Implementation Tracking Framework
Process Indicators
Outcome/Impact/Effect Indicators
Key Performance Indicators for Policy/Action Implementation
Inputs Activities Intermediate
Effect
GHG Effect
Non-GHG Effect
A metric that describes the delivery of resources that support policy implementation
– Finance
– Other inputs Key Factors to consider:
– Any specific sources of funding that have been identified
– Any institution(s) that have been designated as the recipient
– Whether the institution(s) have received the resources designated in a timely manner
Input Indicators
Input function Responsible institution
Indicator Data source(s)
Allocation of $25 million per year to support solar research and development policy
U.S. Department of Energy (DOE)
The amount of funds allocated per year to support solar R&D
DOE budget; funding and financing portion of DOE web site
Example of Finance Indicator
• Policy administration activities that must occur on a regular basis while the policy is in effect.
– Licensing, permitting, and procurement
– Information collection and tracking
– Compliance and enforcement
– Others
Activity Indicators
Example of policies
Examples of administrative government functions
Government buildings retrofit program
Select and procure retrofit products; select contractor for installation (could be through open bidding process or other means)
Fuel economy standard
Issue emission certificates annually; collect information from vehicle manufacturers on cars sold annually; run emission tests on selected vehicles to verify reported performance information
Example of Policies and Functions
Activity Indicators: Licensing, Permitting, and Procurement
• Key Factors to consider:
– Any ongoing administrative functions required by the policy instrument
– The institution(s) responsible for each function
– The quantitative metrics that might indicate functions have been carried out
Licensing, permitting and/or procurement functions
Responsible institution
Indicator Data source(s)
Establish long-term contracts with renewable energy power generators to meet renewable energy goal
Department/Ministry of Energy
Number of megawatts (MW) for which contracts have been issued/total MW or renewable energy
Appropriate government web site; conversations with relevant staff in agency/agencies; existing government monitoring systems (if applicable)
Example of Licensing, permitting, and procurement Indicator
Activity Indicators: Information Collection and Tracking
Information collection and monitoring function
Responsible institution
Indicator Data source(s)
Collect specified information (on model-year vehicle sales, emission levels, fuel economy, vehicle technology, emission test procedure reports etc.) from automakers on an annual basis
Department/Ministry of Transport and/or Government verification agency
Number of automakers supplying information/total number of automakers regulated by the standard
Appropriate government web site; conversations with relevant staff in agency/agencies; existing government monitoring systems (if applicable)
Example of information collection and tracking
Activity Indicators: Compliance and Enforcement
Key Factors to consider:
– Definition of compliance and/or noncompliance
– Consequences of noncompliance
– The institution(s) responsible for compliance
– Action is the responsible authority required or authorized to take to ensure compliance
– Information that would indicate compliance officers are taking action
Compliance/enforcement function
Responsible institution
Indicator Data source(s)
On-site emission testing to ensure compliance
Department/Ministry of Transport and/or Government verification agency
Number of tests conducted on an annual basis
Appropriate government web site; conversations with relevant staff in agency/agencies;
Apply penalties to automakers who fail standards
Number of penalties assigned/number of failed compliance tests
Issue emissions certificates to automakers who comply with standards
Number of emissions certificates issued
Activity Indicators: Compliance and Enforcement
Example: Compliance and Enforcement Indicators for a Fuel Economy Standard
Intermediate effects are changes in behavior, technology, processes, or practices that result from implementation of a policy instrument.
Intermediate Effect Indicators
Examples of policies Examples of intermediate effect indicators
Renewable portfolio standard Total electricity generation by source (such as wind, solar, coal, natural
gas)
Public transit policies Passenger-kilometers traveled by mode (such as subway, bus, train,
private car, taxi, bicycle)
Waste management regulation Tonnes of waste sent to landfills, tonnes of waste sent to recycling
facilities. tonnes of waste sent to incineration facilities
Landfill gas management incentive Tonnes of methane captured and flared or used
Sustainable agriculture policies Soil carbon content, tonnes of synthetic fertilizers applied, crop yields
Afforestation/reforestation policies Area of forest replanted by type
Grants for replacing kerosene lamps
with renewable lamps
Number of renewable lamps sold, market share of renewable lamps,
volume of kerosene used for domestic lighting
Subsidy for building retrofits Number of buildings retrofitted, energy use per building
Information campaign to encourage
home energy conservation Household energy use (sample of households or average use)
• Changes resulted from the policy or action
• Many effects are typically not monitored directly but instead are estimated based on changes in various other parameters.
Effects Indicators
Effect of
policy/action
Historical
2014 2018 2020
• GHG Effect
– Changes in greenhouse gas emissions by sources or removals by sinks that result from the intermediate effects of the policy or action
– Examples: Reduced CO2, CH4 and N2O emissions from the specific policy
• Non-GHG Effect
– Changes in relevant environmental, social, or economic conditions other than GHG emissions or climate change mitigation that result from the policy or action
Effects Indicators
Non-GHG Effects
Examples of non-GHG effects
Environmental
effects
Air quality and air pollutants
Water quality, water pollution, and
water scarcity
Ozone depletion
Waste
Toxic chemicals/pollutants
Biodiversity/wildlife loss
Loss or degradation of ecosystem services
Deforestation and forest degradation
Loss of top soil
Loss or degradation of natural resources
Energy use
Social effects
Public health
Quality of life
Gender equality
Traffic congestion
Road safety
Walkability
Access to energy, thermal comfort, fuel poverty
Stakeholder participation in policy-making
processes
Economic effects
Employment and job creation
Productivity (such as agricultural yield)
Prices of goods and services (such as
decreased energy prices)
Cost savings (such as decreased fuel
costs)
Overall economic activity (such as GDP)
Household income
Poverty reduction
New business/investment opportunities
Energy security/independence
Imports and exports
Inflation
Budget surplus/deficit
Non-GHG Indicators Examples
Indicators
Environment
• annual air pollutant emissions (tons or concentration)
• concentration of pollutants emitted
• number or share (%) of households burning waste
• natural resource consumption (e.g. tons, acres)
• resources consumed per unit of value added (e.g. tons/$)
• level of pollutants in ground/surface water (mg/l)
Social
• share (%) of households/population with access to modern energy
• share (%) of population with access to low-carbon transport
• share (%) of population or households with access to waste management services
• cost per unit of energy
• share (%) of household income spent on fuel and electricity
• number and type of knowledge assets produced
• average cost per passenger-trip
• share of household income spent on transportation
• disease prevalence (various)
• respiratory infections
• number of employees with access to benefits
• per capita or household income
Source: CCAP. MRV of NAMAs: Guidance for Selecting Sustainable Development Indicators
Non-GHG Indicators Examples
Indicators
Economic
• value of imported fossil fuels displaced by incremental renewable power generation ($)
• manufacturing value added (MVA) per unit of energy consumed ($/MWh)
• MVA per value of energy consumed ($)
• avoided cost of energy ($)
• Decrease of energy intensity of GDP (MJ/$)
• Increase of energy production (kWh)
• Decrease of imported fuels (tons of oil equivalent)
• Increase of share (%) of total energy supply from renewables
• Decrease of average fuel consumption per passenger-kilometer
• Decrease of tons of oil equivalent per capita
• net number of jobs created
• investment in new capacity ($)
• deferred or avoided infrastructure costs ($)
• Increase of tax or government fee revenue ($)
• Increase of total annual investment and financial flows in climate change technologies($)
• Value of waste related by-products ($)
Source: CCAP. MRV of NAMAs: Guidance for Selecting Sustainable Development Indicators
Review of Performance Indicators
Inputs
• Finance
• Other inputs
Activities
• Licensing, permitting and procurement
• Information monitoring
• Compliance and enforcement
• Other policy administration activities
Intermediate effects
• Behavioral changes
• Technology changes
• Process changes
Effects
• GHG effects
• Non-GHG effects • Environment
• Social
• Economic
Monitoring and Data
Monitoring Steps
Define key performance indicators
Define data needed for
ex-post assessment
Define the monitoring period
Create a monitoring plan
Monitor parameters over time
Examples of data needs from NAMA MRV plans
Country and Sector Examples of data to be monitored
Mexico (Buildings)
Electricity use (annual, direct metering)
Emission factor from grid electricity
Gross floor area of building units
South Africa
(Energy supply)
Capacity of CSP installed through program
Electricity produced from funded CSP installations
Capacity of wind power installed through program
Chile (Transport)
Number of electric vehicles (quarterly)
Passenger figures (monthly)
Km traveled (monthly)
Tunisia
(Energy supply)
Power installed (MW)
Solar panels produced each year
Amount of exchanged refrigerators
– NAMA implementation period
– NAMA monitoring period
Define the monitoring period
Example
Years
2005 –
2009
2010 -
2014
2015 -
2019
2020 -
2024
2025 -
2029
2030 -
2034
2035 - 2039
NAMA implementation
period
NAMA monitoring
period
• The monitoring plan should describe:
– Measurement or data collection methods and procedures
– Sources of data
– Monitoring frequency
– The level of uncertainty in any measurements or estimates
– Sampling procedures (if applicable)
– Whether the data is verified, and if so, verification procedures
– Entity or person responsible for monitoring and roles and responsibilities of relevant personnel
– Methods for generating, storing, collating, and reporting data on monitored parameters
– Databases and tools (e.g., software) to be used
– Procedures for internal auditing, QA, and QC
Create a monitoring plan
Example of monitoring plan: buildings NAMA
Indicator or parameter (and unit)
Data source Monitoring frequency
Measured/ modelled/ calculated /estimated (and uncertainty)
Responsible entity
Number of houses insulated and insulated area by type (roof, wall, glazing) and m²
ANME information system (to be created)
Annual Measured (Low uncertainty)
National Agency for Energy Conservation (ANME)
• Data availability was a limiting factor for some pilots
• Technical expertise required
• Capacity building may be necessary
• Challenging to apply methodology retrospectively, if appropriate data has not been collected
– Best to put systems in place to collect data and apply the methodologies before the policy or action is implemented
• Once systems are in place, future assessments require less time and resources
Data challenges highlighted during pilot testing
Thank you Ranping Song [email protected]