UNFCCC Meeting on experiences with performance indicators for monitoring and evaluation of capacity building in

developing countries

Rio De Janeiro, Brazil 6-7 November, 2008

Jamil H.Chowdhury

Bangladesh M & E Network

Dhaka, Bangladesh

Thing to be remember 1

“ In the evaluation of capacity development the impact metaphor should be avoided. The militaristic impact metaphor fails to capture the essential features of capacity development which is a process of change and growth.”

(Horton, D. 2002)

Common confronting question in M & E of capacity building2

Has the capacity building been well defined?

Are the widespread examples of M & E of capacity building?

What tools we should use for accessing national capacity in a integrated way?

Do we have any standardized approaches to monitoring and evaluating capacity building interventions?

Do we have standardized logical framework or gold standard for assessing country capacity?

Are we clear about the relationship between capacity and performance i.e. what elements of capacity are critical to performance?

Do we understand the difficulty in quantifying many elements of capacity before attributing capacity change to any single intervention?

Can we separate capacity measurement from the process of building capacity itself?

3

Identifying existing needs and gaps in the process of implementation.Understanding the status of implementation of multilateral decisions.Identifying needs for technical and financial support.Assessing prioritization of climate change issue in national and local planning and decision making.Assessing arrangement for mainstreaming climate change related capacity building within the context of wider sustainable development.Determine the level of country commitment and ownership.

Rationale for M & E of capacity building

Issues of consideration for evaluating country capacity 4

Country level performance monitoring for assessing capacity may be a preferred choice and easy to implement on a regular basis.Evaluating a country capacity is complex and methodologically challenging.Evaluation must be done for specific desired changes or outcome variables in performance with an interval of every three-five years.Capacity evaluation tend to get mix up with performance measurement.Capacity indicators must be well identified and well defined before they are evaluated.Quality of capacity building remain usually unaddressed.Measuring cost of capacity building at any point of time remains a great challenge.

Suggested indicators for assessing country capacity

There are always unlimited range of national capacity that may

include: Level of national response or political commitment. Policy and regulatory changes. Budgetary provisions for capacity building. Ability and level of procurement of external

resources for enhancing capacity. Building organization or strengthening institutions

both at public and private sector. Capacity of vulnerability and adaptation assessment

and implementation of mitigating options.

5

Suggested indicators of country capacity (contd..) 6

Functional availability of climate change secretariat and focal points.

Coverage of climate change issues by the leading news papers.

Institutional arrangement for technology transfer. Level of implementations of adaptations measures. Level of interministerial or multi sector co-

ordination to address climate change issues. Type of bi-lateral and multilateral agreement

signed for strengthening capacity. Availability of baseline data on current

capacity/capacity needs assessment for monitoring future progress.

Suggested indicators of country capacity (contd..) 7

Instances of developing/strengthening research and training institutions.

Level of involvement of civil society, academician and other stakeholders.

Availability of national plan of action and national strategies (inclusions of climate change agenda in the national development plan).

Plan for development of technical human resources (metrological, hydrological land, water & forest management).

Training & awareness of print and electronic media journalists for expanded advocacy.

Common barriers to measurement of capacity8

Capacity building is rarely considered an important management issue and always seen in isolation.

Most people seem uncertain as to what, when and how to monitor when it comes to capacity building.

Absence of baseline data to compare the level of progress.

High staff turnover and repeated transfer of technical/ skilled human resources especially at the public sector.

Lack of availability of quality data or failure to identify data sources.

Lack of proper documentation of best practices and lessons learnt.

Suggested methodologies 9

Combine qualitative and quantitative methods.

Review national and policy and plan documents.

Review organizational records/documents (if available)

Interview key professionals/focal points or conduct focus groups with them.

Review research reports/secondary data.

Review leading newspaper reports for last 1 year for monitoring change and media coverage.

Suggested methodologies (contd…)10

Use scoring system for relevant indicators.

Engage external evaluators.

Consider participatory self assessment methods.

Make separate capacity assessment of organizations/institutions as part of assessment of national capacity.

Try Appreciative Inquiry (AI) for relevant organizations/institutions for identifying strengths.

The way forward11

For appropriate methods of M & E of capacity building

we need to: Assess the current level of capacity. Identify the gaps in performance and capacity. Map what inputs are available to support the

current capacity. Determine which interventions will address the

gaps in capacity and performance. Define performance objectives and determine

the outcome measures to evaluate progress.

The way forward (contd…..)12

Plan long term financial support to develop capacity building / training organizations.

Plan long term support for capacity building of research organizations for systematic research and monitoring of climate change issues.

Develop a time frame for periodic evaluation of outcome.

Example of a success story13

Best known example of capacity building is in the EPI programme in the Bangladesh health sector.Reasons for success

It started as a project with long time continued support from UNICEF, WHO & other few INGOsThe project was mainstreamed into the health system after 5-7 years.The external and technical support continued even after integration into the system.Dynamic leadership at the initial stages brought success.Strong BCC campaign reinforced the effort.Successful community involvement and strong community support helped sustain the mainstream.

Example of a failure case

Bangladesh bureau of statistics has been generating data on

various development indicators for more than 30 years, but

with little institutional capacity attainment.

Reasons for failure It failed to produce quality data to satisfy national &

international users. Lack of institutional leadership and ownership at any point

of time (the head of the institutions frequently changed). Lack of long time technical support from international

agencies. Lack of accountability and good governance. Misdirected energy and waste of resource without

institutional capacity building goal. Lack of institutional commitment.

14

End