Developing a National Framework for Monitoring the Iraqi NDP MEASURING GOVERNANCE
description
Transcript of Developing a National Framework for Monitoring the Iraqi NDP MEASURING GOVERNANCE
Developing a National Framework for Monitoring the Iraqi NDP
MEASURING GOVERNANCE
Marie Laberge Global Programme on Country-Led Governance Assessments
UNDP Oslo Governance Centre
Examine the limitations of global composite indicators
Discuss the rationale for using complementary indicators (input vs. Process vs. outcome – de jure vs. de facto)
Identify indicators that are ‘actionable’
Identify 4 ways in which indicators can be made pro-poor and gender sensitive
Identify common sources of governance data
Types of indicators & data sources
Part OneInternational sources of governance indicators
90888682 84 96 98 00 02 0492 94 0678761974
Global composite indicators of governance
0880
CPIA Freedom in the World
Commitment to
Development Bertelsmann Transformation
Index
Global Accountability
Report
Index of Economic Freedom
Journalists killed
Open Budget IndexPolity
International Country Risk
GuideCorruption Perceptions
Index
Opacity IndexIntegrity Index
Human Rights
Indicators
Bribe Payers Index GAPS in
Workers’ Rights
BEEPS
Press Freedom Survey
Political Terror Scale
Global Competitivenes
s Index
World Governance Assessment
World Values Survey
State Failure Dataset
Weberian Comparative State Project
CIRI Human Rights
Databse
Women in Parliament
Governance Matters
Gender Empowermen
t Measure
Index of Democracy
World Democrac
y Audit
Countries at the Crossroads
Failed States Index
Press Freedom
Index
Democracy Index
Institutional Profiles
Database
Governance and
Democracy Processes
Global Peace Index
Transparency International
Freedom House World Bank Governance Indicators
Bertelsmann Foundation
Global Integrity
(Syria, Saudi Arabia not included)
Top JordanSaudi ArabiaKuwaitMorocco
“Partially free”: Jordan, Morocco, Lebanon, Yemen, Kuwait
Kuwait, Jordan, Saudi Arabia, Morocco
Lebanon, Kuwait, Algeria
Jordan, Kuwait, Egypt, Palestine
Bot-tom
LebanonYemen
“Not free”: Algeria, Egypt, Saudi Arabia, Syria Palestine
Syria Yemen Palestine
Syria Saudi Arabia
MoroccoAlgeria Yemen Lebanon
1) What is each index measuring? TI CPI Public sector only WB ‘Control of Corruption’ indicator public &
private sectors Global Integrity Index anti-corruption mechanisms
2) What types of indicators make up each index?
High-level corruption vs. petty corruption Frequency of bribes vs. size of bribes
How to explain these discrepancies in rankings?
Selected sources
(out of 13 in 2009)
Subject asked (based on
perceptions)
Respondents Coverage
World Economic Forum (WEF)
Undocumented extra payments or bribes
connected with various government functions
Senior business leaders; domestic and international
companies
131 countries
Bertelsmann Transformation
Index (BTI)
The government’s capacity to punish
and contain corruption
Network of local experts / local
business people
125 countries
Economist Intelligence Unit
(EIU)
The misuse of public office for private (or political party) gain
International experts
170 countries
3) What data sources are used?
And… 4) Are data sources the same
every year?
TI Corruption Perception index (CPI) 2009 126th Syria 5 surveys
176th Iraq 3 surveys
179th Afghanistan 4 surveys
2008 147th Syria 5 surveys
176th Afghanistan 4 surveys
178th Iraq 4 surveys
2007 138th Syria 4 surveys
172th Afghanistan 4 surveys
178th Iraq 4 surveys
Part twoNational governance indicators
Monitoring the input, process and outcome sides of governance reforms
Outcome indicators will tell us:
What do we want to achieve?Have we achieved it?
Process indicators will tell us:
How are inputs used to generate outcomes?What are the ‘best’ processes?When & how do processes need adjustment?
Input indicators will tell us:
What is needed to create these processes?Whether it is available?
Three levels of indicators
• De jure (input) indicator: In law, is there an agency with a legal mandate to address corruption? Does it receive regular funding, a professional & full-time staff?
Measuring a change in law, in procedures, in resources
• De facto (process / outcome) indicator: In practice, is the anti-corruption agency effective?
Measuring improved governance in practice (how are citizens benefiting from this new institution?)
Complementarity in the use of indicators
Combining indicators to show discrepancies De jure indicator: In law, is there an agency with a legal mandate to address corruption? (input)
De facto indicators: In practice, is the anti-corruption agency effective?
(outcome) • When necessary, is the ACA able to independently
initiate investigations? (process) • Does the ACA make regular public reports (e.g. to the
parliament)? (process) • Can citizens complain to the ACA without fear of
recrimination? (e.g. whistle-blower mechanisms) (process)
• Does the ACA act on citizen complaints within a reasonable time period? (process)
• Outcome indicators are useful to assess progress towards the desired objectives of governance programmes (new laws / increased expenditures mean nothing in and of themselves)
• But they lack ‘actionability’ (they don’t tell us what needs to be fixed / points of interventions)
• Example: A government cannot ‘choose’ to lower a crime rate (an outcome indicator)
• It can, however, choose to put more police on the streets (input) or toughen penalties for offenders (prosecution process)
Input / process indicators are more ‘actionable’
Outcome indicators (non-actionable)
Input / process indicators (actionable)
% citizens who believe there is a lack of transparency in the operations of local governments
Number of provinces which have a public forum for citizens to discuss with locally elected officialsNumber of provinces which formally publish contracts, tenders, local budget and local development plan
% citizens who do not believe that legal protection is ensured equally to all citizens regardless of their material status, ethnic, religious affiliations, political/party affiliations
Number of provinces where an awareness-raising programme on citizens’ rights to seek legal protection has been conducted% citizens who say they know where to / how to seek legal protection
Actionable indicators: Experience from Morocco
“Not everything that counts
can be counted, and
not everything that can be
counted counts.”
– Albert Einstein
A word of caution:
“Not everything that counts can be
counted, and not everything that can be
counted counts.”– Albert Einstein
Measuring is not an end in itself, but rather a means to an end (actual governance reforms).
So what we measure should help us implement reforms – if not, why
measure it?
A word of caution:
• Risk of measuring things because they are easily measurable, leading to ‘reform illusion’
• Example: Measuring the number of corruption cases brought to trial (as an indicator of the efficiency of the judicial system in combating corruption):
• Does an increase in this indicator mean an increased level of confidence in the reporting mechanism, and in the courts?
• Or rather, does it indicate a higher incidence of corruption?
• Or both...?
In other words, are you really measuring what you intended to measure?
A word of caution
Balanced baskets of indicators
Do you have a balanced basket of indicators that measures progress towards a single aim?
Is the ambiguity inherent in each indicator reduced by the presence of the others?
Balanced baskets of indicators
– What is the aim you want to achieve?
– What are the many reasons why your aim might NOT be achieved?
– For each “reason” develop one indicator
Unbalanced baskets of indicators
Aim: “Equal access to justice” Indicator 1: Number of new courts opened in rural and urban areas Indicator 2: Number of courts per 100,000 residents Indicator 3: % citizens who say that they have access to court systems to resolve disputes Indicator 4: % accused not represented at trial
Indicator 1: Number of new courts opened in rural and urban areas Measures the activity of a court-building programme, but does not tell us the result of the programme: do people have their fair share of courts?
Indicator 3: % citizens who say that they have access to court systems to resolve disputes Tells us nothing about equality
Indicator 2: Number of courts per 100,000 residents
Tells us nothing about the equality of the distribution of courts (serving mainly one group / one region?)
Indicator 4: % accused not represented at trial
The basket as a whole is unbalanced: 3 indicators relate to the courts, and one to legal aid
Balanced baskets of indicators
Checklist for building a well-balanced basket of indicators
Avoid narrow activity indicators What is the strategic aim you want to
achieve? Draw one indicator from each
institution/service that contributes to the overall aim
Or...
What are the many reasons why your aim might NOT be achieved?
For each “reason” develop one indicator
Balanced baskets of indicators
4 key services/institutions for achieving “equal access to justice”
4 indicators:1. Court system
2. Legal aid service3. Police service
4. Prosecution service
Indicator 1 – Court system% citizens who say that they have access to court systems, disaggregated by gender, ethnicity, region, etc. Adapted from version 1: Disaggregated to reveal issues of inequality
Indicator 3 – Police service% citizens who say that the police will respond to them without requiring a bribe if called to resolve a dispute, disaggregated by gender, ethnicity, region, etc. Adds balance: Will alert you to problems in police services that may block access to justice more for some groups than for other.
Indicator 2 – Legal aid service% accused persons legally represented at one or more court appearances, disaggregated by gender, ethnicity, region, etc. Adapted from version 1: Disaggregated to reveal issues of inequality
Indicator 4 – Prosecution serviceRatio of prosecution caseloads in courts serving wealthier communities to those in courts serving marginalized communities Adds balance: Will alert you to problems in prosecution services that may block access to justice more for some groups than for other.
How to make governance indicators ‘pro-poor’ and gender sensitive
What makes a governance indicator ‘pro-poor’ or
gender sensitive?
4 ways to make indicators sensitive to vulnerable groups:
1. Disaggregating by poverty/gender2. Specific to the poor/women3. Implicitly poverty/gender
sensitive4. Chosen by the poor/women
1. Disaggregating by poverty/gender (Parliament)
Information is collected for the general population, then disaggregated by sex or income
• % seats held by women in parliament
• Level of confidence among female citizens that the Parliament represents their interests
• % of Parliamentarians from poorer districtsthat have functioning and accessible localoffices to meet with constituents
2. Specific to the poor/women Measuring governance issues that are specific
to the poor or to women
• Size of funds allocated to legal aid in provincial budgets (per capita)
• Number of attorneys as % of citizens in need of one
• % of local governments practicing gender-sensitive budgeting
3. Implicitly poverty/gender sensitive
• Backlog of small cases of little financial value Makes no explicit reference to poverty status or gender, but by its nature, clear that the indicator is of particular relevance to low-income groups / women
• The number of hours that polling booths are open during on election day
• Frequency of engagement of CSOs in consultations on the legislation-making process
4. Chosen by the poor/women
Identified & measured by using participatory techniques (surveys, focus groups, etc.)
• Acceptance of documentation other than birth certificates in the process of voter registration
• Women’s trust in the police and its ability to provide women with redress if they file a complaint
Sources of governance data
• Important to monitor both the ‘supply’ and ‘demand’ for good governance
• Parallel with private sector: – Assessment of the company’s charter, balance
sheet, internal processes and assembly line OR / AND ? – Assessment of whether consumers are actually
buying their product, whether they were satisfied with it, and likely to keep on buying it
Complementarity of data sources
Implications for data sources?
1) Objective data – Qualitative:Administrative sources: Policy and legal documents, codes of conduct, organizational set-up and management systems, processes for decision-makingOther narrative reports: Newspaper reports, reports by CSOs2) Objective data – Quantitative: Government statistics, data gathered by domestic NGOs, international organizations and academics, expenditure tracking and budgetary information, audit reports, court records3) Opinion and experience-based data from citizens, specific groups (MPs), specific institutions, or private sector through surveys, focus groups, etc.
Assessing the ‘supply side’ of governance
Assessing the ‘demand side’ of governance
IHEC has a permanent voter registry with safeguards to prevent fraud and mechanisms for inclusion of all Iraqis in elections
-% of people prevented from registering (experience-based)-% of people prevented from voting because name not on voter list (experience-based)-% of people prevented from voting due to intimidation (experience-based)-% of people who say results accurate reflection of vote-% of people who feel elections produce legislature representative of people-% of people who feel elections enable them to get rid of bad leaders
Assessing the ‘supply side’ of governance
Assessing the ‘demand side’ of governance
Number of criminal codes and criminal procedure codes harmonized with international Human Rights Law
% who say ordinary people can commit crimes without punishment
% who say high level officials can commit crimes without punishment
% who say people fear wrongful arrest
Opportunity to draw from ‘fresh’ survey data
New ‘Arab Democracy Barometer’
Joint initiative by Arab Reform Initiative & Arab Barometer
Nationally representative public surveyIn 10 countries – including IraqData to be collected by country teams (starting autumn 2010)
75 questions
The Arab Democracy Index• First governance index produced by an Arab institution (“The
Arab Reform Initiative”) • Data collected by consortium of Arab research centres &
universities • Covers 10 countries• 40 indicators
Some measure the ‘means’ of democratic transition (legislation) – ‘de jure’ indicators
Some measure the ‘results’ of democratic transition – ‘de facto’ indicators
• Sources of data: Government & non-government Citizen’s impressions (public survey) used for 25%
indicators• 4 themes:
1) Strong & accountable public institutions2) Rule of law3) Respect for rights & freedoms4) Equality & social justice