1

Metagora:

Current Progress and the Way Forward

PARIS21 Steering CommitteeParis, 13 November 2007

2

A WIDESPREAD NEED FOR EVIDENCE-BASED

ASSESSMENT EMERGING WORLDWIDE

Huge demand for robust data and indicators

Many initiatives in different regions of the world were identified through a global survey and are being documented in the Metagora inventory, available on-line in: www.metagora.org.

Global initiatives to build indicators (WBI, UN-HCHR, etc.) Regional initiatives to set-up evidence-based monitoring

of democracy and governance National and local initiatives to measure specific human

rights and democratic governance issues.

3

4

CAN HUMAN RIGHTS AND DEMOCRATIC GOVERNANCE BE MEASURED?

By whom?

With which methods?

Under which conditions?

For which purposes?

5

through several pilot national experiences carried out in different regions of the world in an interactive fashion.

These pilot experiences were willingly selected to address sensitive issues in real difficult environments and in different political, social and cultural contexts.

METAGORA FORMULATES A RESPONSE

6

A Decentralized Laboratory

Content

7

METHOD OF WORK

A bottom-up approach consisting of:

identifying with stakeholders national key issues for which evidence-based assessment could be policy relevant;

applying statistical methods adapted to the particular context;

assessing these methods for their capacity to provide policy-relevant results;

providing stakeholders with a shared knowledge on the policy issues at stake;

contributing to draw global lessons from the local pilot experiences.

8

RESPONDING TO BASIC QUESTIONS

Can multidimensional human rights and democratic governance issues be measured through surveys?

Will people respond to sensitive questions? Will the gathered information be statistically

significant and politically relevant? Can official statistical agencies be involved? How to build rights-based indicators?

9

LESSON 1

Measuring human rights and democratic governance is technically feasible and politically relevant.

Sensitive data on human rights, democracy and governance can be collected and analysed using statistical tools.

10

Example: measuring irregularities, abuse of power and ill-treatment in Mexico City (Federal District)

Persons withoutabuse47 %

Persons with contact

24 %

Personswith

abuse53 %

Persons with non-physicalabuse93 %

Persons with physical

ill-treatment7 %

Incidence of contact with public security and procurement of justice authorities

Incidence ofabuse

Type ofabuse

Target population:persons aged 15 or more living in the Federal District

(6,400,000 persons)Reference period:

events occurred between November 2003 and October 2004

Measuring method:random sample household survey,

conducted through face-to-face interviews.Persons withoutcontact76 %

11

Example: Non-physical abuse in contacts with law enforcement authorities

(Survey results correspond to 2,300,000 contacts experienced by 1,520,000 persons)

34 000

72 000

114 000

120 000

163 000

248 000

667 000

Were you threatened with hurt to your family?

Were you compelled to confess?

Were you threatened to be hurt?

Were you threatened in order to obtain a confession or some information?

Were you threatened with accusation on false grounds?

Were you insulted or humiliated?

Were you asked for money?

(Number of contacts per type of non-physical abuse)

12

LESSON 2

On the basis of this information, indicators can be produced that are relevant and useful for political decision and action.

13

Example: incidence of corruption within public agencies in Peru

0

5

10

15

20

25

30

35

40

Po

lice

Ju

dic

iary

Min

. o

f A

gri

cu

ltu

re

Dir

. o

f M

igra

tio

ns

Mu

nic

ipa

l a

uth

ori

ty

Offic

e o

f A

rbitra

tio

n

an

d C

on

cilia

tio

n

FO

NC

OD

ES

ON

PE

JN

E

RE

NIE

C

%

National

Urban

Rural

Poor

Non poor

14

Example: levels of corruption and civil servants’ wages in Antananarivo, Madagascar

0

5

10

15

20

25

30

35

40

45

1995 1996 1997 1998 1999 2000 2001 2002 2004

popu

latio

n vi

ctim

of c

orru

ptio

n

90

100

110

120

130

140

150

160

170

180

base 100 = 1995%

Level of corruption Civil servants real w ages (right scale)

15

LESSON 4

Official Statistical Agencies can be efficiently involved in evidence-based assessment of human rights and democratic governance:

they can conduct surveys and produce relevant information, OR

they can provide advice and technical support to HRIs, governmental bodies and SCOs, OR

they can develop monitoring tools and mechanisms in collaboration and interaction with the civil society.

16

Example: comparing expectations for and respect of democratic principles in French speaking Africa

0

20

40

60

80

100

%

Cotonou Ouaga Abidjan Bamako Niamey Dakar Lomé Tana

The six principles of democracy are:

Fundamental Respected

17

Example: how important is democracy for people in Ecuador?

0

10

20

30

40

50

60

70

no education primary school secondaryschool

highereducation

%

very important

important

little important

not at all important

don't know

18

Example: measuring dimensions of democracy: are they fundamental? Are they respected?

Lomé

020406080

100

Free, transparentelections

Political freedom (rightto choose party)

Freedom of expression,of the press

Equality before the law

Freedom to travel

Religious freedom

Fundamental

Respected

19

VALUE ADDED BY OFFICIAL STATISTICS

Data quality

Robustness of tools

Professional standards

Analytical potential

Time series

Inter-relational indicators

20

TWO MAIN CRITERIA FOR DECIDING ON INVOLVING OR NOT OFFICIAL STATISTICS

LEGITIMACY

CAPACITY

Measuring human rights and/or democratic

governance MUST NOT endanger the

development of the national statistical system.

21

COMPARING OFFICIAL STATISTICS WITHEXPERTS’ VIEWS

Appended Module to Regular Household Surveys conducted by National Statistical Offices in French Speaking Africa (35,600 persons interviewed; 4,500 for each capital city in average);*

Expert panel survey (“Mirror survey”) conducted by DIAL-France (246 persons surveyed; 30 experts for each country in average).

* In Madagascar, results are drawn from the 2003 survey.

22

How far can we trust experts’ opinion ?

0

10

20

30

40

50

60

70

%

General population (% of victimsof corruption from householdsurveys)

Expert panel (what they believecould be the percentage ofvictims of corruption)

23

How far can we trust experts’ opinion ?

0

5

10

15

20

25

30

35

40%

General population (% whobelieves that making a bribe isacceptable)

Expert panel (what they thinkcould be the % of population whobelieves that giving a bribe isacceptable)

24

ASSESSING ACHIEVEMENTS AND DRAWING

LESSONS

An independent panel of senior experts is monitoring the process and assessing the outcomes of the Metagora pilot experiences.

Experiences, problems encountered and lessons learned were documented in the form of on-line training materials aimed at facilitating the replication and extension of the pilot experiences in other countries and other contexts.

25

26

METAGORA II (2007 – 2010) : building on the lessons of the pilot phase

Consolidating the working methods

Replicating and extending the pilot experiences

Enhancing the policy impact of the activities

Further enriching the methodology and tool box

Extending operations to developed countries

Handing over the lessons and tools to the international community

27

Warm Thanks for Your Attention !