1

Using Household Surveys as a tool for monitoring Poverty, Governance and

Democracy in Africa and the Andean Region

Javier Herrera Mireille Razafindrakoto

François Roubaud DIAL - IRD

Workshop on Governance Indicators New-Delhi, April, 20-22, 2005

2

The global frameworkThe global framework

Objective Main partners The implementation process

Importance of the dissemination policy The Tool The Mirror Survey

Results / Type of indicators

Desagregated data (by sex, by level of income, level of education, ethnic groups, etc.) Comparable data Geographical data Time-series Multi-dimensional approach

DIALDéveloppement Institutions et Analyses de Long terme

3

To refine measurement tools and to test the capacity of statistical household surveys (1-2-3 survey) to provide relevant information on key aspects of democracy and governance.

Implementation of representative sample surveys on households’ experiences, perceptions and expectations

Methodology : Qualitative modules on Governance and Democracy (=opinion polls) grafted onto conventional household quantitative survey (the 1-2-3 survey on employment, informal sector, consumption and poverty extended to Governance and Democracy issues).

Multi-country approach:• In Africa: 7 capital cities of Western African countries (Benin, Burkina-Faso, Côte d’Ivoire, Mali, Niger, Senegal, Togo), + 7 cities of Madagascar.• In Latin America: 5 countries of the Andean Community (Bolivia, Columbia, Ecuador, Peru and Venezuela

DIALDéveloppement Institutions et Analyses de Long terme

Objective

4

Main partners:• Andean Community General Secretariat and

Statistical offices of the region; • Afristat and Statistical Offices of the region

Surveys are conducted by National Statistical offices DIAL : responsible for coordination and technical assistance

Integration in the National Statistical System ( Institutionalisation can be considered)

Statistical Office’s function: to provide data to help decision-making process In many very poor countries, Statistical office is the most qualified to implement surveys with large sample and with reliable results

Mobilisation of existing tools (Household surveys already implemented) Marginal cost

The global framework

DIALDéveloppement Institutions et Analyses de Long terme

5

Implementation process

Design of the questionnaire is based on more than five years experiences in Madagascar (started in 1995)

First phase in Madagascar : variable thematic modules (adjusted every years)

Test of the relevance of questions included in the survey

Importance of the dissemination policy :Systematic presentation / publication of the result wide audience : public conference, dissemination in the media, etc. (thirst for evidence-based information, for meaningful figures) Reactions validation of the relevance of the questions Gives raise to specific demand from different stakeholders ( partnership with civil society, media)

Second phase at a regional level : the same questionnaire in 7 capital cities in West Africa (test of relevance)

The 1-2-3 household survey was already planned in these countriesThe suggestion of added modules on governance and democracy

met the demand in some countries gave raise to demand in others

Need for evidence, convincing result (to demonstrate the relevance of the approach and the questions addressed) Initiative from countries to reconduct the survey OWNERSHIP

DIALDéveloppement Institutions et Analyses de Long terme

6

The tool : Module grafted onto conventional household survey

Main principles• Light, flexible tool Reconductible Time-series• Transparency, representativeness of the information collected (voicing) • Quantification of the phenomenon.

combination and monitoring over time of 3 types of information: - subjective issues (assessment of the working of democracy and the State, level of support for policies, perception of discrimination, value systems), - objective facts on social practices and behaviour (access to public services, political involvement, incidence of corruption etc.) - conventional socio-economic characteristics (gender, education, level of income, age, migration, employment, income, consumption etc.).

multiple possibilities to break down the information according to different category of households or individuals (by level of income, by sex, by ethnic group, etc.).

DIALDéveloppement Institutions et Analyses de Long terme

7

The mirror survey

To round out the collection of surveys on Governance and Democracy in seven West African capitals and in Madagascar (Antananarivo),

an additional survey to get the opinions of a certain number of Southern and Northern experts (researchers, development practitioners, decision-makers, senior civil

servants, politicians, etc.).

The aim to compare answers from the population surveyed in each country with the experts’ point of view.

8

The mirror survey

Two sets of questions: (on one of the eight countries the expert know the best, his reference)

- what did the experts believe the respondents answered on average. - their own answer to these same questions

(“What is your personal opinion?”)

Questions : - Specificity of the answers of the population / experts, specialists?-- Knowledge of Northern or Southern experts on what happens and on people’s thinking in the South?

Relevance and reliability of indicators based only on appreciation of panel of experts?

9

Some results to illustrate the usefulness of this type of surveys

Results of the “mirror survey”

In general, it seems that experts are more “pessimistic” on how “Africa works” than the african citizens. Answers (values) of the population in the South are less specific than experts think (democratic values etc.)

confirmation that it is interesting to know people’s opinion on reform ownership

10

Cotonou Abidjan Dakar Bamako Lomé Niamey Mean Mirror ExpertSurvey Opinion

EVALUATIONThe administration is well run 46 53 61 53 20 54 48 35 33Faith in the administration 42 58 64 57 34 54 51 38

OBJECTIVE INDICATOR Incidence of corruption (%) 9 17 11 10 10 8 11 55

OPINIONNumber of civil servants too high 5 8 18 9 5 7 9 39 18Number of civil servants too low 73 60 59 66 82 70 69 37 32Wages of civil servants too high 7 7 21 8 11 14 11 39 21Wages of civil servants too low 76 73 63 74 82 61 72 45 60

OPINIONS ON REFORMSLink wages to performance 85 96 98 89 95 94 93 62 77Disciplinary action against substandard workers 84 83 93 84 70 82 83 59 84

In favour of cost-recovering for Health services 80 66 60 65 68 23 60 33 55In favour of cost-recovering in Education syst. 80 65 60 58 65 26 59 34 58

Preliminary results of the mirror survey

11

How far can we trust the experts’ opinion on corruption?An experiment based on surveys in francophone Africa

Percentage BeninBurkina Faso

Ivory Coast

Mada-gascar

Mali

NigerSeneg

alTog

oAverage

Incidence of corruption

General population* 8,7 15,2 16,5 16,3 10,1 8,2 10,8 9,6 13,1

Expert panel (what they believe public would reply) 54,1 35,2 60,7 57,1 52,0 56,1 51,1 62,5 54,0

Believes that making a bribe is acceptable behaviour

General population* 3,6 8,2 5,2 10,5 5,0 3,1 2,2 3,8 5,2

Expert panel (what they believe public would reply) 31,3 28,7 29,2 32,9 33,3 33,8 35,5 21,8 31,5

Corruption is a major problem

General population* 94,2 87,4 91,0 96,9 88,4 91,6 87,9 82,8 90,3

Expert panel (what they believe public would reply) 84,8 67,4 72,3 76,4 67,2 62,3 69,4 84,0 72,9

Expert panel (personal opinion) 96,3 65,0 94,1 88,9 78,1 72,7 80,5 92,3 85,3

Sources: General Household Survey (35,594 persons interviewed; 4500 for each country in average); Expert panel survey (246 persons surveyed; 30 experts for each country in average).

12

How far can we trust the experts’ opinion on corruption?

experts overestimate the extent to which the general population experiences corruption.

Overall, experts hold a far more negative view of reality than the general public.

Not only overestimation: there are significant differences between the two surveys concerning the relative positions of the eight countries.

There is in fact no correlation between the rates of corruption measured by the household surveys and by the mirror survey (-0,19, n.s.)

On the other hand, the expert opinion results drawn from the mirror survey are similar to corruption indicators found in international databases (-0.52 with KKZ “control of corr.”)

13

How far can we trust the experts’ opinion on corruption?

Experts don’t necessarily have a good understanding of the real extent of corruption in the 8 countries surveyed.

The question: what is really being measured in these perception indicators (drawn from experts panel)?

The results do not invalidate the relevance of such indicators since they do capture a common perception linked to corruption phenomena, even if they don’t correspond to the true experiences of corruption.

They should be combined with a new set of indicators based on objective measure and not only on perception in order to understand the full complexity of corruption.

14

Results of the household surveysIncidence and determinants of the petty corruption in West Africa

Sources: 1-2-3 Surveys, PARSTAT, WAEMU 2001-2003 Note : logit model (selection bias correction procedure). ++ (resp. --) : significant coefficient (positive resp. negative) at 5 %. + (resp. -) : idem at 10 %. n.s. : non significant at 5 %).

some robust findings profile of groups that are victims of corruption. -The wealthiest and heads of household appear to be more especially targets of corrupt officials- Youth (the risk decreases with age) and being a foreigner (this is the case in Abidjan) increase vulnerability, other things being equal. - Overall and contrary to the preconceived idea--> ethnic group and religion do not affect the rate of corruption (except very few cases) - Civil servants appear to be less likely to be victims of corruption (civil servant solidarity).

Bénin Côte d’Ivoire Mali Niger Sénégal Togo Incidence (population) 8,7 % 16,5 % 10,3 % 8,1 % 10,9 % 9,1 % No contact with the administration 43,1 % 28,7 % 37,3 % 33,0 % 19,0 % 41,1 % Incidence (have contact with administr) 15,3 % 23,1 % 16,4 % 12,1 % 13,4 % 15,5 % Factors of risk Sexe n.s. -- (women) n.s. n.s. n.s. n.s. Age -- n.s. -- -- n.s. n.s. Statut : head of household n.s. + ++ ++ ++ n.s. Ethnic group/Nationality n.s. -- (Ivoirian) n.s. n.s. -- (Diola) n.s. Religion n.s. -- (catholics) n.s. n.s. n.s. --(catholics Education n.s. n.s. n.s. n.s. n.s. n.s. Income ++ ++ ++ + ++ n.s. Civil servants -- - -- - - n.s. Sample 6 330 4 760 4 530 6 330 6 590 1 900 Paires concordantes 71 % 63 % 65 % 70 % 65 % 67 %

15

Incidence et determinants of the petty corruption in West Africa

Sources: 1-2-3 Surveys, PARSTAT, WAEMU 2001-2003

Poorest quartile (in terms of income) is less victim of corruption

The annual total amount paid by households (victims): 16% of the income of the poorest quartile of the population 1% of the income of the wealthiest quartile

Total Fonctionnaires Quartiles de revenus par tête 1er quartile 2ème quartile 3ème quartile 4ème quartile Incidence (population totale) 8,2 14,1 5,2 6,6 9,7 11,4 Sans contact avec l’administration 32,9 9,2 40,2 39,1 29,8 21,3 Incidence (population touchée) 12,2 15,5 8,7 10,9 13,9 14,5

Montant (1 000 FCFA/an) Montant moyen (ménages touchés) 41,6 21,2 49,3 48,8 33,7 38,7 Montant médian (ménages touchés) 5,0 5,0 10,0 5,0 5,0 5,0 % du revenu (ménages touchés) 2,2 0,6 15,8 6 ,0 2,6 0,9 % du revenu (total ménages) 0,5 0,2 2,6 1,1 0,5 0,3 Incidence (ménages touchés) 23,4 26,0 19,9 21,4 23,5 28,7

Incidence and cost of corruption in Niamey

16

Public wages and corruption in Madagascar 1995-2001 Measure of the incidence of corruption (objective extent of this phenomenon : the proportion of inhabitants of the capital who have fallen victim to corruption during the past year (small-scale corruption))

The measurement of this type of indicator is quite exceptional in developing countries. quantifies the problem & spurs on the authorities to take steps to deal with it. Beyond monitoring over time (since 2000 these indicators have been included every year

in official statistical surveys) improves understanding of the phenomenon. Figure illustrates the link between improving civil servant wages and a sharp drop in the incidence of corruption between 1995 and 2001.

Source :Source: Razafindrakoto and Roubaud (2001) based on employment surveys 1995-2001, MADIO/INSTAT.

0

10

20

30

40

50

1995 1996 1997 1998 1999 2000 2001

%

90

100

110

120

130

140

150

160

Wag

e in

dex

(199

5=10

0)

Corruption Civil Servants real Wages (right scale) Inflation Pluri-activity

17

For more information

DIAL website

www.dial.prd.fr

18

ANNEXES

19

Preliminary results of the mirror survey

Mirror Expert

Cotonou Abidjan Dakar Bamako Lomé Niamey Mean Survey Opinion

SYSTEM OF VALUESIn Favour of democracy 95 94 91 76 91 71 86 83Freedom of expression : fundamental 99 98 97 92 95 91 95 84 95

Importance of work 100 99 99 99 100 99 99 83Importance of leisure 85 77 81 79 84 65 79 50Importance of politics 52 50 63 61 64 56 58 42

Judgement on not paying taxes = acceptable 3 3 2 4 5 2 3 36 on accepting bribe 4 5 2 5 4 3 4 29 joining “new religion” 7 10 14 9 19 5 11 37 consulting a witchdoctor (or equivalent) 12 14 38 24 12 51 25 60 Spending ostentatious sums on ceremonies 6 15 5 11 11 13 10 54

Proud to belong to the country 80 78 56* 68 75 83 64 64

For Senegal, 3 modalities (country/ethnic group/religion, sect) instead of 2 like in the other countries (country/ethnic group)Sources : 1-2-3 Survey 2001/02, PARSTAT. Representative sample of adults aged 18 and over.Mirror survey : 106 individuals (32% on Abidjan, 17% Cotonou, 15% Dakar, 11% Ouagadougou, 10% Bamako)

A priori, the expert’s origine (Northern/Southern) is not a discriminatory factor explaining answers for the mirror surveyDifferences / ‘mistakes’ concerning experts’ appreciation confirmed when results are compared country by country.