LEARNING PROGRAMME Questionnaire design as related to analysis Intermediate Training in Quantitative...

LEARNING PROGRAMME

Questionnaire design Questionnaire design as related to analysisas related to analysis

Intermediate Training in Intermediate Training in Quantitative Analysis Quantitative Analysis

Bangkok 19-23 November 2007Bangkok 19-23 November 2007

LEARNING PROGRAMME - 2

Objectives Understand the implications of questionnaire

design on the analysis Illustrate examples and detect shortcomings

of questions in different questionnairesShare experience by participants in their

surveys


General remarks

First think about the objective of the study and the analysis ( what and how do you want to know) – then the questionnaire

Difference between kind of survey CFSVA and EFSA CFSVA- provides baseline information that can feed into

monitoring systems, not emergency related information, has more time to be collected and analysed;

EFS(N)A- needs results within short period, 10-15 days vs. 4 weeks plus.


General remarksPartner involvement

Partners are important to have a wide buy-in into the results, synergy effects, cost sharing etc. but also might add or change type of information that is collected beyond the need of WFP.

Quality of collected data: Length of the questionnaire - shorter is usually

better- if you don’t sacrifice important details. Sacrifice details that are not analysed to avoid

response fatigue


General remarks cont. Language and translation

Differences between original and translated version (timing) Do the questions in the original language mean the same thing

as the working language? If not- the analyst can mis-interpret the results.

Number of categories for responses used in the questions. Recode later or maintain the same categories? Present graph/table with many categories

Homogeneity of Numbering of questions (letters or number)


Open vs closed question Open question

can be answered with either a single word or a short phrase.

Closed questionCan be answered with one of the categories/options included in the questionex. What is the major material of the roof?

Observe and record. Do not ask question! Circle one1Straw / thatch2Earth / mud3Concrete4Tiles 5CGI sheet6Other, specify ____________________


Open question When to use an open question

Names (household head, village, unit measure) Other (when the category is not included in the question) Community / focus groups questionnaire Small survey (max 50 households )

When not to use an open question When we have an exhaustive list of categories (crops,

livelihoods) ‘Other’ should not be used as alternative to a category

(ex. in Sudan 23% of the pop answer ‘other’ to livelihood activities and we were not able to specify what ‘other’ meant)


Closed questions

When to use a closed question When we know the categories - especially for

question related with materials (roof, floor) and questions related with the context (ex. crops, livelihood activities etc.).

When we are not interested in a continuous variable (ex. age) and we want to collect it in categorical variable.


Example


Continuous vs categorical

When to recode a continuous variable to minimize errors

Demography of the households Land size Stocks and agricultural production Other?


When to use a continuous variable

FCS (1 to 7)Number of animalsProportions (proportional piling)ExpenditureMeasurement (height, weight, muac, child

age in months)


Yes/no questions

Coding of a yes/ no question It is helpful to code yes=1 and no =0, this allows

the analysis to check the % of yes or no running a simple mean.

Have electricity-- for wealth index

18976 77.4 80.1 80.1

4702 19.2 19.9 100.0

23678 96.5 100.0

849 3.5

24527 100.0

No

Yes

Total

Valid

SystemMissing

Total

Frequency Percent Valid PercentCumulative

Percent

Descriptive Statistics

23678 .1986

23678

Have electricity--for wealth index

Valid N (listwise)

N Mean


Skip rule

It is important that the skips for the questions are correct, if not the analyst will have problem in deciding which is the right variable and in the majority of the case he can not use both of them.


Skip rule- example


Missing values

How to recode missing values Difference between missing and not applicable.

Be sure that you know the difference in the analysis!

Negative coding For values as expenditure or income, the value 999 or

888 can be a real value. In these cases might be better to code the missing or not applicable as a negative number -999


Household ID

Importance of HHID in linking one module with different modules of the questionnaire and with different questionnaires ex. household /child / mother, household / village

Importance of the coding (village, cluster, state, community)

It always has to be unique!


ID-Section – Darfur


Household ID - Exercise

What are the important elements? How can we ensure this part is done correctly?


Modules

Now we are going to see some examples of modules of the questionnaire and how they are linked with the analysis


Demographics – example of indicators

Average size of household Number of educated people in an household Incidence of absenteeism amongst school-going

children; enrolment ratio, drop-out Literacy of household heads Percentage of male, female and children-headed

households Percentage of disabled/chronically ill in the

households Dependency rate


Demographic - issues

Age as continuous or categorical variables? Age categories should be related to standards

School age, productive members, children, etc.

Polygamy / number of wives Household size (1.1 & 1.7 should be the same) Number of categories in the education level Education of the mother of children as opposed to

simply spouse’s education


Housing – example of indicators

Crowding (how many people sleep in the house)

Most common building materials used in housing (of floors, roofs and walls)

Availability of toilet facilities and typeSource of lighting, cooking fuel and waterWealth index


Housing example

What is the major material of theroof - other specified

2983 99.9 99.9 99.9

1 .0 .0 99.9

2 .1 .1 100.0

1 .0 .0 100.0

2987 100.0 100.0

OLD IRON

PAPYRUS

TURPULIN

Total

Valid


Percent

What is the major construction material of the outside walls - other specified

2958 99.0 99.0 99.0

1 .0 .0 99.1

2 .1 .1 99.1

1 .0 .0 99.1

2 .1 .1 99.2

2 .1 .1 99.3

2 .1 .1 99.4

2 .1 .1 99.5

0 .0 .0 99.5

4 .1 .1 99.6

1 .0 .0 99.6

1 .0 .0 99.7

0 .0 .0 99.7

1 .0 .0 99.7

0 .0 .0 99.7

0 .0 .0 99.8

2 .1 .1 99.8

1 .0 .0 99.9

1 .0 .0 99.9

1 .0 .0 99.9

1 .0 .0 99.9

1 .0 .0 100.0

1 .0 .0 100.0

2987 100.0 100.0

BRICKS

BURNED B

CEMENT W

GRASS

GRASS HU

GRASS SH

grass, p

IRON SHE

MUD

MUD, STI

MUD/BURN

NOT BURN

PAPYRUS

STONES

STONES &

STRAW

STRAW ON

STRW ONL

TURPULIN

UN BURNE

UNBURNED

WOOD

Total

Valid


Percent

From Uganda database


Housing - issues Pilot testing the questionnaire – it’s useful to

explore possible answers to a question After the pilot the possible answers are included

with codes, so that the ‘other’ will not be as necessary

recode the meaning of “other” when you have a lot of them (when the enumerator has entered in a string response)

Exclude the possibility of other for material questions (ex. Housing)


Housing - issues

The number of digits should be limited for any figure through boxes |_| (helps in data entry and cleaning)

Distance in km or minutes? To water source, market, school, health centre -

HH vs. community? One way vs round trip, waiting time and means of

transport


Agricultural – example of indicators

Percentage of households having access to land

Most common types / methods of land access

Common crops cultivated and amount Source of seedsPeople involved in agricultural activitiesStocks and agriculture production


Agriculture - issues

Units of land measurement Acres, hectares, parcels, etc.

Land size in absolute value or in categories? What is more relevant in the analysis: the mean land size or the division in categories?

Mis-leading cash crop definition


Income – example of indicators

Income diversificationThe most common activities Average contribution of each of the income

generating activities to a household’s income


Expenditure – example of indicators

The most common expenditure items- food & non-food

The average monthly expenditure of a household or per capita for each of the above items

Food /non food expenditure quintiles Proportion of food expenditure versus non

food expenditure


Income and expenditure – issues

Proportional piling (100%) Income in absolute real value or express in

categories The number of digits should be limited for any figure

through boxes |_| for data cleaning and entry Different recall period for expenditures are often

used- so this means it’s necessary to carefully calculate the monthly expenditure values in the analysis.


Food consumption – example of indicators

Average number of meals an adult and a child ate the previous day

Diet Diversity and Food FrequencyFood consumption profilesSource of foods


Food consumption – issues

Collection of gender disaggregated data (meals per day)

Specify the child age range (infant vs children)

Don’t consider 0 if the household has no children

Rank the sources of food (main and second)


Maternal health and nutrition – example of indicators

Percentage of households with children aged between 6 – 59 months

Malnutrition indicators for: children (waz, haz, whz) Mother ( bmi)

Incidence of miscarriages / still-births (averaged for the sample) Percentage of mothers who breast-fed their children Information on prenatal and antenatal care available and used by

mothers Information on incidence and treatment of diseases such as

malaria, diarrhea, fever, cholera, measles, cough etc Information on prevalent hygienic practices followed


Maternal and child - issues

Link mother with child databaseDate of birth – local calendar Fever and diarrhoea separate question Child size at birth, continuous or

categorical? (subjective) Mosquito net only for the mother or even

for the child?


Conclusions Data analysts must participate in the design of the

questionnaire to avoid difficulties or missing information in the analysis Even if PDAs are used, the analyst should carefully examine

all the skip rules to be sure the correct information will be collected

The questionnaire designers and enumerator trainers should be involved in the analysis (if the analyst him/herself was not) to be sure the questions are understood by the analyst.

Information should be collected in order to calculate key indicators during analysis- questions that are not necessary in the analysis should not be included.

LEARNING PROGRAMME Questionnaire design as related to analysis Intermediate Training in Quantitative...

Documents

Transcript of LEARNING PROGRAMME Questionnaire design as related to analysis Intermediate Training in Quantitative...