SAMPLING:REQUIREMENTS OF A GOOD SAMPLE

SELECTION BIAS

MEASUREMENT BIAS

SAMPLING CONTROVERSY

QUESTIONNAIRE DESIGN

REQUIREMENTS OF A GOOD SAMPLE

Will reproduce characteristics of interest in the population as closely as possible

Representative: each sampled unit will represent the characteristics of a known number of units in the population

DEFINITION OF TERMS

•Observation Unit: an object on which a measurement is taken sometimes called an Element•Target Population: The complete collection of observations we want to study•Sample: a subset of the population•Sampled Population: The collection of all possible observation units that might have been chosen in a sample; the population from which a sample was taken

DEFINITION OF TERMS

Sampling Unit: The unit we actually sample e.g households

Sampling Frame: The list of sampling units

Selection Bias;

•Occurs when some part of the target population is not in the sampled population. A good sample will be as free from selection bias as possible.

EXAMPLES OF SELECTION BIAS

Sample of Convenience; it is based on the elements that are available to participate in a study. It is often biased since the units that are easy to select or are likely to respond are usually not representative of the harder to select or nonresponding units.Using a sample-selection procedure that, unknown to the investigators, depends on some characteristic associated with the properties of interest.

EXAMPLES OF SELECTION BIAS cont’d

Judgment Sample: the investigator uses her or his judgment to select the specific units to be included in the sample. The choice of the sample is normally intended to be ‘representative’

Misspecifying the target population; e.g when the target population for the polls is the registered voters who participated in previous elections but are used to predict the outcome of the current elections.

EXAMPLES OF SELECTION BIAS cont’d

Under Coverage: failing to include all the target population in the sampling frameSubstituting a convenient member of a population fro a designated member who is not readily available.Non Response: failing to obtain adequate number of responses from the chosen sample.Allowing Samples to consist entirely of volunteers, such is the case with radio and television call in polls, and the statistics cannot be trusted.

WHAT GOOD ARE SAMPLES WITH SELCTION BIAS?

Though bias, purposive or judgment samples can provide valuable information particularly in the early stages of the investigation.

MEASUREMENT BIAS

This occurs when the measuring instrument has a tendency to differ from the true value in one direction. As with selection bias it must be considered and minimized at the design stage of the survey.

Unavoidable Instances of Measurement Bias

People sometimes do not tell the truth

People do not always understand the questions

People Forget

People give different answers to different interviewers

People may say what they think the interviewer wants to hear or what they think will impress the interviewer

Unavoidable Instances of Measurement Bias cont’d

A particular interviewer may affect the accuracy of the response by misreading questions, recording responses inaccurately, or antagonizing the respondents. Certain words mean different things to different people, e.g. ‘do you own a car? may be answered yes or no depending on the respondent’s interpretation of you ( household or individual)?Question wording and order have a large impact on the responses obtained.

A SAMPLE CONTROVERSY

From Shere Hite’s book Women and Love: A Cultural Revolution in Progress: Some quoted Results;

Quoted Results84% of women are ‘not satisfied emotionally with their relationships’(p804)70% of all women ‘married five or more years are having sex outside of their marriages’ (p856)95% of women’ report forms of emotional and psychological harassment from men with whom they are in love relationships’ (p810)84% of women report condescension from the men in their love relationships (p809)

Criticism of Hite’s ReportThe Sample was self selected-i.e. Recipients of the questionnaire decided whether they would in the sample or not. Hite mailed 100 000 questionnaires; only 4.5% of these were returnedThe questionnaire mailed to organizations such as professional women’s groups, counseling centers, church societies and senior citizens’ centers.The survey had 127 essay questions and most of the questions have several parts, who will tend to return such a survey

Criticism of Hite’s Report cont’d

Many of the questions are vague using words such as love; love maybe interpreted in many different ways by different people.

Many of the questions are leading- they suggest to the respondents which response she should make e.g. ‘does your husband/lover see you as an equal? etc.

Criticism of Hite’s Report cont’d

The final sample is not representative of women in the U.S; and the statistics can only be used to describe the women who would have responded to the survey

Hite claim that results could be generalized because the characteristics like age, education, occupational profiles of women in sample matched those of the U.S

QUESTIONNAIRE DESIGN

Decide what you want to find out; very important step in the writing of a questionnaire: write down the goals of your survey and be precise, then write or select questions that will elicit answers to the research questions and that will encourage persons in the sample to respond to the questions.

Questionnaire Design cont’d

Always test your questions before taking the survey; ideally tested on a small sample of the members of the target population

Keep it simple and clear; questions that seem clear to you as the researcher might not be clear to the respondent being interviewed over the phone or respondent with a different native language

Questionnaire Design cont’dUse specific questions instead of general ones if possible;Relate your questions to the concept of interest; In some disciplines a standard set of questions has been developed and tested, and these are then used by subsequent researchers.Decide Whether to use Open or Closed QuestionsReport the Actual Question Asked; Avoid Questions the Prompt or Motivate the Respondent to say what you would like to hear Use Forced Choice, Rather than Agree/Disagree, Questions: Some people will agree with everything

Questionnaire Design cont’d

Ask Only one Concept in Each Question; in particular avoid Double-Barreled Questions

Pay Attention to Question-Order effects