Page 1

Chapter 14: Data Collection in the Field, Nonresponse Error, and Questionnaire Screening

Chapter 14

DATA COLLECTION IN THE FIELD, NONRESPONSE ERROR,

AND QUESTIONNAIRE SCREENING

GENERAL CONCEPT MULTIPLE CHOICE QUESTIONS

1.In an effort to increase the response rate in surveys, researchers sometimes have interviewers say "I am not selling anything" at the beginning of the conversation. Some research on this has concluded that including this statement: a. does increase response rates by about 2 percentb. does increase response rates by about 10 percentc. has no effect on response ratesd. actually lowered response rates by about 2 percente. actually lowered response rates by about 10 percentAnswer: (a) Difficulty: (Difficult) Page: 3912.Nonsampling error includes:

a. all types of nonresponse error

b. data gathering and handling error

c. data analysisd. data interpretation error

e. All of the above

Answer: (e) Difficulty: (Moderate) Page: 3923.Which is true about nonsampling error?a. it cannot be measured as can sampling error

b. it can be measured as accurately as sampling error

c. it can be measured more accurately than sampling error

d. it cannot be measured in standard units

e. it can be measured in standard unitsAnswer: (a) Difficulty: (Moderate) Page: 3924. Which of the following best represents what should be done about nonsampling error?a. Because it cannot be measured, greater attempts must be made to measure it.b. Because it cannot be measured, researchers must know the sources of nonsampling error and learn how to minimize it.c. Researchers must be aware that nonsampling error exists but because it exists only in a theoretical sense, there are no known safeguards to minimize it.d. Nonsampling error should be measured and controlled at all stages of the research process in order to ensure a 100 percent accurate study.e. Nonsampling error should be measured and controlled at all stages of the research process in order to ensure a 95 percent accurate study.Answer: (b) Difficulty: (Difficult) Page: 392

5.A variety of nonsampling error occurs during data collection. These data collection errors may be divided into:

a. fieldworker error and respondent error

b. data error in; data error out

c. intentional and fieldworker

d. unintentional and respondent error

e. directional and nondirectional nonsampling errors

Answer: (a) Difficulty: (Easy) Page: 3926.Which of the following is a type of intentional fieldworker error?

a. falsehoods

b. fatigue

c. nonresponse

d. cheating

e. misunderstandings

Answer: (d) Difficulty: (Easy) Page: 3937.Interviewer cheating may actually be encouraged by a compensation system that:

a. pays interviewers per completed interview

b. pays interviewers more than the competitive rate

c. pays interviewers by the hour

d. pays interviewers the competitive rate

e. pays interviewers above minimum wage

Answer: (a) Difficulty: (Easy) Page: 393

8.An interviewer gives the respondent very subtle cues such as saying "yes" to statements with which the interviewer agrees. This is an example of:

a. "yes-saying"b. unintentional respondent fraud

c. leading the respondent

d. interviewer misunderstanding

e. misunderstandings

Answer: (c) Difficulty: (Moderate) Page: 3949.An interviewer's accent, sex, and demeanor of the interviewer may be considered:

a. important for hiring purposes but have no effect on the respondent

b. are not to be considered in hiring interviewers due to equal employment opportunity laws

c. personal characteristics that can be a source of bias

d. personal characteristics that the researcher may not consider due to the Interviewer Code of Responsible Behavior Act

e. important only in the training process

Answer: (c) Difficulty: (Moderate) Page: 39510.Interviewer misunderstanding would be considered:

a. intentional interviewer error

b. unintentional interviewer error

c. intentional fieldworker error

d. unintentional respondent error

e. intentional respondent error

Answer: (b) Difficulty: (Moderate) Page: 395

11. Which of the following were mentioned in your textbook as being characteristics of outstanding telephone interviewers?

a. work orientation

b. team players

c. affinity

d. empathy

e. All of the above were mentioned.Answer: (e) Difficulty: (Easy) Page: 396

12. Falsehoods occur when:

a. respondents fail to tell the truth in surveys

b. fieldworkers fail to tell the truth in surveys

c. fieldworkers become fatigued

d. "e," allowable margin of error is "too low"

e. "e," allowable margin of error is "too high"Answer: (a) Difficulty: (Easy) Page: 39613.As stated in your text, some believe that nonresponse is a problem because:

a. companies have reduced monetary incentives

b. distaste for survey participation, busy schedules, and desire for privacy

c. a more withdrawn public, involvement in night TV and the Internet

d. the newly adopted response rate formula adopted by the Marketing Research Society is much more conservative and yields lower rates than the former formulae. None of the above; response rates are actually increasing dramatically as professional researchers have a better knowledge of what motivates potential respondents.Answer: (b) Difficulty: (Difficult) Page: 39714. Marketing researchers in the UK studied different types of potential respondents and labeled them as "Easy to Get" and "Hard to Get," among others. The findings of their research shows that "Hard to Gets" are _________ and ________ than "Easy to Gets."

a. taller; weigh more

b. taller; are more talkative

c. younger; taller

d. younger; less likely to be employed

e. younger; more likely to be employed

Answer: (e) Difficulty: (Difficult) Page: 399

15.An unintentional respondent error occurs when a respondent's interest in the survey wanes, and this is known as:

a. ADD

b. attention loss

c. attention sampling error

d. misunderstanding

e. leading behaviorAnswer: (b) Difficulty: (Easy) Page: 39916.Interviewer cheating may be controlled by:

a. hiring only interviewers who are honest

b. using one-way mirrors

c. supervision and validation

d. ensuring anonymity and confidentiality

e. None of the above; cheating cannot be controlled.

Answer: (c) Difficulty: (Moderate) Page: 40017.When it comes to validation, an industry standard is to randomly select ________ of respondents for call back.

a. 50 percentb. 75 percentc. 33.3 percentd. 25 percente. 10 percentAnswer: (e) Difficulty: (Difficult) Page: 40118.To help control interviewer fatigue, some researchers require interviewers to:

a. read the interviewer code of ethical behaviorb. take frequent breaks

c. alternate surveys

d. A and Be. B and CAnswer: (e) Difficulty: (Easy) Page: 40219.Assuring anonymity or confidentiality is used as control for:

a. fatigue

b. leading the respondent

c. falsehoods

d. misunderstandings

e. interviewer cheatingAnswer: (c) Difficulty: (Moderate) Page: 40220. Validation is used to reduce the intentional respondent error of:a. falsehoods

b. misunderstandings

c. distractions

d. guessing

e. third-person techniques

Answer: (a) Difficulty: (Difficult) Page: 402

21.The "third-person technique" is useful for:

a. reducing fatigue by bringing in another interviewer

b. reducing falsehoods and nonresponsec. reducing leading questions because a third party is listening

d. reducing fieldworker cheating by having a third party witness the interviews

e. None of the above; there is no such "technique."Answer: (b) Difficulty: (Difficult) Page: 40222.Reversal of scale endpoints may be useful for controlling:

a. sampling error

b. falsehoods

c. nonresponse

d. attention loss

e. misunderstandings

Answer: (d) Difficulty: (Difficult) Page: 40323."Prompters" may be used to:

a. control for attention loss or fatigue

b. interviewer error

c. facilitate proper training

d. direct interviewers where to find respondents

e. eliminate misunderstandings

Answer: (a) Difficulty: (Moderate) Page: 40324.Which of the following is true regarding the role professional field data collection companies play in terms of nonsampling error?a. Thus far, these companies have failed to recognize nonsampling error.

b. Only a handful of these companies can be counted on to effectively control nonsampling error.c. Although these companies give "lip service" to nonsampling error reduction, they offer little help in the control of nonsampling error.

d. These firms are waiting on technological advances to help in the control of nonsampling error.

e. Because their existence relies on providing high quality data, field data collection firms are relied on by researchers to know both the sources of and ways to control nonsampling error.Answer: (e) Difficulty: (Difficult) Page: 40325. Which of the following is a data collection error that is unique to online surveys?

a. multiple submissions by the same respondent

b. fatigue

c. misunderstandings

d. cheating

e. distractions

Answer: (a) Difficulty: (Moderate) Page: 40426.Which of the following has been labeled the marketing research industry's biggest problem?

a. sampling error

b. falsehoods on the part of respondents

c. nonresponse error

d. item omissions

e. population misrepresentation

Answer: (c) Difficulty: (Difficult) Page: 40427. Which of the following best describes the trend in nonresponse rates in telephone surveys conducted from 1979 to 2003?

a. response rates have continually increased

b. response rates have remained unchanged primarily due to the effects of online surveys

c. response rates have slowly declined over this period of years

d. response rates increased dramatically from 1979 to about 1988, and then saw a downward fall that was just as dramatic

e. response rates increased dramatically from 1979 to about 1985, and then saw a downward fall that was just as dramatic

Answer: (c) Difficulty: (Difficult) Page: 405

28.What are the types of nonresponse errors?

a. refusals, break-offs, and hang-ups

b. hang-ups, refusals, and refusal to answer a particular question (item omission)

c. refusals, break-offs, and refusal to answer a particular question (item omission)

d. hang-ups, attention loss, and call back

e. hang-ups, attention loss, and misunderstandingsAnswer: (c) Difficulty: (Moderate) Page: 40529.A respondent reaches a certain point in the interview, and then decides not to answer any more questions. This is known as:

a. quitting

b. nonresponse

c. item(s) omission

d. break-offs

e. failure to communicateAnswer: (d) Difficulty: (Moderate) Pages: 40540630. Of the factors that determine whether or not a person will participate in a survey, which of those listed below was found by researchers to increase participation?

a. any topic dealing with sports

b. topics related to current TV shows that are also popular

c. topics relating to taxation and social welfare

d. topics related to youth

e. topics of interest to the potential respondent

Answer: (e) Difficulty: (Moderate) Page: 406

31. One decision rule used in the research industry relating to how to define a "completed survey" is to define a completed survey as one in which all the:

a. primary questions have been answered

b. primary and secondary questions have been answered

c. primary, secondary, and tertiary questions have been answered

d. questions have been answered

e. classification questions have been answered

Answer: (a) Difficulty: (Difficult) Page: 408

32. Incentives often used to reduce nonresponse error in survey research include:

a. advance notification, assurance of insurance, and claims to be certified

b. post notification, monetary incentives, and follow-up contacts

c. advance notification, monetary incentives, and follow-up contacts

d. ex post facto notification, assurance of insurance, and follow-up contacts

e. promises not to "sell anything"Answer: (c) Difficulty: (Difficult) Page: 408

33. Callback attempts are important to reduce nonresponse and usually about ________ callback attempts should be made to the not-at-homes, busy signals, and no answers.

a. 8 to 10

b. 7 to 10

c. 5 to 10

d. 3 to 10

e. 3 to 4

Answer: (e) Difficulty: (Difficult) Page: 408

34.Which of the following methods used to reduce nonresponse error is best for reducing the number of not-at-homes, busy signals, and no answers?

a. advance notification

b. monetary incentives

c. increasing the "wrong targets"d. callbacks

e. decreasing the "ineligibles"Answer: (d) Difficulty: (Difficult) Page: 408

35.The response rate:

a. essentially enumerates the percentage of the total sample with which interviews were completed

b. essentially enumerates the percentage of the population with which interviews were completed

c. essentially enumerates the number of the total sample with which interviews were completed

d. is the percentage of positive responses to questions dealing with important research objectives e. is the percentage of qualified respondents who are screened out of the sample due to the incidence rate

Answer: (a) Difficulty: (Easy) Page: 40936.If your response rate is 55 percent, your nonresponse rate is:

a. 55 percentb. 45 percentc. 55 percent/55 percent or 100 percentd. 45 percent/55 percent or 82 percente. You cannot calculate the nonresponse rate by knowing only the response rate.

Answer: (b) Difficulty: (Easy) Page: 40937.Which of the following organizations has provided the industry with a standard for calculating response rates?

a. American Marketing Association

b. International Marketing & Research Association (IMRA)

c. Council of American Survey Research Organizations (CASRO)

d. Marketing Research Associatione. International Council for Response Rate Standardization

Answer: (c) Difficulty: (Difficult) Page: 40938.The nonresponse rate is calculated by:

a. dividing the number of completed interviews by the number of eligible units in the sample

b. dividing the number of completed interviews into the number of eligible units in the sample

c. dividing the percentage of completed interviews by the percentage of eligible units in the sample

d. dividing the percentage of incomplete interviews by the incidence rate

e. None of the above

Answer: (e) Difficulty: (Moderate) Page: 40939. If, after conducting a survey, it is determined that nonresponse is a problem, one alternative that may be considered is weighting. Essentially, weighting averages means:

a. applying weights (to accurately reflect the proportions of subgroups that represent the population) to subgroup, means to compute an overall score that adjusts for the nonresponse of that subgroup

b. applying weights to answers of subgroups until the answer is believed to accurately reflect the way the true subgroup in the population would have answered the research question

c. reproportioning subgroups in the population in a way that matches how they are proportioned in the sample, and then recalculating the survey data

d. applying a weighted value of .5 or less to all sample respondents' answers who are felt to be underrepresented in the sample data

e. applying a weighted value of .5 or less to all sample respondents' answers who are felt to be overrepresented in the sample data

Answer: (a) Difficulty: (Difficult) Page: 412

40.After the surveys have been completed:

a. data entry follows

b. there should be a preliminary screening of the questionnaires

c. they should be sent to the tabulation room

d. they should be shown to the client to verify that they were completed

e. all respondents should be recalled by a supervisor to ensure that they were actually interviewed

Answer: (b) Difficulty: (Easy) Page: 41341.When respondents have a tendency to respond favorably or unfavorably regardless of the question, it is known as:

a. yea- or nay-saying

b. endzoning

c. unreliable response

d. research with low variability

e. None of the above; the respondent is obviously favorable or unfavorable toward the topic being evaluated.

Answer: (a) Difficulty: (Easy) Page: 41442. Unreliable respondents, identified in questionnaire screening, should:a. be resurveyed

b. be oversampled to reduce unreliability

c. be weighted so as to provide reliable estimates as to how their population subgroup would have responded to the survey question

d. not be encouraged to participate in future research and their incentive, if promised, should be withheld

e. be eliminated from the sample

Answer: (e) Difficulty: (Easy) Page: 415

TRUE/FALSE QUESTIONS

43. Some research has concluded that including the statement "I am not selling anything" has a small beneficial effect on response rates but does not affect the percent of persons who terminate mid-interview.

Answer: (True) Difficulty: (Difficult) Page: 39144.There are two kinds of error in surveys; sampling error and nonsampling error.

Answer: (True) Difficulty: (Easy) Page: 39245.Several types of nonsampling error exist, but fortunately we can eliminate this error by having a lower margin of allowable error (e).

Answer: (False) Difficulty: (Difficult) Page: 39246.We can predetermine (and control) the level of sampling error; we cannot predetermine the amount of nonsampling error we will have.

Answer: (True) Difficulty: (Moderate) Page: 39247.Nonsampling errors stem primarily from either fieldworker errors or respondent errors.

Answer: (True) Difficulty: (Easy) Page: 39248.One of the primary causes for interviewer cheating stems from the compensation method: payment per completed interview.

Answer: (True) Difficulty: (Moderate) Page: 39349.Very subtle "cues" from the interviewer may lead the respondent to answer the questions correctly. These "cues" should be properly taught to the interviewers during training so that the results may be guaranteed (within limits).

Answer: (False) Difficulty: (Easy) Page: 39450.If an interviewer has an accent, it may bias the respondent.

Answer: (True) Difficulty: (Easy) Page: 39551. One characteristic of outstanding telephone interviewers is work orientation: a hard worker who considers interviewing a personal skill.

Answer: (True) Difficulty: (Easy) Page: 39652. Falsehoods and nonresponse are two unintentional respondent errors.

Answer: (False) Difficulty: (Easy) Page: 39653. Respondents can misunderstand, guess, lose attention, suffer distractions, and suffer respondent fatigue. These are all examples of unintentional respondent error and they all contribute to nonsampling error.

Answer: (True) Difficulty: (Easy) Page: 39954. In research conducted to identify "hard to get" respondents and compare them with "easy to get" respondents, "hard to gets" are typically older and less likely to be employed than "easy to gets."Answer: (False) Difficulty: (Difficult) Page: 399

55.Supervision and validation are ways the marketing research industry controls for interviewer cheating.

Answer: (True) Difficulty: (Moderate) Page: 40056. Supervision and validation are ways the marketing research industry controls for interviewers leading respondents.

Answer: (True) Difficulty: (Moderate) Page: 400

57.An industry standard for validation is to recall 50 percent of the respondents to verify that they were, in fact, interviewed.

Answer: (False) Difficulty: (Difficult) Page: 40158.Anonymity occurs when the respondent is given assurances that his or her answers will remain private; the researcher will not divulge the name of the respondent with the responses.

Answer: (False) Difficulty: (Difficult) Page: 40259.The "third-person technique" may be used to reduce falsehoods.

Answer: (True) Difficulty: (Moderate) Page: 40260.Reversal of scale endpoints is helpful in order to change the results to more closely match the research objectives. If the objectives are met, the client will have greater satisfaction and will likely use the services of the research firm again.

Answer: (False) Difficulty: (Easy) Page: 40361.One of the problems with online surveys is that they may lead more people to submit bogus responses about themselves, creating "bogus respondents."Answer: (True) Difficulty: (Easy) Page: 40462.One of the problems with online surveys is that, because many consumers are not connected to the Internet, they may misrepresent the intended population.

Answer: (True) Difficulty: (Easy) Page: 40463. Nonresponse is composed of refusal and item omission; break-offs are not considered a form of nonresponse.

Answer: (False) Difficulty: (Difficult) Page: 40564. The University of Michigan's Survey of Consumer Attitudes has shown that response rates are declining.

Answer: (True) Difficulty: (Difficult) Pages: 404-40565.An item omission refers to the failure to include an important question on a survey questionnaire. Therefore, item omissions may seriously threaten the achievement of the research objectives.

Answer: (False) Difficulty: (Moderate) Page: 40566. It was long thought that response rates increased if the respondents have a high level of interest in the research topic. However, recent research has shown this is false.Answer: (False) Difficulty: (Difficult) Page: 40667. A completed interview is always defined as an interview in which 100 percent of the questions have been answered and in a way that they are all supportive of the research objectives.

Answer: (False) Difficulty: (Easy) Page: 40768.A good definition for a completed interview is one that has all questions answered up to the demographic questions at the end of the survey.

Answer: (True) Difficulty: (Difficult) Page: 40769. A method that may reduce nonresponse error is to provide potential respondents with an advance notification, either by telephone or by letter, that they are going to be asked to take part in some research.

Answer: (True) Difficulty: (Easy) Page: 408

70. If a respondent is called and is "not at home," that sample element is no longer usable and callbacks should not be made. To do otherwise will distort the response rate.

Answer: (False) Difficulty: (Moderate) Page: 408

71.If a respondent is not at home or if their telephone is busy, it is standard practice in the research industry to immediately substitute that sample member with the next person on the sample frame.

Answer: (False) Difficulty: (Difficult) Page: 40872.CASRO, the Council of American Survey Research Organizations, has provided standards for calculating response rates.

Answer: (True) Difficulty: (Difficult) Page: 40973.Response rate and nonresponse rates are related; if the response rate is 60 percent, then the nonresponse rate is 40 percent.

Answer: (True) Difficulty: (Easy) Page: 40974.Response rate is calculated as the number of units in the sample divided by the number of completed interviews.

Answer: (False) Difficulty: (Difficult) Page: 40975. In calculating the response rate for studies with screening questions and, therefore, a certain percentage of the sample who are "eligible," it is impossible to estimate the percentage of those members of the sample who are not called (i.e. not at homes, busy signals, etc.) who are "eligible." Therefore, the response rate formula omits the concept of eligibility, thus giving us response rates much lower than they should be.Answer: (False) Difficulty: (Difficult) Page: 40976. If a study is conducted and nonresponse error is a problem, one remedy may be applying weighted averages that essentially increase or decrease a subgroup's proportion in the sample to that proportion known to exist in the population.Answer: (True) Difficulty: (Difficult) Page: 41277. One remedy for studies that are thought likely to have high nonresponse rates is to oversample, which means we draw a sample larger than calculated in order to meet the required sample size calculated by the sample size formula.

Answer: (False) Difficulty: (Difficult) Page: 41278.When a researcher sees a "middle-of-the-road pattern" during the questionnaire screening process, it signals that a good, average response was obtained by the survey and this is a good sign that the research process did not produce strange abnormalities.

Answer: (False) Difficulty: (Moderate) Page: 41479.Sometimes a researcher will include an internal consistency check to ensure that reliable answers are being given by respondents. This is normally carried out by having two different questions that measure the same construct imbedded somewhere in the questionnaire.

Answer: (True) Difficulty: (Moderate) Page: 415APPLICATION QUESTIONS80. Rachel Luecht was responsible for conducting a survey of her university to determine what extent the students were in favor of a number of changes the university administration was considering. She would need to collect four hundred completed interviews in a five-day period, and the administration had given her a budget of $2,000.00 to collect the information. Rachel put posters up around the campus asking for students who wanted to earn extra cash by interviewing other students on campus. At the meeting of the volunteers, Rachel offers to pay them $8.00 an hour and told them they should complete at least three interviews for each hour they worked. Rachel's decision to pay the interviewers this way will:

a. help ensure there are no misunderstandings among the fieldworkers collecting the information

b. help ensure that there is a minimum of fatigue among the fieldworkers collecting the information

c. help ensure that her fieldworkers do not cheatd. help ensure that leading respondents will be held at a minimum because she is giving the fieldworkers ample time to conduct the interviews

e. help control respondent misunderstandings

Answer: (c) Difficulty: (Difficult) Page: 393

81. Margaret Day Craddock is the director of MIFA, a charitable organization in Memphis, Tennessee. Margaret has contracted with a marketing research firm in Memphis to conduct a telephone survey of Memphis MSA residents to assess their awareness of charitable organizations, tax code provisions for charitable donations, attitudes toward charitable organizations, and intentions to contribute in the coming year. Not everyone contacted will be eligible to take part in the study as there is a screening question which asks if the potential respondent is the person in the household that is most likely to make decisions regarding the family's contributions to charity. Because Margaret plans on using the research information to help her set objectives and the budget for the coming year, she has instructed the research firm to ensure that the information is representative of the MSA. Bob Smith, of the research firm, is using a probability sample and he is taking safeguards to minimize nonsampling error. Today, Bob is working on factors that will affect the response rate. Because there will be a screening question, Bob knows that this will:

a. not have any impact on the response rate

b. not have any impact on the nonresponse rate

c. have an impact on the response rate in that only those respondents who are "eligible" (i.e. pass the screen), will be included in the denominator of the response rate formula

d. have an impact on the response rate in that only those respondents who are "ineligible" (i.e. fail to pass the screen), will be included in the denominator of the response rate formula

e. lower the response rate because many persons contacted will not be responsible for making contribution decisions

Answer: (c) Difficulty: (Difficult) Page: 410

82. Bob Smith is conducting a survey for Margaret Day Craddock, the director of MIFA, a charitable organization in Memphis, Tennessee. Margaret has contracted with Bob's marketing research firm in Memphis to conduct a telephone survey of Memphis MSA residents. Not everyone contacted will be eligible to take part in the study as there is a screening question which asks if the potential respondent is the person in the household that is most likely to make decisions regarding the family's contributions to charity. Bob will use the CASRO formula, discussed in your text, to determine the response rate for the study. The formula essentially divides the number of completed interviews by the number of eligible units (respondents) in the sample. Which of the following best represents how the number of eligible units in the sample is calculated?a. 100 percent of the samplethose failing the screening question

b. 100 percent of the sample; ALL respondents are "eligible" whether they pass the screen or notc. ("eligibles" consist of those passing the screen) plus (the percentage of those eligible among respondents we talked with) times (the number of respondents we did not talk with due to refusals [before the screen], no answers, busy signals, or not at homes)

d. ("ineligibles" consist of those passing the screen) plus (the percentage of those ineligible among respondents we talked with) times (the number of respondents we did not talk with due to refusals [before the screen], no answers, busy signals, or not at homes)

e. None of the abovebecause there is no CASRO formulaAnswer: (c) Difficulty: (Difficult) Pages: 409410208PAGE 209