Effects of mode and mood on self-report by college students

Rebecca L. Rosen, New School for Social Research

Michael F. Schober, New School for Social Research

Frederick G. Conrad, University of Michigan

2009 ITSEW, Tällberg, Sweden

Mode differences in answering sensitive questions

• Rs answer sensitive questions differently when they are interviewer administered vs. self-administered

• General finding: in more anonymous and confidential modes– more reports of “undesirable” and sensitive

behaviors– fewer reports of “desirable” behaviors

Mechanisms are not yet fully understood

• What is embarrassing or interpersonally challenging may vary in predictable ways across– different subpopulations– cultural styles/norms of interaction– individual style of interacting

What we are examining:– mood states of respondents

A potentially relevant mood state: Depression

• Depressed people:– experience interpersonal dysfunction (Brim, et al.,1982)

– feel discomfort in interactions (Brugha et al., 1982)

– are socially unskilled with strangers (Gotlib & Robinson, 1982)

– withdraw from others (Rippere, 1980)

– feel shame (Hobletzelle, 1987)

– have low self esteem (DSM-IV-TR, 1996)

• This affects all interpersonal interaction

Possible effects• Being with an interviewer may be particularly

embarrassing for depressed people

• OR Presence of interviewer could temper extreme negative self-appraisal (reality check)

Both predict greater mode differences for depressed Rs than for non-depressed Rs for Qs they might find embarrassing

Reliable mode differences: more reporting when more anonymous/confidential

• Drug use (e.g., Aquilino, 1994)

• Sex partners (e.g., Tourangeau & Smith, 1996)

• Alcohol use (e.g., Aquilino, 1994)

• Smoking (e.g., Brittingham, Tourangeau & Kay, 1998)

• Personal distress (e.g., Epstein, Barker & Kroutil, 2001; Moum, 1998

Study

• Compare answers to neutral, sensitive and personal distress Qs FTF and in CASI

• In a population that is – likely to experience notable rates of depression– likely to engage in measurable amounts of

stigmatized behavior

Strategy

• Relatively small lab sample– But believed large enough to replicate reliable large-

sample mode differences– Full control over mode of administration

• All interviews conducted in same room• Same female FTF interviewer wearing same clothes • Anonymity and confidentiality of data made evident

• College student-age population– Known to have highest rates of personal distress– Highest rates of stigmatized behaviors likely to be

relevant

Design: two arbitrarily assigned groups

• 87 FTF (CAI) interviews– Same female I for all – I asks Qs aloud– enters Rs spoken responses on laptop

• 87 CASI interviews– R reads Q on desktop screen– Enters answers by typing or clicking

37 survey Qs• 9 neutral Qs

– from National Survey of Student Engagement, 2007

• 18 sensitive behavioral Qs– from 7 large scale surveys, including

• Behavior Risk Factor Surveillance System, 2007• National Survey of Drug Use and Health, 2006

• National Survey of Family Growth, 2003

• 10 personal distress Qs– Patient-Health Questionnaire-8

• Used in slightly modified form in BRFSS (2006)• Used in clinical practice

– Generalized Anxiety Disorder-2

Post-interview measures

• Paper and pencil self-administered, in different room:

• Beck Depression Inventory-II • Beck Anxiety Inventory• Marlowe-Crowne X-2 Social Desirability Scale

FTF and CASI groups did not differ on any

Respondents

• 174 undergraduate college students, balanced across FTF and CASI for– Age (Mean age 20, range 18-24)– Gender (half F, half M) – Year in college (1/4 in each year)– Race/ethnicity (almost balanced)

• 45% white, 17% Black or African-American (non-Hispanic), 16% Asian, 8% Hispanic or Latino, 6% multiracial, 7% other

– 91% from New York area schools– Rs paid $15 for 1hr

Depression Level

• Depression level assessed after the interview–Rs not selected or assigned to mode

based on depression level• Almost equal numbers of depressed

and non-depressed in each mode– 113 total non-depressed, 61 depressed– Depressed = any BDI-II score above 13

Mode effects

• Sample large enough to detect reliable mode effects - 1/3 of Qs

• Replicates larger-sample findings that Rs report more sensitive/stigmatized behaviors in CASI than FTF

How many times in the past 4 weeks

have you masturbated on your own? Mode

0

1

2

3

4

5

6

7

8

9

10

Number of times

FTFCASI

Mode and mood: Across items

• 9 neutral Qs: no reliable effects

• 12 numerical-response sensitive Qs:– Mode effect for depressed Rs (more reports CASI)– No mode effect for non-depressed Rs– Significant interaction by MANOVA

• 10 personal distress Qs:– Mode effect for depressed Rs (more reports CASI)– No mode effect for non-depressed Rs– Significant interaction by MANOVA

Sensitive example 1:During the past 30 days, on how many days did

you use marijuana or hashish?

Mode x Depression Level cov Gender

0

1

2

3

4

5

6

FTF CASI

Number of days

not depresseddepressed

Sensitive example 2:On the days that you drank during the past 30

days, how may drinks did you usually have each day?

Mode x Depression Level, cov Gender

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

FTF CASI

Number of drinks

not depresseddepression

Personal distress example:Over the last 2 weeks, how often have you been bothered by

feeling bad about yourself – or that you are a failure or have let yourself or your family down?

Mode x Depression Level cov Gender

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

FTF CASI

How often

not depresseddepressed

Response options0 = not at all3 = nearly very day

Summary

• In general, depressed college student Rs – reported more sensitive behaviors and more personal

distress in CASI than FTF

• Non-depressed college student Rs either – Reported equally FTF and in CASI– OR reported more sensitive behaviors and personal

distress FTF

• Effects are not because of gender (despite greater female reporting of depression):– effects are even stronger with gender covaried out

Why? Could be:• Fear of negative evaluation by others

– can cause people to hide, not want to be “seen”– E.g., Tangney Wagner & Gramzow, 1992; Lewis, 1987

– shame linked with depression – e.g. Hobletzelle, 1987; Tangney, et al., 1992

• Depressed Rs: Shame might cause less reporting FTF

• Non-depressed: Less shame, natural variation in what Rs might find embarrassing; or see behavior/experience as “just me”

Humanness of “interviewer”?

• Humanness of “interviewer” (human or computer) may be important

• Partially human interfaces (recorded voices, video interviewers, animated agents) may activate different interpersonal schemas

Need more nuanced theories of mechanisms for mode effects

Mode effects may be moderated by• Interpersonal dynamics• Nature of what is being asked about

– Behaviors vs. subjective experience

• Why people think they’re answering– e.g., help-seeking vs. survey responding

• Mood states and variations other than demographic

Acknowledgments

• NSF grant SES-0551294 (Methodology, Measurement, and Statistics program)

• Laura Hahn Lind• Michelle Levine• Shawn Mason• Deb Schwartz• Josh Rosen• Courtney Kellner• Stefanie Fail• Shireen Rizvi• Ali Khadivi