Wearable Cameras in Psychological Science - in presspsy2.fau.edu/~shermanr/Wearable Cameras in...

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE 1

Accepted Manuscript at Social Psychological and Personality Science 2/24/2017

A snapshot of the life as lived:

Wearable cameras in social and personality psychological science

Nicolas A. Brown, Andrew B. Blake, & Ryne A. Sherman Florida Atlantic University

Authors’ Note

This research was supported by National Science Foundation grant #1420105 to Ryne A.

Sherman, Principal Investigator. Any opinions, findings, conclusions, or recommendations

expressed in this article are those of the individual researchers and do not necessarily reflect the

views of the National Science Foundation. All statistical analyses were conducted using R (R

Core, Team, 2016).

Correspondence regarding this article may be sent to Ryne A. Sherman, Department of

Psychology, Florida Atlantic University, Boca Raton, FL 33431. Email: [email protected]


Abstract

Recently advances in mobile sensing technology provide innovative methods for understanding

how individuals think, feel, and behave in vivo. Despite the utility of these new methods, as of

yet researchers have not been able to see the environments people encounter in their daily lives.

In this article, we introduce wearable cameras as a new tool for sampling from participants’

everyday situations. Wearable cameras are small devices that capture pictures on a fixed interval

(e.g., 30 s). This article discusses the benefits and disadvantages of incorporating wearable

cameras into personality and social psychological research. Drawing on our experiences using

wearable cameras in research, we provide insights into ethical and legal considerations when

designing studies using these devices. Lastly, we report data on situation change and perceived

obtrusiveness from, to our knowledge, the first wearable camera study in personality and social

psychology.


“…lives are lived day by day, one day at a time, from day to day, day after day, day in and day out.”

(Craik, 2000, p. 234)

A fundamental goal of psychological science is the description and explanation of human

behavior. For social psychology, the laboratory experiment has proven to be an indispensable

method for advancing knowledge of human behavior. For personality psychology, surveys and

personality assessments have been equally indispensable. Despite their utility for psychological

science, laboratory experiments and personality assessments offer, at best, an incomplete and

impoverished view of lives as they are lived. The purpose of this article is to introduce a new

method for studying lives as they are lived: wearable cameras. To do so, we (1) provide a brief

historical overview of lived day analyses in social/personality psychology, (2) describe wearable

camera technology with attention to its advantages, limitations, and ethical considerations, and

(3) present some empirical results from a large wearable camera study.

A Brief History of Measuring Lives as Lived

In perhaps the most well-known study of an individual’s daily life, Barker and Wright

(1951) sent observers to meticulously document the day of a 7-year-old boy in the Midwestern

United States. Each observer followed this boy every place he went, keeping notes about every

situation that he experienced (e.g., getting dressed, going to school) over the course of his entire

day. Despite such pioneering efforts to overcome the long-identified limitations of traditional

psychological research methods (Reis, 2012), it is only recently that a large wave of research

focused on studying psychological processes as they naturally occur began (Bolger, Davis, &

Rafaeli, 2003). One reason for this is undoubtedly that recent advances in technology have made

studying psychology outside the lab more feasible than ever.


Prior to the 1980s, the most state-of-the-art method for studying life as it is lived was the

daily diary method (Wilhelm, Perrez, & Pawlik, 2012). Since then, a number of methods have

emerged that allow researchers to collect ecological/environmental/contextual data. These

methods include experience sampling1 (ESM; Conner, Tennen, Fleeson, Barrett, 2009; Scollon,

Kim-Prieto, & Diener, 2003; Shiffman et al., 2008), the Electronically Activated Recorder

(EAR; Mehl, Pennebaker, Crow, Dabbs, & Price, 2001), and mobile sensing using smartphones

(Dufau et al., 2011; Miller, 2012; Sandstrom, Neal, Mascolo, & Rentfrow, in press). Table 1

provides a brief description of these methods and their relative advantages and disadvantages.

While all of these methods have enriched our view of individuals in their everyday

environments, none provide a window for researchers to actually see the environments people

experience.

Being able to (objectively) see another person’s natural environment is advantageous for

many reasons. While methods such as the EAR or ESM are excellent, neither captures objective

information about the physical world (e.g., Is it dark? How many people are present?) very well.

For example, while the EAR can be used to estimate how much time a person spends reading

(i.e., by listening to pages turn; Mehl et al., 2001), this method cannot inform the researcher

about what the person is reading. In our view, reading a textbook on thermodynamics seems

psychologically different from reading Teen Magazine. While ESM provides some ability to

combat this issue (e.g., you can ask a person what they were reading), typical ESM studies limit

the amount of information a participant can provide because long surveys increase participant

fatigue. Moreover, ESM requires the participant to first perceive and filter environmental

information before responding, making the information they provide more subjective and

1 Other similar terms include ecological momentary assessment (EMA; Shiffman et al., 2008) and intensive repeated measures in natural settings (Moskowitz & Sadikaj, 2012).


potentially biased. Wearable cameras provide raw, unfiltered, and objective information about

the visual environment. No other method offers this.

Wearable cameras may be useful for many research endeavors. Close relationships

researchers may ask partners to simultaneously wear camera to investigate time spent together,

talking to each other, facing one another, etc. Such dyadic designs could permit researchers to

understand interdependent behavior (Kelley & Thibaut, 1978), processes underlying phenomena

such as perceived partner responsiveness, or conflict resolution. Likewise, aggression researchers

may equip participants with cameras to better understand the antecedents of aggressive behavior

in real-world contexts. Cameras may be particularly adept at picking up signs of implicit

discrimination (e.g., lack of eye contact) in everyday situations. In our view, wearable cameras

are useful for answering an innumerable number of research questions because they extensively

and conveniently record the daily situations a person encounters. As situations are a key driver of

behavior, wearable cameras provide a powerful tool for measuring them.

Wearable Cameras in Psychological Science

Until recently, studies that permit researchers to see the environments that people

experience have been quite rare. One ambitious study hired a professional video-recording team

to follow a few volunteers around for a 24-hour period (Craik, 2000). While this study

demonstrated the value of capturing every moment in a person’s day for understanding that

person, it is not feasible to conduct such a study on a large group of participants (e.g., N=200 or

more). Further, the size and obvious presence of the recording team and equipment increases the

likelihood of measurement reactivity (e.g., biased data due to the act of being measured; Barta,

Tennen, & Litt, 2012). Such reactivity could sacrifice the ecological validity supposedly gained

by conducting research in the real-world (Reis, 2012).


Table 1.

Methods for collecting ecological data

Method Description Advantages Disadvantages

Daily diary Assessment of construct of interest on a once-a-day interval

• Suitable for behavior that occurs relatively infrequently (e.g., about once a day)

• Relies on retrospective reporting which may be prone to memory biases

• Does not capture phenomenon of interest in real time

Experience sampling (ESM)

Brief surveys sent to participants at set intervals via smartphone app, text messaging, etc.

• Can capture psychological processes in vivo

• Relatively inexpensive to implement

• Can cause survey fatigue through repeated assessments

• Unable to glean precise information about participant location

• Requires participant to own a mobile phone (or borrow similar device)

Electronically Activated Recorder (EAR)

Device that records brief audio clips throughout the day at semi-regular intervals

• Can hear behavior via language expression as it occurs in real time

• Legal concerns in certain states regarding privacy laws

• Sound files must be manually transcribed Mobile sensing Uses participant’s smartphone to

collect data such as physical location, responses to brief surveys

• Do not have to buy new equipment (use participant’s existing smartphone)

• Relatively inexpensive • Access to numerous smartphone

sensors (e.g., GPS, light, accelerometer)

• Assumes participants own a mobile phone (or must borrow one from lab)

• Must develop an application that can work across different smartphones (e.g., iPhone, Android)

Wearable cameras

Small cameras that take images, and collect information about participant’s location

• Permits researcher to see situations that participants experience

• Legal concerns in certain states regarding privacy laws

• Images may be unusable in certain conditions (e.g., at night, in poor lighting conditions)


Modern Wearable Cameras

In an effort to better know themselves, millions of individuals now track a variety of

physiological and psychological data such as sleep quality, number of steps walked, and daily

emotion (Swan, 2013). Wearable cameras that capture pictures of daily life are one of the latest

devices added to this Quantified Self movement. For example, the Narrative Clip is a small,

lightweight camera that automatically captures pictures according to preset intervals (see Figure

1).2 The device contains many features including an accelerometer, a wide-angle lens, GPS, and

Bluetooth connectivity. Although there are many sensors available on the Narrative Clip, our

focus here is on the mobile camera feature. We began using the Narrative Clip as part of a large-

scale investigation of person-situation transactions (Sherman, 2014). In what follows, we draw

on our experiences from this investigation to describe the advantages and limitations of the

Narrative Clip (simply referred to as “wearable camera” hereafter).

Advantages. First, as Figure 1 demonstrates, the wearable camera is small and

lightweight—participants can easily wear the device on their shirt or blouse (or on a lanyard

around their neck). The camera is small enough to be unobtrusive to both the participant wearing

the device, and to casual observers. This feature is particularly desirable due to possible

problems introduced by reactivity (but see Limitations). Second, the wearable camera is simple

to operate. The device captures images at regular intervals while it detects light, putting itself in

sleep mode when no light is detected. As a result, participants can simply place the device in

their pocket when they do not wish to record.

2 For a full description of the Narrative Clip, see www.getnarrative.com. Although other wearable cameras may be available, because our experience only pertains to the Narrative Clip, we emphasize its functionality here. There may be specific advantages and drawbacks associated with other cameras, but the ethical, legal, and logistical concerns discussed here apply to wearable cameras in general.


Third, the wearable camera possesses adequate battery life. In our most recent study, the

device was capable of holding a charge for at least a participant’s waking day (approximately 16

h). However, studies spanning multiple days are also possible as the device can be easily charged

with a USB cable.

Figure 1. Volunteers wearing the wearable camera on shirt collars and with necklace.


Fourth, the wearable camera software and hardware are customizable. In terms of

software, users may customize the intervals between capturing images from as short as every 10

seconds to as long as every two minutes.3 This is ideal for studies that may require less (or more)

temporal resolution. Hardware may also be customized by exchanging wearable mounts and

lanyard-type accessories. Finally, the cost for the camera is on par with other ambulatory

assessment devices such as iPods or Palm Pilots (approximately USD$200). Depending on the

duration of the proposed study, researchers do not need to purchase many cameras.

Limitations. While new methodological and/or technological innovations present

promising ways to collect ecological data, there are often some accompanying limitations.

Wearable cameras are no exception. First, when participants wear the cameras, there is the

possibility for reactivity. That is, participants may behave differently than they would during

their normal day. This might include avoiding certain situations due to privacy concerns (e.g.,

going to the gym locker room).4 Furthermore, other individuals may behave differently around

the participant as a result of the participating wearing the camera. Second, because participants

can temporarily stop wearing the camera due to privacy concerns (e.g., in a restroom) or when it

is not physically feasible to do so (e.g., swimming in a pool), there is the possibility of forgetting

to continue wearing the camera (e.g., leaving it in one’s pocket). The resulting impact on data

collection could vary from relatively minor (only a few situations are not recorded) to major

(most of the participant’s day is missed). To counteract these potential issues, researchers could

periodically send messages (via e-mail or text-messaging) to remind participants when to start

wearing the device, and throughout the day(s) to ensure compliance. Nonetheless, because each

3 Altering the time between intervals impacts battery life—shorter intervals lead to decreased battery life, whereas longer intervals may increase battery life. 4 This can be mitigated by properly instructing the participant that the wearable camera can be temporarily removed.


photo is date and time stamped, researchers can estimate the total number of photographs

“missing” from the record.

Third, the quality of captured images may vary significantly as a result of three key

factors: time of day, how the camera is worn, and general movement (see Figure 2). Because the

camera does not possess a flash, images captured in low-light conditions are generally of poor

quality. While the images are still useful for participants to recall what they were doing, they

may be unusable for certain data analyses such as object recognition (see Future Directions).

Wearing the camera on a shirt or blouse may cause the camera angle to capture images of the

ceiling and so forth, instead of a straightforward, egocentric perspective. In these circumstances,

wearing the camera on a lanyard or relatively tight fitting necklace is preferred (see Figure 1).

General movement may also affect the quality of camera images—abrupt or sudden movements

can cause image blurring, although the device does have an accelerometer to help mitigate this

issue. As with all technologies, there is the possibility for researcher to encounter technical

difficulties with the cameras. In our experience, these problems were infrequent, but resulted in

the loss of participant images, or failure of the camera to capture any images on occasion.

Finally, relying on any third party software for research comes with risks associated with

that party’s existence. During the revision stage of this manuscript, the company producing

Narrative Clip announced it had filed for dissolution (Narrative, 2016). Although the company

has now been acquired and not shut down service, it is presently unclear how long the Narrative

Clips currently in existence will continue to be supported and if production of new Narrative

Clips will resume. Despite this, we believe that wearable cameras (as a research tool) are

unlikely to disappear from the market.


Figure 2. Sample images captured with the wearable camera.

Image quality exemplars Good Poor


Ethical and Legal Considerations using Wearable Cameras

Researchers hoping to incorporate wearable cameras into their research program might

understandably be concerned about both legal issues and securing approval from their

institutional review board (IRB) or local ethics committee. Having gone through the process

ourselves, we understand that significant ethical concerns may arise. Privacy concerns using

wearable cameras generally fall into one of two categories—the loss of privacy for the

participant, and for that of bystanders. Although IRBs have their own idiosyncrasies and might

interpret federal regulations differently, it is our hope that the following information will help

researchers anticipate and address these ethical concerns.

Protecting participant privacy. Because participants receive and sign informed consent

about the research, and are voluntarily capturing images of their daily lives and sharing them

with the research team, the potential risk to participant privacy is relatively easy to maintain.

Nonetheless, protocols should be put into place to ensure that participants may screen all of their

images prior to sharing them with the research team. This is as simple as downloading all of the

photos and allowing the participant to privately delete any photos that they do not wish to share

before any member of the research team sees them. Such a procedure is similar to those used by

EAR researchers (Mehl et al., 2001). Of course, this screening procedure alone does not absolve

the researcher of their responsibility to safeguard participants’ identity once the remaining

images are stored. To do so effectively, there are two primary strategies.

First, all images should be stored on devices that are password protected, and if possible,

encrypted. In the event these devices (e.g., external hard drives) are ever stolen or lost, the

possibility of the images being accessed is greatly diminished. Second, access to the repository

of participant images should be tightly controlled. Consent documents should describe who will


have access to the images – typically only members of the research team, which may include

research assistants, graduate students, and collaborators. Any individual that access the images

(e.g., research assistants in the coding process) should sign a confidentiality agreement agreeing

not to disclose any information regarding the content of the images.

Protecting the privacy of others. Although participants wearing the device are agreeing

to share parts of their public or private lives, it is inevitable that images including bystanders will

be captured. Thus, the following question arises: is it legal to take a picture of another individual

without their consent? Unlike wiretapping laws which govern audio recording, there are few

laws that prohibit visual recording (i.e., video or photography).5 In general, visual recording is

considered unlawful only if there is a reasonable expectation of privacy. For example, a person

outdoors at a public park, or on a public beach, has no reasonable expectation of privacy.

However, it could be deemed illegal, and is certainly unethical, to wear the camera in a private

residence, such as a friend’s dorm room, the restroom, or in a locker room. In these instances,

participants should take off the device until they return to a public area, or ensure that it is okay

with all other parties at risk to be photographed to continue wearing the device (e.g., a friend’s

dorm room). Additionally, we instruct participants that, should anyone in the situation feel

uncomfortable with them wearing the camera, it is best to remove the device temporarily. Lastly,

there may be some concerns that the camera could be used for surveillance purposes (i.e., spying

on others). Although the camera is a small device and may go unnoticed to the casual observer,

the camera is a poor surveillance device. With a battery life insufficient to sustain longer than 24

hours, poor performance under low lighting conditions, and relatively unpredictable photograph

timing, there are far better choices for surveillance activities on the market. In sum, the legal and

5 Although we use the term visual recording here, we refer specifically to photographing images.


ethical obstacles to wearing the device are surmountable, so long as reasonable precautions and

instructions to participants are provided.

An Application of Wearable Cameras in Social/Personality Psychological Science

Thus far we have introduced wearable cameras as a novel method for researchers to see

participants’ everyday lives and environment. In what follows we describe some results from a

recently completed, large scale study of person-situation transactions using wearable cameras.

Wearable cameras are particularly useful for person-situation transaction research because they

allow researchers to photographically capture virtually every (public) situation a person

encounters. Thus, one impetus for the study was a basic understanding of how people transact

from one situation to another. Here we present results of that investigation as well as data

regarding the perceived obtrusiveness of the wearable camera.

Method

Participants

Two-hundred and ninety-eight participants (Mage=21.11, SD=6.21) recruited from the

Florida Atlantic University community participated in exchange for up to USD$160.6

Participants were compensated USD$150 for completing all portions of the study (see Procedure

section below). Further, a USD$10 bonus was awarded to participants that recruited two of their

peer acquaintances to complete a personality survey regarding the target participant. The gender

breakdown of the sample was 40.6% female, 58.7% male, and 0.7% did not indicate. The sample

was comprised of 17% African-American, 6% Asian, 40% Caucasian, 27% Latino/a-Hispanic,

6 The target sample size for this study was initially 200 to detect effect sizes of approximately r=.20 with at least 80% power. Furthermore, effect sizes in the typical range for personality and social psychology tend to stabilize around N=200 (Schönbrodt & Perugini, 2013). In an effort to match the demographics of the local community, the total N was increased to 300. The project ended at N=299 when the final participant failed to complete the protocol after long delay.


8% Other, and 2% did not indicate, closely matching the ethnic diversity of the broader Florida

Atlantic University community.

Measures7

Situation change questionnaire. To assess situation change, we developed a categorical

forced-choice measure with five response options based in part on Buss’ (1987) theory of

person-environment interaction (see also Rauthmann & Sherman, 2016a). These items tapped

selection (“I left the situation for another situation”), manipulation (“I did something to

intentionally change the situation”), evocation (“The situation change was beyond my control”),

and construal (“My perception or goals in the situation changed”). If none of the four choices

accurately represented why their situation changed, participants were asked to select “Other.”

Obtrusiveness questionnaire. To assess the obtrusiveness of the camera, we adapted an

obtrusiveness questionnaire developed for EAR research (see Mehl & Holleran, 2007).

Participants rated the degree to which they felt the camera was obtrusive, both personally and to

bystanders, using a 5-point Likert-type scale ranging from 1 (Not at all) to 5 (A great deal). The

eight obtrusiveness questionnaire items are provided in Table 1. Participants also indicated if

they chose not to record any situations (yes or no) and were provided with an open-ended text-

box for further explanation. Lastly, participants were asked whether others asked them to take off

the camera (yes or no).

Procedure

This study was comprised of three parts. In Part 1, participants visited the laboratory and

were interviewed during a brief “getting-to-know-you” style interview. Afterward, participants

7 Because this is part of a large-scale project on person-situation transactions, participants completed many other measures including personality assessments, situation ratings, emotions, and goal checklists. Because this article is concerned only with the wearable camera methodology aspect of the project, we do not discuss them here.


completed a number of personality inventories, and were provided detailed instructions on how

to operate the wearable camera. For Part 2, participants were instructed to wear the camera on

the next day—from the time they wake up until they fall asleep—in all situations in which it was

feasible to do so.

In Part 3 of the study, participants returned to the laboratory where their images were

downloaded. Participants were allowed to privately screen their pictures and asked to delete

those that they did not wish to share with the research team (e.g., inadvertently wore camera in a

private location, etc.) Next, participants were provided the following instructions:

“For the next part of the study we are going to have you divide the pictures into

different episodes. The goal is for you to indicate each time your situation

changed. This might include each time you changed locations, someone else came

into the situation, or the topic of conversation changed. Use your own judgment to

decide when one situation ends and another begins.”

After completing this task, participants indicated the reason why each situation changed using

the situation change questionnaire (see Measures) and provided ratings of the situational

characteristics, momentary goals, and behavior. After rating all of their situations, participants

completed the obtrusiveness questionnaire.

Results

Participants provided data on a total of 5,280 situations, with an average of 18.46

situations per participant (SD=11.50; Median=16). A total of 254,208 images were gathered from

this study, an average of 889 images per participant (SD=405; Median=886.5; Min=18,

Max=1707). The average situation contained 48.15 images (SD=69.82; Median=23; Min=0;

Max=995). Participants indicated that their situations primarily changed due to selection (46.6%


of total situations), followed by manipulation (23.9%), construal (13.8%), and evocation

(11.0%). Only 4.7% indicated “Other” as a reason why their situation changed (see Figure 3),

suggesting that the four substantive categories capture the near totality of reasons for situation

change.

The descriptive statistics from the obtrusiveness measure are presented in Table 1 and are

visually depicted in Figure 4. Overall, participants were generally aware that they were wearing

the camera, however most did not find it uncomfortable to wear. Furthermore, participants

reported that bystanders were generally aware of the camera, and it was often talked about. To

some degree, the camera may have influenced the behavior of others, but judging by the median

(2), this was infrequent. It is important to note that the present study took place over only one

day. As other studies employing similar technology, such as the EAR, have demonstrated,

participants stop talking about the device after several days (Mehl & Holleran, 2007). We

suspect a longer study would find a similar result for wearable cameras. Importantly, participants

reported that the camera did not impede their activities, nor affect their behavior (including

places they went, and things they did). Lastly, nearly all of the participants (84.5%) reported that

they chose not to record certain situations (e.g., using the restroom), however only small

proportion of participants stated that bystanders asked them to temporarily stop wearing the

camera (11.3%). Sample reasons for not recording situations include, “Personal situations

involving hygiene,” “I didn’t record them because they may have been deemed illegal or

inappropriate,” and “I went on a date, and I took it off for that. I wanted to keep it personal.”

However, the vast majority involved being in private places (e.g., restrooms).


Table 2. Descriptive statistics for the obtrusiveness questionnaire

M SD Med Min Max Obtrusiveness for participant To what degree… …were you generally aware of the Narrative? 3.03 1.30 3 1 5 …did you feel uncomfortable wearing the Narrative? 1.86 1.31 1 1 5 …did the Narrative impede your daily activities? 1.42 0.82 1 1 5 …did the Narrative change your actual behavior? 1.34 0.74 1 1 4 …did the Narrative influence places you went? 1.42 0.87 1 1 5 Obtrusiveness for others To what degree… …were people around you aware of the Narrative? 3.14 1.30 3 1 5 …did you talk to people around you about the Narrative? 2.94 1.29 3 1 5 …did the Narrative influence the behavior of the people around you?

1.99 1.18 2 1 5

Note. N = 281. The average intercorrelation among the item is .17 (SD = .13, median = .15, min = -.03, max = .55). Figure 3. Frequency distribution of reasons for situation change.


To estimate the number of pictures “missing” from the record (i.e., either not recorded or

deleted by participants), we first assumed a 16 hour waking day for each participant. Thus, on

average, each participant would be expected to yield 1,920 pictures if they never removed the

camera or deleted any photos. As noted above, the mean number of pictures per participant was

889, indicating that, on average, 1,031 photos (out of the total possible) were not provided to us.

While this may seem like a large number missing, it is important to keep in mind that the 1,920

figure is an unreasonable expectation based on a 16 hour waking day, never removing the

camera, and deleting zero photos. Indeed, no participant actually reached this hypothetical total.

Flipping these numbers on their head, our cameras captured visual information for nearly 7.5

hours of participants’ daily lives.

Discussion

When developing a new research method, it is incumbent upon the researchers to

demonstrate that this new method provides insight or advantages over prior methods. That is,

mere existence of a method is not enough reason to use it. In the case of wearable cameras, the

results for situation change just described provide preliminary evidence that the wearable camera

method can provide new insights for social scientists.

Although theory about situation change has existed for nearly 30 years (Buss, 1987), only

this approach to measuring situation change provided us with an accurate description what kinds

and how much of situation change occur. We also believe that wearable camera methods are

superior to the day reconstruction method (Kahneman et al., 2004), because the photo-stream

produced by the cameras provides stronger memory cues for people attempting to recall their day

than time-period prompts. Although it seems rather obvious that people would recall more when

prompted by photographic streams than day reconstruction methods, one small study (n=14)


investigated this empirically and found that the wearable camera method identified

approximately twice as many activities as the day reconstruction method (Kelly et al., 2015).

At a broader level, the preliminary evidence from (to our knowledge) the first wearable camera

study in personality and social psychology, indicates that participants felt generally comfortable

wearing a small, wearable camera throughout their day. The feasibility of this methodology for

research is at least partially dependent on the willingness of participants to wear the device and

continue wearing it throughout the study. We acknowledge that the participants in the current

study are younger, are likely to be technologically savvy (so-called “digital natives”), and are

generally accepting of newer technology. However, it is important to note that wearable cameras

have been used previously in research, with both younger (e.g., Pauly-Takacs, Moulin, Estlin,

2011), and older participants (e.g., Hodges, Berry, & Wood, 2011). Nevertheless, because the

cameras are relatively easy to use, they appear to be a novel, yet promising method for

measuring lives as they are lived. In what follows, we discuss future directions for designing new

wearable camera studies and possible solutions for analyzing camera data.

Study design. Although the study presented here took place in multiple phases in the lab,

it is important to note that studies do not need to be overly complicated. For example, in a

hypothetical study, participants could complete a number of surveys before coming to the lab to

pick up a wearable camera. After wearing the camera for a specified period of time (e.g., a few

hours, an entire day), the participant could download the images on their own computer and

complete any necessary ratings via an online survey. For studies lasting longer than a day,

participants could charge the device on their own at home. Furthermore, instead of relying on

retrospective reports, researchers could integrate wearable cameras with an ESM to collect in

vivo assessments. These ESM reports could be matched to the date and time of the images.


Lastly, wearable cameras could be coupled with the EAR or other mobile sensing devices to

provide information about the raw, physical stimuli in the participant’s environment as well as

any auditory stimuli or geolocation data. Indeed, we believe that the future of real-time, real-life

studies will combine several of these methods.

Coding data. A major advantage of using wearable cameras to study daily life is the

large number of photographs that provide a visual record of the person’s encounters. These

photographs can be coded and analyzed in a variety of ways. First, as described previously,

participants can provide their own ratings of the photos. These ratings are aided by the memory

cues the photographs provide. Second, trained or naïve coders can later view these photos

themselves and code/rate them for a wide-variety of cues or characteristics. Coding schemes will

undoubtedly vary depending on the specific questions of interest to the researchers.

Analyzing data. Advances in computer vision have made manually analyzing each and

every single image unnecessary. For example, commercial computer software such as Clarifai

(https://www.clarifai.com/) uses machine vision to detect objects in images. In the study

presented here, these objects—or situation cues (Rauthmann, Sherman, & Funder, 2015) —can

be connected to the psychological characteristics of the situation. Consider a situation where a

participant is spending time with a significant other. This situation might be described by its

psychological characteristics—it is likely to positive and contains some social aspect

(Rauthmann, et al. 2014)—but also by its physical properties (another person is present, a glass

of wine, food, etc.). One goal in this research area is to predict situational characteristics from the

physical aspects gleaned through object recognition technology such as Clarifai (see also Aghaei,

Dimiccoli, & Radeva, 2016).


Figure 4. Plot of obtrusiveness questionnaire means with 95% confidence intervals and medians. N = 28


Furthermore, as described in this article, one of the primary goals of this research was to

understand the processes of situation change. Research in computer science has already begun

using convolution neural networks to automatically segment images captured by wearable

cameras (Bolanos, Mestre, Talavera, Giro-i-Nieto, Radeva, 2015; Doherty et al., 2011). That is,

given a set of images, a computer algorithm can segment the images into situations roughly

similar to the manual process conducted by participants in our study. While no single algorithm

can be expected to perfectly identify objects, or segment situations, the algorithms grow more

precise over time with each additional analysis.

Conclusion

The laboratory experiment and the psychological questionnaire are essential and

indispensable tools in the social/personality scientist’s toolbox. However, standing on the

shoulders of recent technological advances, the time is now right for social/personality scientists

to study people in their natural environments. Experience sampling, unobtrusive eavesdropping,

and mobile sensing have made it easier than ever to understand how people act and interact in

their everyday situations. To these we now add the wearable camera method, which provides

researchers with a window into the natural environments people encounter in their daily lives.

We hope that this article serves as a guide for other social scientists to use wearable camera

methods in their own research.


References

Aghaei, M., Dimiccoli, M., & Radeva, P. (2016). With whom do I interact? Detecting social

interactions in egocentric photo-streams. Presented at the International Conference on

Pattern Recognition. December. Cancun, Mexico.

Barker, R. G., & Wright, H. F. (1951). One boy’s day. A specimen record of behavior. New

York: Harper & Row.

Barta, W. D., Tennen, H., & Litt, M. D. (2012). Measurement reactivity in dairy research. In M.

R. Mehl & T. S. Conner (Eds.), Handbook of Research Methods for Studying Daily Life (pp.

108-123). New York: Guilford Press.

Bolanos, M., Mestre, R., Talavera, E., Giro-i-Nieto, X., & Radeva, P. (2015). Visual summary of

egocentric photostreams by representative keyframes. Paper presented at the 2015 IEEE

International Conference on Multimedia & Expo Workshops (ICMEW). DOI:

http://doi.ieeecomputersociety.org/10.1109/ICMEW.2015.7169863

Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: Capturing life as it is lived. Annual

Review of Psychology, 54, 579-616.

Buss, D. M. (1987). Selection, evocation, and manipulation. Journal of Personality and Social

Psychology, 53, 1214-1221.

Conner, T. S., Tennen, H., Fleeson, W., Barrett, L. (2009). Experience sampling methods: A

modern idiographic approach to personality research. Social Personality Psychology

Compass, 3, 292-313.

Craik, K. (2000). The lived day of an individual: A person-environment perspective. In. W.B.,

Walsh, K.H. Craik, R. H. Price (Eds). Person-Environment Psychology: New Directions

and Perspectives. (pp. 233-266). Mahwah: Lawrence Erlbaum Associates.


Doherty, A. R., Caprani, N., Conaire, C., Kalnikaite, V., Gurrin, C., Smeaton, A. F., &

O’Connor, N. E. (2011). Passively recognising human activities through lifelogging.

Computers in Human Behavior, 27, 1948-1958.

Dufau, S., Duñabeitia, J. A., Moret-Tatay, C., McGonigal, A., Peeters, D., Alario, F. X., …

Grainger, J. (2011). Smart phone, smart science: How the use of smartphones can

revolutionize research in cognitive science. PLoS ONE, 6, 1-3.

Gurrin, C., Smeaton, A. F., & Doherty, A. R. (2014). Lifelogging: Personal big data.

Foundations and Trends in Information Retrieval, 8, 1-107.

Hodges, S., Berry, E., & Wood, K. (2011). SenseCam: A wearable camera that stimulates and

rehabilitates autobiographical memory. Memory, 19, 685-696.

Kahneman, D., Krueger, A. B., Schkade, D. A., Schwarz, N., & Stone, A. A. (2004). A survey

method for characterizing daily life experience: The day reconstruction method. Science,

306(5702), 1776-1780.

Kelly, P., Thomas, E., Doherty, A., Harms, T., Burke, Ó., Gershuny, J., & Foster, C. (2015).

Developing a method to test the validity of 24 hour time use diaries using wearable

cameras: A feability pilot. PLoS ONE, 10: e0142198. doi:10.1371/journal. pone.0142198

Kelley, H. H., & Thibaut, J. W. (1978). Interpersonal relations: A theory of interdependence.

New York, NY: Wiley-Interscience.

Mehl, M. R., & Holleran, S. E. (2007). An empirical analysis of the obtrusiveness of and

participants’ compliance with the Electronically Activated Recorder (EAR). European

Journal of Psychological Assessment, 23, 248-257.

Mehl, M. R., Pennebaker, J. W., Crow, M. D., Dabbs, J., & Price, J. H. (2001). The

electronically activated recorder (EAR): A device for sampling naturalistic daily


activities and conversations. Behavior Research Methods, Instruments, and Computers,

33, 517–523.

Microsoft Corporation (2016). Microsoft Research SenseCam. Retrieved from

http://research.microsoft.com/en-us/um/cambridge/projects/sensecam/

Miller, G. (2012). The smartphone psychology manifesto. Perspective on Psychological Science,

7, 221-237.

Narrative (2016, October 10). Narrative ending sales and support of Narrative Clip [Blog post].

Retrieved from http://blog.getnarrative.com/2016/10/an-open-letter/

Pauly-Takacs, K., Moulin, C. J. A., & Estlin, E. J. (2011). SenseCam as a rehabilitation tool in a

child with anterograde amnesia. Memory, 19, 705-712.

Rauthmann, J. F. (2015). Structuring situation information: A road map of the multiple pathways

to different situational taxonomies. European Psychologist, 20(3), 176-189.

Rauthmann, J. F., Gallardo-Pujol, D., Guillaume, E. M., Todd, E., Nave, C. S., Sherman, R. A.,

Ziegler, M., Jones, A. B., & Funder, D. C. (2014). The situational eight DIAMONDS: A

taxonomy of major dimensions of situation characteristics. Journal of Personality and

Social Psychology, 107, 677-718.

Rauthmann, J. F., Sherman, R. A., & Funder, D. C. (2015). Principles of situation research:

Towards a better understanding of psychological situations. European Journal of

Personality, 29, 363-381.

Reis, H. T. (2012). Why researchers should think “real-world”: A conceptual rationale. In M. R.

Mehl & T. S. Connor (Eds.), Handbook of Research Methods for Studying Daily Life (pp.

3-21). New York: Guilford.


Sandstrom, G. M., Neal, L., Mascolo, C., & Rentfrow, P. J. (in press). Putting mood in context:

Using smartphones to examine how people feel in different locations. Journal of Research in

Personality.

Schlotz, W. (2012). Ambulatory psychoneuroendocrinology: Assessing salivary cortisol and

other hormones in daily life. In M. R. Mehl & T. S. Connor (Eds.), Handbook of Research

Methods for Studying Daily Life. New York: Guilford.

Schönbrodt, F. D., & Perugini, M. (2013). At what sample sizes do correlations stabilize?

Journal of Research in Personality, 47, 609-612.

Scollon, C. S., Kim-Prieto, C., & Diener, E. (2003). Experience sampling: Promises and pitfalls,

strengths and weaknesses. Journal of Happiness Studies, 4, 5-34.

Sherman, R. A. (2014). Situation change: Understanding person-environment transactions in real

life. Grant proposal submitted to the National Science Foundation.

Shiffman, S., Stone, A. A., Hufford, M. R. (2008). Ecological momentary assessment. Annual

Review of Clinical Psychology, 4, 1-32.

Wilhelm, F. H., Grossman, P., & Muller, M. I. (2012). Bridging the gap between the laboratory

and the real world: Integrative ambulatory psychophysiology. In M. R. Mehl & T. S.

Conner (Eds.), Handbook of Research Methods for Studying Daily Life (pp. 210-234).

New York: Guilford Press.

Wilhelm, P., Perrez, M., & Pawlik, K. (2012). Conducting research in daily life: A historical

review. In M. R. Mehl & T. S. Conner (Eds.), Handbook of Research Methods for Studying

Daily Life (pp. 62-86). New York: Guilford Press.

Wolf, P. S. A. & Jacobs, W. J. (2009). GPS technology and human psychological research: A

methodological proposal. Methods and Measurement in the Social Sciences, 1, 1-5.

Wearable Cameras in Psychological Science - in presspsy2.fau.edu/~shermanr/Wearable Cameras in...

Documents

Transcript of Wearable Cameras in Psychological Science - in presspsy2.fau.edu/~shermanr/Wearable Cameras in...