Available online at www.sciencedirect.com

www.elsevier.com/locate/brainres

b r a i n r e s e a r c h 1 5 1 6 ( 2 0 1 3 ) 6 6 – 7 5

0006-8993/$ - see frohttp://dx.doi.org/10.

nCorrespondence toFax: +1 517 353 1652

E-mail address: jm

Research Report

The psychometric properties of the late positivepotential during emotion processing and regulation

Tim P. Morana, Alexander A. Jendrusinab, Jason S. Mosera,n

aMichigan State University, Department of Psychology, MI 48823, United StatesbUniversity of Illinois at Chicago, Department of Psychology, Chicago, IL 60607, United States

a r t i c l e i n f o

Article history:

Accepted 7 April 2013

The late positive potential (LPP) is a commonly used event-related potential (ERP) in the

study of emotion and emotion regulation. The LPP has also been evaluated as a neural

Available online 17 April 2013

Keywords:

Emotion

Emotion regulation

Event-related potentials

Late positive potential

Psychometrics

nt matter & 2013 Elsevie1016/j.brainres.2013.04.01

: Department of Psychology.oser@msu.edu (J.S. Mos

a b s t r a c t

marker of affective psychopathology. The psychometric properties of this component have

not been examined, however. The current study was conducted with the aim of addressing

two questions: how internally consistent is the LPP, and how many trials are necessary to

obtain a stable LPP? Fifty-eight participants completed an emotion regulation task. First,

split-half reliabilities were computed for the LPP and for difference waves revealing

emotion effects (negative minus neutral) and regulation effects (reappraise minus

negative). Second, averages including progressively more trials were evaluated and

compared to overall participant averages. These data indicated good-to-excellent reliability

for neutral, negative and reappraise trials, as well as difference waves. Furthermore, the

LPP varies little after 8 trials are added to the average and the difference waves vary little

after 12 trials are included. Together, the findings of the current study suggest that the LPP

demonstrates good internal consistency and can be adequately quantified with relatively

few trials.

& 2013 Elsevier B.V. All rights reserved.

r B.V. All rights reserved.8

, Michigan State University, Psychology Building, 110-B, East Lansing, MI 48823.

er).

1. Introduction

Psychophysiological measures are critical to the study of basiccognitive/affective processes as well as the specific abnormal-ities underlying psychiatric disorders. However, psychophy-siological measures are only useful to the extent to which theymeet standard psychometric criteria (e.g. Tomarken, 1995).Despite this statistical fact, psychometric properties of psy-chophysiological measures are rarely assessed. The currentstudy aimed to help address this gap in the literature byevaluating the psychometric properties of the late positive

potential (LPP), a commonly-used event-related potentialcomponent in the study of emotion and emotion regulation.Specifically, we examined the internal consistency (split-halfreliability) and stability with increasing trials – that is, thenumber of trials necessary to obtain a stable measure – ofthe LPP.

The LPP is a slow positive deflection in the ERP that developsapproximately 300–400ms post-stimulus and is thought toarise from reciprocal activation of frontal and occipital-parietal regions (Cuthbert et al., 2000; Moratti et al., 2011).Enhanced LPP amplitudes are reliably observed in response to

b r a i n r e s e a r c h 1 5 1 6 ( 2 0 1 3 ) 6 6 – 7 5 67

motivationally significant images. Specifically, emotionallyarousing images elicit a larger LPP compared to neutral images(Cacioppo et al., 1993, 1994; Cuthbert et al., 2000; Keil et al., 2002;Palomba et al., 1997; Schupp et al., 2000, 2003) and thisdifference is sustained over prolonged picture presentations(e.g. 6 s; Cuthbert et al., 2000). Moreover, the LPP elicited byemotional images is highly related to autonomic and self-reported indices of arousal (Cuthbert et al., 2000). Such findingssuggest that the LPP indexes engagement of attentionalresources by motivational systems in service of basic survivalbehaviors (Lang et al., 1997).

The LPP is also sensitive to the implementation of emotionregulation strategies—that is, attempts to modify the inten-sity or time course of emotional experiences (Gross andThompson, 2009; Thompson, 1994). Specifically, the LPPelicited in response to a negative image is attenuated whenparticipants are instructed to reinterpret the image in neutralterms (Hajcak and Nieuwenhuis, 2006; Moser et al., 2006,2009; Moser et al., 2010; Thiruchselvam et al., 2011). Addi-tionally, recent investigations have begun studying aberra-tions in the LPP in groups demonstrating emotion regulationdeficits, including individuals with phobias (Leutgeb et al.,2009), generalized anxiety (Weinberg and Hajcak, 2011;MacNamara and Hajcak, 2010), and depression (Foti et al.,2010; Kujawa et al., in press). Thus, some researchers havesuggested that the LPP may be a useful neural marker fordetecting risk for disruptions in emotion and emotion regula-tion (Dennis and Hajcak, 2009). Together, research points tothe utility of the LPP as a neural marker of emotion proces-sing and regulation in healthy and clinical contexts.

Despite its use in examining emotional processing andemotion regulation, no studies to date have examined theinternal consistency of the LPP. Internal consistency metrics(e.g. intraclass correlations) assess the degree to which ameasure is free from random error due to variation betweentrials and are generally considered to be necessary for estab-lishing an upper bound on the magnitude of correlationsbetween the measure of interest and external measures (e.g.self-reports of anxiety symptoms; Nunnally, 1970; Nunnallyand Bernstein, 1994). Thus, the lack of such an assessment ofthe LPP represents a significant gap in the extant literature.

Indeed, although this has yet to be addressed with the LPP,several investigations have begun evaluating other commonlystudied psychophysiological measures. These studies have reported moderate-to-high reliability for the oddball P3 (Fallgatteret al., 2001; Sandman and Patterson, 2000; Segalowitz andBarnes, 1993; Walhovd and Fjell, 2002; Williams et al., 2005),emotion-modulated electro-myographic (EMG) activity (Larsonet al., 2000, 2005; Lee et al., 2009), the error-related negativity(ERN; Olvet and Hajcak, 2009b; Larson et al., 2010; Segalowitzet al., 2010) and several other stimulus-locked ERPs (i.e. the P2,N2 and P3; Hämmerer et al., 2013). Given the importance ofattaining reliable measures (Nunnally, 1970; Tomarken, 1995),this recent focus on evaluating the reliability of commonlystudied psychophysiological measures is both encouraging andquite valuable as it can facilitate the further refinement of thesemeasures. The aim of the current work was to build on thisfoundation by being the first to conduct such analyses of the LPP.

A second issue arises when one considers the fact thattesting conditions are often not ideal. Artifacts arising during

ERP recordings (e.g. sweat potentials, noisy testing areas, task-irrelevant motor activity, utility frequency contamination etc.)can lead to a significant loss of trials which poses problems forattaining a stable measure. This question has begun to beaddressed with ERPs such as the ERN (Olvet and Hajcak, 2009a)and the oddball P3 (Cohen and Polich, 1997), and this work hashelped provide the field with an empirically-based standardfor including/excluding participants from data analysis. Thesecond aim of the current work, then, was to further providethis basis for the LPP.

The current study was conducted with two main objec-tives. First, we sought to determine the internal consistencyof the LPP during a passive viewing/emotion regulation task.To do this we calculated split-half reliabilities and Pearson'srs (Schmitt, 1994) for LPPs elicited when participants wererequired to naturally view and regulate their emotionalresponse (see methods for more information) to the images.Second, we sought to examine the stability of the LPP withincreasing trials. To do this, we created bins of progressivelygreater numbers of trials (2, 4, 6, 8, 10, 12, 14, and 16) andcomputed several metrics with the aim of examining howmany trials are necessary to obtain a stable LPP (see meth-ods). We also conducted identical analyses on the negativeminus neutral wave (emotion effect) and the reappraiseminus negative view wave (emotion regulation effect) asthese difference waves are often of interest to individual-difference researchers.

2. Results

2.1. Overall

On average, participants had 72.70 (SD¼10.32) trials availablefor analysis. The number of trials did not differ by condition(F(2,114)¼1.6, p¼ .20) as indicated by a repeated-measuresANOVA.

First we sought to establish basic experimental effects(Fig. 1). To do so, we conducted repeated-measures ANOVAswith a single three-level factor (Condition: Neutral vs. Nega-tive vs. Reappraise). We then conducted follow-up t-tests toclarify results obtained in the omnibus analysis.

For the 400–700 ms time window, we observed a signifi-cant effect of condition (F(2,114)¼54.63, po.001, η2p¼ .52) suchthat both negative (t(57)¼9.13, po.001) and reappraise (t(57)¼7.64, po.001) trials elicited larger LPPs than neutral trialsthereby confirming the neutral/negative differentiation.Negative and reappraise trial LPPs did not differ in this timewindow (t(57)¼ .54, p¼ .57). Similar results were obtained forthe 700–1000 ms time window. There was a significant effectof condition (F(2,114)¼49.52, po.001, η2p¼ .49). Both negative(t(57)¼7.93, po.001) and reappraise (t(57)¼7.62, po.001) LPPswere larger than neutral trial LPPs. Negative and reappraisetrial LPPs did not differ in this time window (t(57)¼ .23, p¼ .82).

The test of the 1000–6000 ms time window also revealed asignificant effect of condition (F(2,114)¼8.03, po.001, η2p¼ .14).In this time window, however, both neutral (t(57)¼4.68,po.001) and reappraise (t(57)¼2.17, p¼ .03) trial LPPs weresmaller than negative trial LPPs. Neutral and reappraise trial

Fig. 1 – Top Panels: The grand average of the LPP by condition (Left) and the grand average of the difference waves (Right).

Bottom Panels: The LPP computed from 8 trials (Left) and the difference waves computed from 12 trials (Right).

b r a i n r e s e a r c h 1 5 1 6 ( 2 0 1 3 ) 6 6 – 7 568

LPPs did not differ (t(57)¼1.62, p¼ .10). Thus, participantssuccessfully reappraised negative images.

2.2. 400–700 ms Time window

2.2.1. Internal consistencyPearson's rs and ICC's are presented in Table 1. Split-halfreliability metrics were good-to-excellent for all conditionsand difference waves.

2.2.2. Mean differences between successive trial binsSuccessive bins are depicted in Fig. 2. The omnibus analysesdid not detect a significant effect of bin on mean amplitudesfor either the LPP or difference waves (Fso1.4, ps4.15).

2.2.3. Stability with increasing trialsFig. 3 displays Cronbach's α as more trials were added to theaverage. High reliability for mean amplitudes (α4.7) for allconditions was achieved when at least 8 trials were includedin the average and α surpassed .8 when 12 trials were includedin the average. Fig. 4 displays Pearson's r values for each trialbin. Correlations ranged between .4 and .97 and achieved .5 by4 trials, .8 by 8 trials and .9 by 16 trials; po.01 for all pairs.

For difference waves, high reliability (α4.7) for both com-parisons was achieved when at least 12 trials were included inthe average and α surpassed .8 when 14 trials were included inthe average. Correlations ranged between .09 and .91 andachieved .5 by 8 trials, and .8 with 12–14 trials; po.05 for allpairs with at least 4 trials.

Thus, for the 400–700 ms time window, LPPs appearedrelatively stable after the inclusion of 8 trials and differencewaves appeared stable after the inclusion of 12 trials.

2.3. 700–1000 ms Time window

2.3.1. Internal consistencyPearson's rs and ICC's are presented in Table 1. Split-halfreliability metrics were good-to-excellent for all conditions anddifference waves.

2.3.2. Mean differences between successive trial binsThe omnibus analyses did not detect a significant effect ofbin for either the LPP or difference waves (Fso1.2, ps4.4).

2.3.3. Stability with increasing trialsFig. 3 displays Cronbach's α as more trials are added to theaverage. For mean amplitudes, high reliability (α4.7) for allconditions was achieved when at least 8 trials were includedin the average and α surpassed .8 when 14 trials were includedin the average. Fig. 4 displays Pearson's r values for each bin.Correlations ranged between .25 and .95; rs reached .5 oncebins included at least 4 trials, .8 by 10–12 trials and .9 by 16trials. All correlations conducted on bins with at least 4 trialsreached significance (pso.001).

For difference waves, both comparisons reached high relia-bility (α4.7) by 10 trials, α surpassed .8 when 12–14 trials wereincluded in the average. Correlations ranged between .08 and.91; rs reached .5 once bins included at least 8 trials, .8 by 12trials and .9 by 16 trials. All correlations conducted on bins withat least 6 trials reached significance (pso.05).

Thus, for the 700–1000ms time window, LPPs appearedrelatively stable after the inclusion of 8 trials and differencewaves appeared stable after the inclusion of 10 trials.

Table 1 – Pearson’ r and ICC values for the LPP as a function of condition and time window.

Condition Time window (ms) Pearson’s r ICC

Neutral 400–700 .79* .88*

700–1000 .76* .87*

1000–6000 .72* .84*

Negative 400–700 .77* .89*

700–1000 .74* .83*

1000–6000 .66* .80*

Reappraise 400–700 .77* .87*

700–1000 .76* .87*

1000–6000 .67* .81*

Negative minus neutral 400–700 .58* .72*

700–1000 .60* .71*

1000–6000 .57* .69*

Reappraise minus negative 400–700 .56* .71*

700–1000 .55* .71*

1000–6000 .54* .68*

n po.001.

b r a i n r e s e a r c h 1 5 1 6 ( 2 0 1 3 ) 6 6 – 7 5 69

2.4. 1000–6000 ms Time window

2.4.1. Internal consistencyPerson's rs and ICC's are presented in Table 1. Split-halfreliability metrics were good-to-excellent for all conditionsand difference waves.

2.4.2. Mean differences between successive trial binsThe omnibus analysis did not detect a significant effect of bin(Fso1.7, ps4.10).

2.4.3. Stability with increasing trialsFig. 3 displays Cronbach's α as more trials are added to theaverage. High reliability (α4.7) for all conditions was achievedwhen 6 trials were included in the average and α surpassed .8when at least 10 trials were included in the average. Fig. 4displays Pearson's r values for each bin. Correlations rangedbetween .14 and .96; rs reached .5 once at least 6 trials wereincluded in the average, and .8 by the inclusion of 8 trials.All correlations including bins with at least 4 trials reachedsignificance (pso.01).

For difference waves, both comparisons reached highreliability (α4.7) by 12 trials, α surpassed .8 when 16 trialswere included in the average. Correlations ranged between.16 and .91; rs reached .5 once bins included at least 6 trialsand .8 by the inclusion of 12–14 trials. All correlationsconducted on bins with at least 6 trials reached significance(pso.001).

Thus, for the 1000–6000 ms time window, LPPs appearedrelatively stable after the inclusion of 6 trials and differencewaves appeared stable after the inclusion of 12 trials.

3. Discussion

The present study was conducted with the aim of examiningcertain psychometric properties of the LPP. Specifically, weevaluated the internal consistency and stability with increasingtrials of LPPs recorded from 58 participants during the perfor-mance of a passive viewing/emotion regulation task. The LPP

and difference waves showed good-to-excellent split-half relia-bility across all conditions and time windows (Anastasi, 1998;Cicchetti, 2001; Cicchetti and Sparrow, 1981). Our findingsindicate that the LPP shows similar reliabilities to thosereported for other psychophysiological measures (e.g. the ERNand P3; as well as greater internal consistency than EMGactivity) and shows sufficiently high internal consistency asto be considered reliable during a recording session. Addition-ally, the internal consistency of the LPP is sufficient to warrantits continued use in the study of individual differences inemotion processing/regulation (Leutgeb et al., 2009; Weinbergand Hajcak, 2011; MacNamara and Hajcak, 2010; Foti et al.,2010; Moser, 2012; Kujawa et al., in press) as well as the study ofthe LPP as a neural marker for affective psychopathology(Dennis and Hajcak, 2009).

With respect to stability with increasing trials, the successivebinning procedure indicated that when mean amplitudes arebased on 8 trials all indices suggest that the LPP has becomerelatively stable. In addition to the LPP, we also examinedmultiple difference waves. As is common for difference waves(Luck, 2005), the negative minus neutral and reappraise minusnegative waves showed somewhat greater variability. Reliabilitymetrics indicated high stability for all conditions/windowswhen averages included at least 12 trials. The current findingsprovide multiple sources of converging evidence indicating thatit is possible to compute a stable LPP with as few as 8 trials anda stable difference wave with 12 trials during a passive viewing/emotion regulation task. When these criteria are applied to ourentire sample (N¼87; see methods), 79 participants retained anadequate number of trials in each condition. Thus, thesecriteria resulted in the loss of less than 10% of participants.

This study is part of a small but growing body of researchexamining the psychometric properties of commonly studiedpsychophysiological measures. Using methodologies similar tothose presented here, Olvet and Hajcak (2009a,2009b) demon-strated that the error-related negativity can be quantified withrelatively few trials (6) and shows high internal consistency(4.7). Similar results (rs4.6) have been reported by Larsonet al. (2010) and Segalowitz et al. (2010). Relatedly, the oddballP3 waveform, an ERP thought to index similar processes as

Fig. 2 – The mean of the LPP and difference waves for each of the successive bins the grand average (GA) depicted separately

for each condition and time window.

1In order to examine the effect that a different baseline wouldhave on our results, we also analyzed the data using a −200–0 msbaseline. The findings were comparable to those presented in thetext. With respect to the binning procedure, results indicated thatstable amplitudes (α4.7; r4.5) were achieved after 8 trials for theLPP and 12 trials for the difference waves. For internal consis-tency, ICC's ranged between .78 and .89 for LPPs and .68 and .73for difference waves.

b r a i n r e s e a r c h 1 5 1 6 ( 2 0 1 3 ) 6 6 – 7 570

the LPP, shows comparable reliability to those reported here(4.6; Fallgatter et al., 2001; Sandman and Patterson, 2000;Segalowitz and Barnes, 1993; Walhovd and Fjell, 2002;Williams et al., 2005; however, it should be noted that thesestudies largely focused on test–retest reliability) and can bequantified with as few as 20 trials (Cohen and Polich, 1997).Finally, Davidson and colleagues (Larson et al., 2000, 2005; Leeet al., 2009) have demonstrated moderate reliability (4.5) forEMG activity and startle modulation recorded during emotionprocessing and regulation. In comparison, the current findingssuggest that the LPP may be more a reliable indicator ofemotion processing and regulation than EMG-related mea-sures. The current paper expands upon such work in threemain ways. First, it is the first to examine the stability of theLPP during emotion processing and regulation. Second, weknow of no other study to examine the internal consistency of

LPP during such a task. Finally, we examined the character-istics of the LPP during performance of a passive viewing/regulation task during which affective stimuli were presentedand no overt response was required—rather than duringan oddball task consisting of neutral stimuli that require aresponse (as is the case for studies of the P3).

One potential limit on the generalizability of the currentinvestigation is our choice of data preparation parameters1

Fig. 3 – Cronbach’s alpha for the LPP and difference waves for each of the successive bins depicted separately for each

condition and time window.

b r a i n r e s e a r c h 1 5 1 6 ( 2 0 1 3 ) 6 6 – 7 5 71

(e.g. the particular low-pass cutoff we used). Although ourchoice of filtering cutoffs, artifact rejection parameters andthe use of the ocular correction method developed by Grattonet al. (1983) are common in the literature, such choices do varybetween labs. It should be noted, however, that our parameterswere fairly liberal (e.g. we could have chosen a stricter low-passcutoff, such as 20 Hz) and the use of more stringent cutoffswould have likely resulted in less noisy/more reliable data.Additionally, while Pz is a commonly-used electrode site in theliterature, there is inter-lab variability; CPz is also commonlyused. However, as the activity recorded at Pz and CPz werehighly related (rs ranged between .91 and .96), it is likely that anexamination of CPz would have yielded similar results. Simi-larly, some researchers choose to pool across multiple electrodesites; such methods attenuate electrode-specific noise andtherefore would also tend to produce less noisy/more reliabledata. Finally, the current study employed a set of active

electrodes (Biosemi, The Netherlands). Although active elec-trode montages are becoming an increasingly popular methodof shielding, they are not ubiquitous in the literature. It will beinteresting for future studies to compare such results acrossactive and passive electrode sets. Overall, we believe ourfindings provide a conservative estimate for the internal con-sistency and necessary number of trials for the LPP and areapplicable to a wide range of data preparation strategies.

The current study examined the reliability of the LPPduring performance of a passive viewing/emotion regulationtask. We examined multiple metrics of reliability on severalmeasures of interest (i.e. LPP and difference waves) acrossmultiple time windows in order to provide convergentsources of evidence. The data indicate that a reliable LPPcan be achieved with as few as 8 trials—12 trials for differ-ence waves. Future research should consider these findingswhen planning inclusion/exclusion criteria; that is, our

Fig. 4 – Pearson’s r for the LPP and difference waves for each of the successive bins depicted separately for each condition and

time window.

b r a i n r e s e a r c h 1 5 1 6 ( 2 0 1 3 ) 6 6 – 7 572

results provide an empirically-based standard for inclusion ina study (cf. Olvet and Hajcak, 2009a; 2009b). The data alsoindicated good-to-excellent internal consistency for the LPPfor all conditions and time windows (ICCs≥.68). The currentstudy focused on cognitive reappraisal—a regulation strategywhich involves reinterpreting negative stimuli in a moreneutral manner (e.g. Gross and Thompson, 2009; Moseret al., 2006). Other regulation strategies, such as emotionalsuppression (Murata et al., in press) and deploying attentionto non-arousing areas of images (Dunning and Hajcak, 2009a;2009b), have also been shown to modulate the LPP. A focus offuture research should be to determine if reliability/stabilityare similar across regulation strategies. Relatedly, this studyfocused on examining the LPP in response to neutral andnegative IAPs images; whether these results can be general-ized to a wider range of images (i.e. other neutral/negative

IAPS as well as positive images) remains to be seen. Finally,future studies might further examine the reliability of the LPPby measuring its test–retest reliability over extended periodsof time.

4. Experimental procedures

4.1. Participants and stimuli

Eighty-seven participants completed a passive viewing/emo-tion regulation task. Only participants who retained at least20 trials were included in order to ensure that a sufficientnumber of trials were available for the binning procedure (seerejection criteria below; N¼58; 36 female). No participantsdiscontinued their involvement after the experiment began.

b r a i n r e s e a r c h 1 5 1 6 ( 2 0 1 3 ) 6 6 – 7 5 73

The stimulus set consisted of 60 images2 taken from theInternational Affective Picture System (IAPS; Lang, et al., 1999):30 neutral, low-arousal images (M normative valence¼4.92; Mnormative arousal¼2.85) and 30 negative, high-arousal images(M normative valence¼2.30; M normative arousal¼6.37).

The task was administered on a Pentium D class compu-ter, using Presentation software (Neurobehavioral Systems,Inc., Albany, CA) to control the presentation and timing of allstimuli. Each picture was displayed in color and occupied theentirety of a 19 in. (48.26 cm) monitor. Participants were seatedapproximately 60 cm from the monitor.

4.2. Procedure

For each trial, participants first viewed an instruction phrase(“REAPPRAISE NEGATIVE”, “LOOK NEGATIVE”, or “LOOK NEU-TRAL”—referred to as “reappraise,” “negative,” and “neutral,”respectively, in subsequent sections) for 2 s that directed themhow to respond to the following picture. The “REAPPRAISENEGATIVE” phrase indicated that the participant should engagein cognitive reappraisal. Specifically, participants were told toimagine that the pictured scene improved and to think of theimage in a more positive light so as to decrease the intensity oftheir negative emotions (Ochsner et al., 2004). The participantswere further told not to generate unrelated thoughts or imagesto decrease their responses. The instruction phrases “LOOKNEGATIVE” and “LOOK NEUTRAL” indicated that the partici-pant should respond naturally to the presented negative andneutral images, respectively. For all instructions, participantswere told to view the pictures for the entire display period andto not look away or close their eyes. After the instruction phrasewas presented, a blank screen was presented for 500msfollowed by a centrally presented white fixation cross lasting500ms. Following the fixation cross, the IAPS images weredisplayed for 6 s. A period of 2.5 s was inserted between theoffset of images and the presentation of the next instructionphrase during which time participants were instructed to relaxand clear their minds.

Participants first completed two practice blocks of the task tofamiliarize themselves with the timing of events and instruc-tions. The experimental task then included 90 cue-picture trials:30 reappraise, 30 negative, and 30 neutral trials presented in arandom order. The 30 negative IAPS images were used for boththe reappraise and negative trials as reappraisal effects andneutral/negative differentiation have been successfully demon-strated in tasks using similar methods (e.g. Moser et al., 2009;Hajcak and Nieuwenhuis, 2006; Dennis and Hajcak, 2009).Previous research further suggests that repeated viewings ofthe same image is not associated with habituation of theemotion effect (Olofsson and Polich, 2007).

2This study included the following IAPS images. Neutral: 2190,2200, 2210, 2230, 2570, 2840, 5500, 5531, 7000, 7002, 7009, 7010,7020, 7025, 7035, 7050, 7080, 7100, 7150, 7160, 7170. 7175, 7190,7217, 7224, 7233, 7235, 7550, 7700, 5950, 9070. Negative: 2688, 6312,6313, 6825, 9425, 9428, 9620, 9622, 9908, 3181, 3350, 3507052, 0,3530, 6212, 6821, 2683, 2811, 3301, 6550, 6520, 8485, 9050, 9183,9414, 6242, 6231, 6230, 3170, 3220, 9903.

4.3. Psychophysiological recording and data reduction

Continuous electroencephalographic (EEG) activity wasrecorded using the ActiveTwo Biosemi system (Biosemi,Amsterdam, The Netherlands). Recordings were takenfrom 64Ag–AgCl electrodes embedded in a stretch-lycracap. Additionally, two electrodes were placed on the leftand right mastoids. Electro-oculogram (EOG) activity gen-erated by eye-movements and blinks was recorded at FP1and three additional electrodes placed inferior to the leftpupil and on the left and right outer canthi. During dataacquisition, the Common Mode Sense active electrode andDriven Right Leg passive electrode formed the ground perBiosemi's design specifications. The function of the CMS-DRL loop, in addition to forming a reference, is simply toconstrain the common mode voltage (i.e. the averagevoltage of the participant) which limits the amount ofcurrent that can possibly return to the participant. Allsignals were digitized at 512 Hz and an online lowpassfilter was applied with a cutoff of 104 Hz.

Offline analyses were performed using BrainVision Ana-lyzer 2 (BrainProducts, Gilching, Germany). Scalp electroderecordings were re-referenced to the mean of the mastoidsand band-pass filtered (cutoffs: 0.01–30 Hz; 12 dB/oct roll-off). Ocular artifacts were corrected using the methoddeveloped by Gratton et al. (1983). Stimulus-locked datawere segmented into individual epochs beginning 500 msbefore cue and picture onset and continuing for 6 s. Picture-locked epochs were computed separately for reappraise,negative, and neutral trials. Physiologic artifacts weredetected using a computer-based algorithm such that trialsin which the following criteria were met were rejected: avoltage step exceeding 50 μV between contiguous samplingpoints, a voltage difference of 300 μV within a trial, and amaximum voltage difference of less than 0.5 μV within100 ms intervals.

The LPP was quantified at electrode site Pz. First, abaseline equal to the average activity in the −500–0 mspre-picture window was subtracted from each data pointsubsequent to stimulus onset. Then, as is common is LPPresearch (e.g. Cuthbert et al., 2000; Moser et al., 2006;Krompinger et al., 2008; Dennis and Hajcak, 2009; Murataet al., in press) the LPP was quantified in multiple windowsaround its peak amplitude as well as during thesustained positivity which continues throughout picturepresentation: specifically, we defined the LPP as the aver-age activity in three successive time windows followingpicture presentation: 400–700 ms, 700–1000 ms, and 1000–6000 ms3.

4.4. Data analysis procedures

The LPP was statistically evaluated using SPSS software (v 20.0).We first computed two averages based on even- and odd-

3In similar studies, other authors have also included peakamplitudes and latencies in addition to mean amplitudes. How-ever, as peak amplitudes and latencies are rarely, if ever,quantified for the affective LPP, this investigation will focus onmean amplitudes.

b r a i n r e s e a r c h 1 5 1 6 ( 2 0 1 3 ) 6 6 – 7 574

numbered subsets of available trials for each condition and timewindow. We then calculated Pearson's rs and intraclass correla-tion coefficients (ICCs) between both subsets. Interpretations ofICC values vary, but several authors have suggested that ICCso.4are to be considered poor; values ranging between .41 and .59 aremoderate, between .6 and .74 are adequate/good and valuesexceeding .75 are excellent (Cicchetti and Sparrow, 1981;Anastasi, 1998; Cicchetti, 2001). As these metrics were computedon averages using half of available trials, the Spearman–Browncorrection was applied (Spearman, 1910; Brown, 1910;Helmstadter, 1964). The focus of the current investigation wasthe strength of the relationships between bins and the partici-pant's average; therefore, we will focus mainly on themagnitudeof the associations. However, p-values will be presented for thesake of completeness4.

With respect to stability with increasing trials, weinitially generated bins of increasingly greater numbersof trials (2, 4, 6, 8, 10, 12, 14, and 16 trials; for a similarmethod, see Cohen and Polich, 1997; Olvet and Hajcak,2009a; Larson et al., 2010) separately for each condition andtime window; trials were randomly selected for inclusion.We then evaluated mean amplitudes using a single-factorrepeated measures ANOVA (Number of Trials: 2, 4, 6, 8, 10,12, 14, 16, grand average). Follow-up t-tests were con-ducted on successive bins (2 vs. 4, 4 vs. 6, 6 vs. 8, 8 vs. 10,10 vs. 12, 12 vs. 14, 14 vs. 16, 16 vs. grand average) when theomnibus analysis indicated a significant result. As we wereattempting to determine the number of trials at whichdifferences ceased to be significant, we did not apply aBonferroni correction to attenuate family-wise error. Uti-lizing a more stringent p-value would result in non-significant differences between bins with fewer trials thanif .05 were used. Thus, such a correction would result in aless, not more, conservative estimate of the necessarynumber of trials.

To further examine the stability of LPP amplitudes withincreasing numbers of trials, we calculated Pearson's rsbetween each bin and the grand average as well asCronbach's α for each bin (for a similar method, seeOlvet and Hajcak, 2009a). Hinton et al. (2004) suggests thata Cronbach's α below .5 is to be interpreted as poor, .5–.7 asmoderate, .7–.9 as high, and values exceeding .9 are to beconsidered excellent. With respect to Pearson's r, valuesexceeding .5 are considered acceptable (Helmstadter, 1964;Segalowitz et al., 2010).

4.5. Supplementary analyses

In addition to the analyses presented here, we also examinedestimates of signal-to-noise ratios (SNR) for the LPP. Specificdetails and results can be found in supplementary materials.

4We are reporting r's and ICC's as descriptive, rather thaninferential, statistics. However, for the interested reader, a corre-lation exceeding approxima45 would still be considered signifi-cant after conducting a Bonferroni correction.

Appendix A. Supplementary material

Supplementary data associated with this article can be foundin the online version at http://dx.doi.org/10.1016/j.brainres.2013.04.018.

r e f e r e n c e s

Anastasi, A., 1998. Psychological Testing, sixth ed. Macmillan,New York: NY.

Brown, W., 1910. Some experimental results in the correlation ofmental abilities. Br. J. Psychol. 3, 296–322.

Cacioppo, J.T., Crites Jr., S.L., Bernston, G.G., Coles, M.G.H., 1993. Ifattitudes affect how stimuli are processed, should they notaffect the event-related brain potential?. Psychol. Sci. 1,108–112.

Cacioppo, J.T., Crites Jr., S.L., Gardner, W.L., Bernston, G.G., 1994.Bioelectrical echoes from evaluative categorizations: I. A latepositive brain potential that varies as a function of traitnegativity and extremity. J. Pers. Soc. Psychol. 67, 115–125.

Cicchetti, D.V., 2001. The precision of reliability and validityestimates re-visited: distinguishing between clinical andstatistical significance of sample size requirements. J. Clin.Exp. Neuropsychol. 23, 695–700.

Cicchetti, D.V., Sparrow, S.A., 1981. Developing criteria forestablishing interrater reliability of specific items: applicationsto assessment of adaptive behavior. Am. J. Ment. Def. 86,127–137.

Cohen, J., Polich, J., 1997. On the number of trials needed for P300.Int. J. Psychophysiol. 25, 249–255.

Cuthbert, B.N., Schupp, H.T., Bradley, M.M., Birbaumer, N., Lang,P.J., 2000. Brain potentials in affective picture processing:covariation with autonomic arousal and affective report. Biol.Psychol. 52, 95–111.

Dennis, T., Hajcak, G., 2009. The late positive potential: aneurophysiological marker for emotion regulation in children.J. Child Psychol. Psychiatry 50, 1373–1383.

Dunning, J.P., Hajcak, G., 2009. See no evil: directing visualattention within unpleasant images modulates theelectrocotical response. Psychophysiol. 46, 28–33.

Fallgatter, A.J., Bartsch, A.J., Strik, W.K., Mueller, T.J., Eisenack, S.S.,Neuhauser, B., et al., 2001. Test–retest reliability ofelectrophysiological parameters related to cognitive motorcontrol. Clin. Neurophysiol. 112, 198–204.

Foti, D., Olvet, D.M., Klein, D.N., Hajcak, G., 2010. Reducedelectrocortical response to threatening faces in majordepressive disorder. Depress. Anxiety 27, 813–820.

Gratton, G., Coles, M.G.H., Donchin, E., 1983. A new method foroff-line removal of ocular artifact. Electroencephol. Clin.Neurophysiol. 55, 468–484.

Gross, J.J., Thompson, R.A., 2009. Emotion regulation: conceptualfoundations. In: Gross, J.J. (Ed.), Handbook of EmotionRegulation. The Guilford Press, New York, NY.

Hajcak, G., Nieuwenhuis, S., 2006. Reappraisal modulates theelectrocortical response to unpleasant pictures. Cogn, Aff,Behav. Neurosci. 6, 291–297.

Hämmerer, D., Li, S.-C., Völkle, M., Müller, V., Lindenberger, U.,2013. A lifespan comparison of the reliability, test–reteststability, and signal-to-noise ratio of event-related potentialsassessed during performance monitoring. Psychophysiology50, 111–122.

Helmstadter, G.C., 1964. Principles of Psychological Measurement.Appleton-Century-Crofts, New York: NY.

b r a i n r e s e a r c h 1 5 1 6 ( 2 0 1 3 ) 6 6 – 7 5 75

Hinton, P.R., Brownlow, C., McMurray, I., Cozens, B., 2004. SPSSExplained. Routledge, New York: NY.

Keil, A., Bradley, M.M., Hauk, O., Rockstroh, B., Elbert, T., Lang, P.J.,2002. Large-scale neural correlates of affective pictureprocessing. Psychophysiology 39, 641–649.

Krompinger, J.W., Moser, J.S., Simons, R.F., 2008. Modulations ofthe electrophysiological response to pleasant stimuli bycognitive reappraisal. Emo 8, 132–137.

Kujawa, K., Hajcak, G., Torpey, D.C., Kim, J., Klein, D.N. (in press).Electrocortical reactivity to emotional faces in young childrenand associations with maternal and paternal depression.J. Child Psychol. Psychiatry.

Lang, P.J., Bradley, M.M., Cuthbert, B.N., 1997. Motivated attention:affect, activation and action. In: Lang, P.J., Simons, R.F.,Balaban, M.T. (Eds.), Attention and Orienting: Sensory andMotivational Processes. Lawrence Erlbaum Associates,MahWah: NJ, pp. 97–135.

Lang, P.J., Bradley, M.M., Cuthbert, B.N., 1999. InternationalAffective Picture System: Instruction Manual and AffectiveRatings. Technical Report A-4. Gainsville, FL: The Center forResearch in Psychophysiology, University of Florida.

Larson, M.J., Baldwin, S.C., Good, D.A., Fair, J.E., 2010. Temporalstability of the error-related negativity (ERN) and post-errorpositivity (Pe): the role of number of trials. Psychophysiology47, 1167–1171.

Larson, C.L., Ruffalo, D., Nietert, J.Y., Davidson, R.J., 2000.Temporal stability of the emotion-modulated startle response.Psychophysiology 37, 92–101.

Larson, C.L., Ruffalo, D., Nietert, J.Y., Davidson, R.J., 2005. Stabilityof emotion-modulated startle during short and long picturepresentation. Psychophysiology 42, 604–610.

Lee, H., Shackman, A.J., Jackson, D.C., Davidson, R.J., 2009.Test–retest reliability of voluntary emotion regulation.Psychophysiology 46, 874–879.

Leutgeb, V., Schäfer, A., Schienle, A., 2009. An event-relatedpotential study on exposure therapy for patients sufferingfrom spider phobia. Biol. Psychol. 82, 293-300.

Luck, S.J., 2005. The Event-Related Potential Technique. MITPress, Cambridge, MA.

MacNamara, A., Hajcak, G., 2010. Distinct electrocortical andbehavioral evidence for increased attention tothreat in generalized anxiety disorder. Depress. Anxiety 27,234–243.

Moratti, S., Saugar, C., Strange, B.A., 2011. Prefrontal-occipitoparietal coupling underlies late latency humanneuronal responses to emotion. J. Cogn. Neurosci. 31,17278–17286.

Moser, J.S., Hajcak, G., Bukay, E., Simon,s, R.F., 2006. Intentionalmodulation of emotional responding to unpleasant pictures:an ERP study. Psychophysiology 43, 292–296.

Moser, J.S., Krompinger, J.W., Dietz, J., Simons, R.F., 2009.Electrophysiological correlates of decreasing and increasingemotional responses to unpleasant pictures.Psychophysiology 46, 17–27.

Moser, J.S., Most, S.B., Simons, R.F., 2010. Increasing negativeemotions be reappraisal enhances subsequentcognitive control: a combined behavioral andelectrophysiological study. Cogn. Affect. Behav. Neurosci. 10,195–207.

Murata, A., Moser, J.S., Kityama, S. Culture shapes electrocorticalresponses during emotion suppression. Soc. Cogn. Aff.Neurosci. in press.

Nunnally, J.C., 1970. Introduction to Psychological Measurement.McGraw-Hill, NY.

Nunnally, J.C., Bernstein, I.H., 1994. Psychometric Theory, thirded. McGraw-Hill, NY.

Olofsson, J.K., Polich, J., 2007. Affective visual event-relatedpotentials: arousal, repetition, and time-on-task. Biol. Psychol.75, 101–108.

Olvet, D.M., Hajcak, G., 2009a. The stability of error-related brainactivity with increasing trials. Psychophysiology 46, 957–961.

Olvet, D.M., Hajcak, G., 2009b. Reliability of error-related brainactivity. Brain Res. 1284, 89–99.

Ochsner, K.N., Ray, R.D., Cooper, J.C., Robertson, E.R., Chorpa, S.,Gabrieli, J.D.E., 2004. For better or for worse: neural systemssupporting the cognitive down- and up-regulation of negativeemotion. NeuroImage. 23, 483–499.

Palomba, D., Angrilli, A., Mini, A., 1997. Visual evoked potentials,heart rate responses and memory to emotional pictorialstimuli. Int. J. Psychophysiol. 27, 55–67.

Sandman, C.A., Patterson, J.V., 2000. The auditory event-relatedpotential is a stable and reliable measure in elderly subjectsover a 3 year period. Clin. Neurophysiol. 111, 1427–1437.

Schmitt, N., 1994. Uses and abuses of coefficient alpha. Psychol.Assess. 8, 350–353.

Schupp, H.T., Cuthbert, B.N., Bradley, M.M., Cacioppo, J.T., Ito, T.,Lang, P.J., 2000. Affective picture processing: the late positivepotential is modulated by motivational relevance.Psychophysiology 37, 257–261.

Schupp, H.T., Junghöfer, M., Weike, A.I., Hamm, A.O., 2003.Emotional facilitation of sensory processing in the visualcortex. Psychol. Sci. 14, 7–13.

Segalowitz, S.J., Barnes, K.L., 1993. The reliability of ERPcomponents in the auditory oddball paradigm.Psychophysiology 30, 451–459.

Segalowitz, S.J., Santesso, D.L., Murphy, T.I., Homan, D.,Chantziantoniou, D.K., Khan, S., 2010. Retest reliability ofmedial frontal negativities during performance monitoring.Psychophysiology 47, 260–270.

Spearman, C.C., 1910. Correlation calculated from faulty data. Br.J. Psychol. 3, 271–295.

Thiruchselvam, R., Blechert, J., Sheppes, G., Rydstrom, A., Gross, J.J.,2011. The temporal dynamics of emotion regulation: an EEGstudy of distraction and reappraisal. Biol. Psychol. 87, 84–92.

Thompson, R.A., 1994. Emotion regulation: a theme in search ofdefinition. Monogr. Soc. Res. Child Dev. 59, 25.

Tomarken, A.J., 1995. A psychometric perspective onpsychophysiological measures. Psychol. Assess. 7, 387–395.

Walhovd, K.B., Fjell, A.M., 2002. One-year test–retest reliability ofauditory ERPs in young and old adults. Int. J. Psychophysiol. 46,29–40.

Williams, L.M., Simms, E., Clark, C.R., Paul, R.H., Rowe, D., Gordon, E.,2005. The test–retest reliability of a standardized neurocognitiveand neurophysiological test battery: “neuromarker”. Int. J.Neurosci. 115, 1605–1630.

Weinberg, A., Hajcak, G., 2011. Electrocortical evidence forvigilance then avoidance in generalized anxiety disorder: anevent-related potential study. Psychophysiology 48, 842–851.