Psychiatry Research 210 (2013) 82–88

Contents lists available at ScienceDirect

Psychiatry Research

0165-17http://d

n CorrE-m1 Jo

journal homepage: www.elsevier.com/locate/psychres

Detecting improvements in acute psychotic symptoms usingexperience sampling methodology

Suzanne Ho-wai So a,n, Emmanuelle Roisin Peters b,c, Joel Swendsen d,Philippa Anne Garety b,c,1, Shitij Kapur c,e,f,1

a Department of Psychology, The Chinese University of Hong Kong, Shatin, New Territories, Room 321 Wong Foo Yuan Building, Hong Kong SAR, Chinab King's College London, Institute of Psychiatry, Department of Psychology, London, United Kingdomc NIHR Biomedical Research Centre for Mental Health, South London and Maudsley NHS Trust, United Kingdomd University of Bordeaux, National Center for Scientific Research, Bordeaux, Francee King's College London, Institute of Psychiatry, Section on Schizophrenia, Imaging and Therapeutics, London, United Kingdomf King's College London, Institute of Psychiatry, Department of Psychological Medicine, London, United Kingdom

a r t i c l e i n f o

Article history:Received 26 October 2012Received in revised form9 May 2013Accepted 12 May 2013

Keywords:PsychosisAntipsychotic treatmentEcological momentary assessmentMobile device signaling

81/$ - see front matter & 2013 Elsevier Irelanx.doi.org/10.1016/j.psychres.2013.05.010

esponding author. Tel.: +852 3943 6211; fax:ail address: shwso@psy.cuhk.edu.hk (S.H. So).int last authors.

a b s t r a c t

This study aimed to explore the feasibility and validity of using experience sampling methodology(ESM, or ecological momentary assessment or mobile device signaling) to measure temporal changes andfluctuations in psychotic symptoms in patients with acute psychosis at the start of antipsychotictreatment.

Twenty-six in-patients with delusions were assessed within 2 weeks of starting antipsychotic treat-ment using ESM on a personal digital assistant (PDA), seven times a day for 14 consecutive days. Theywere also interviewed at baseline, 1 week and 2 weeks after using standardized symptom measures.

Sixteen patients (61.5%) completed at least one-third of the entries, with a compliance rate of 70.7%.Responses to the ESM items were internally consistent. At baseline, ESM and clinical ratings convergedon suspiciousness and images, but not on voices and most of the delusion dimensions.

Conducting ESM with patients in an acute episode was found to be feasible and internally valid. Thereis some divergence in symptom data obtained by ESM and standard symptom interviews, but ESMcaptures rich information about change that may not be represented by observer ratings.

& 2013 Elsevier Ireland Ltd. All rights reserved.

1. Introduction

Improvement in psychotic symptoms, including delusions, maytake place rapidly following the start of antipsychotic treatment.Meta-analyses have shown that symptoms begin to abate withinthe first week of treatment, and that the improvement during thefirst 2 weeks is much greater than in subsequent weeks (Agidet al., 2003; Leucht et al., 2005). In fact, there are some suggestionsthat improvement in psychotic symptoms may be seen in the veryfirst few hours (Agid et al., 2006). Therefore, examining changes inpsychosis during the very early phase of antipsychotic treatment isimportant.

Existing interview-based assessments of psychotic symptomsmeasure experiences over the past week and are not designedto assess more rapid changes. Therefore, reliable methods for

d Ltd. All rights reserved.

+852 2603 5019.

assessing early and rapid changes in symptoms in a more detailedmanner (e.g., including delusion dimensions) are needed.

1.1. Experience sampling methodology

The experience sampling method (ESM), also known as eco-logical momentary assessment and mobile device signaling(Collins et al., 1998; Shiffman et al., 2008; Stone and Shiffman,2002), is a structured diary technique, assessing current contextand psychological phenomena (and their interactions) in the flowof daily life. Prompted by a signal emitted from a wristwatch orother electronic devices, at irregular intervals throughout the day,participants are asked to fill out self-assessment forms (Wilhelmet al., 2012). ESM assessment has high ecological validity, allowsfor variability across beeps, days, and individuals to be assessed,and minimizes the possibility of observer bias and socially desir-able answers (Csikszentmihalyi et al., 1977; Delespaul, 1995; Trulland Ebner-Priemer, 2009).

With the advance in software packages, recent studies haveconducted ESM electronically using personal digital assistants

S.H.-w. So et al. / Psychiatry Research 210 (2013) 82–88 83

(PDAs), mobile phones, and “PsyMate” (Granholm et al., 2008; Kimhyet al., 2006, 2010; Le et al., 2006; Myin-Germeys et al., 2011; Schareret al., 2002a, 2002b). Computerized ESM is advantageous because itreduces the chance of human error as data are not entered manually(Barrett and Barrett, 2001) and time-stamped data are stored in thedevice's memory (Conner Christensen et al., 2003).

ESM has been applied with success in studies covering a widevariety of psychiatric conditions (e.g. Ben-Zeev et al., 2012;Hintzen et al., 2010; see reviews by Trull et al., 2012; Myin-Germeys, 2012). Previous applications of ESM have demonstratedfeasibility among patients with schizophrenia (reviewed by Myin-Germeys et al., 2003, 2009; Oorschot et al., 2009). Satisfactorycompliance rates have been reported in studies with outpatientswith psychosis (60–87%; Granholm et al., 2008; Kimhy andCorcoran, 2008; Peters et al., 2012) and in-patients with schizo-phrenia (79–81%; Kimhy et al., 2006, 2010). Patients have alsorated the experience of ESM assessment positively (Granholmet al., 2008; Kimhy and Corcoran, 2008; Kimhy et al., 2006).

Previous studies of psychosis have typically investigated valid-ity of ESM in two ways. The first is to examine within-personcorrelations between ESM variables to ascertain if they areinternally consistent. Granholm et al. (2008) reported that asso-ciations between perceived level of environmental stress andnegative feelings were as predicted, and Kimhy et al. (2006) alsofound a negative association between ESM assessments of positiveand negative affect. The second is to examine correlations betweenESM scores and established clinical ratings of the same construct.While Granholm et al. (2008) reported that ESM items onpsychotic symptoms were positively associated with the Positiveand Negative Syndrome Scale (PANSS; Kay et al., 1987) positivesymptom subscale, Kimhy et al. (2006) did not find a significantassociation between ESM and the Schedule for the Assessmentof Positive Symptoms (SAPS; Andreasen, 1984) on suspiciousness.Peters et al. (2012) also reported weak associations betweenESM and the Psychotic Symptom Rating Scales (PSYRATS;Haddock et al., 1999) on dimensions of hallucinations anddelusions.

Other more sophisticated approaches of analyzing validity andsensitivity of change in ESM include the Generalisabilty Theorymodel (Cranford et al., 2006), the factor-analytic model (Lane andShrout, 2011), and a multi-level modeling approach distinguishingitem variances and covariance between and within individuals(Wilhelm and Schoebi, 2007). These methods are good for deli-neating the extent to which variance is attributed to respectivelevel of measurement (person, day, construct, etc.). However, alarge number of participants and observations are often requiredfor data to be divided into several levels (e.g. N¼187 in Wilhelmand Schoebi, 2007), and these methods make assumptions that arelikely to be violated, such as constant variance and equal relevanceof specific items to the final score.

In summary, ESM is potentially a well-suited method forunderstanding fluctuations in psychological phenomena in theflow of daily life and can be used in patients suffering from severemental illnesses such as psychosis. While previous ESM studiesincluded patients with delusions (e.g. Peters et al., 2012;Thewissen et al., 2008, 2011), the present study was the first todate that included only in-patients during an acute episode, at thebeginning of antipsychotic treatment i.e. within the first fewweeks of admission before they were stabilised on medication.

1.2. The present study

This study aimed to explore the feasibility and validity of usingESM as a method to measure temporal changes in psychoticsymptoms over the first 2 weeks of hospital admission in anacutely ill psychotic sample.

Our major hypotheses were:

1.

Feasibility—Patients in an acute episode will be able to com-plete the 14-day ESM assessment with a compliance ratecomparable to previous studies.

2.

Internal validity—ESM ratings will be internally consistent. 3. Convergent validity—ESM scores will correlate with more

standard measures of psychotic symptoms at baseline.

4. Sensitivity to change—Changes over time in psychotic symp-

toms will be detectable using ESM, and will reflect interviewer-rated change.

2. Method

2.1. Participants

Ethical approval for the study was granted by the South EastLondon Research Ethics Committee 4. In-patients with acutedelusions (scoring 4 or above on at least one of the PANSS delusionitems) and a clinical diagnosis (based on clinical notes) of schizo-phrenia spectrum disorder or bipolar disorder were recruited.Patients with drug-induced psychosis or organic psychosis, andpatients with a primary diagnosis of substance misuse wereexcluded. Participants were recruited as soon as possible afterstarting or restarting antipsychotic treatment. Patients had beenfree from antipsychotics for at least 1 month prior to the currentadmission. The first assessment took place within a 2 weekswindow post treatment commencement for the current episode.

A total of 68 acute inpatients were contacted, among whom 26consented to participate in this study. Half of the sample was maleand the mean age was 36.12 years (range 20–63 years). Psychiatricdiagnoses from the case notes were available for 22 patients asfollows: schizophrenia (n¼9), unspecified psychosis (n¼5), mooddisorder with psychotic symptoms (n¼3), schizoaffective disorder(n¼2), schizophreniform disorder (n¼1), delusional disorder (n¼1),and acute and transient psychotic disorder (n¼1). Except for thepatient who was recruited from a community assessment andtreatment service, 25 participants were recruited from locked wardsfor patients with acute and severe mental illnesses (eight were fromfemale in-patient units, four from male in-patient units, and 13 frommixed in-patient units). The average number of admissions (includ-ing the current one) was 1.72 (S.D.¼1.54; range¼1–7), and themean length of current in-patient was 10.70 days (S.D.¼10.07 days,range¼1–36 days) before the baseline interview.

The average number of days on antipsychotics at the timeof the first assessment was 4.91 (S.D.¼2.93; range 0–13); 20 of the26 patients were assessed within the first week of treatment.Baseline mean scores on the PANSS were as follows (N¼26):positive¼20.88 (S.D.¼4.19), negative¼11.85 (S.D.¼5.38), general¼31.50 (S.D.¼9.04), total¼64.23 (S.D.¼14.41), indicating a “mildlyill” to “moderately ill” level of severity (Leucht et al., 2005).The average baseline rating of delusion on the SAPS was 3.96(S.D.¼0.53), indicating a moderate to marked level of severity(Andreasen, 1984). Information on medication was available for 21patients, out of whom 20 were on atypicals (Risperidone, Olanza-pine, Aripiprazole, Quetiapine, Amisulpiride, and Clozapine), whileone was on typical antipsychotic (Piportil). The mean starting doseof antipsychotics in chlorpromazine equivalents (Andreasen et al.,2010) was 95.48 mg/day (S.D.¼93.69).

2.2. Measures

2.2.1. ESM itemsSince the sample consisted of patients in an acute psychotic

episode who had not been stabilized with antipsychotics, care was

Table 1ESM items.

Section Question

Environment The following question concerns where you are right now.1. Where are you right now?

Affect The following questions ask about how you are feeling at this moment.2. How cheerful do you feel right now?3. How irritated do you feel right now?4. How relaxed do you feel right now?5. How content do you feel right now?6. How low do you feel right now?7. How tense do you feel right now?

Psychotic symptoms The following questions ask about your thoughts or experiences at this moment.8. Other than conversations with other people, do you hear voices right now?9. Do you see images right now?10. How suspicious do you feel right now?11. How well can you concentrate right now?12. How safe do you feel right now?

Delusion The following questions concern the problem discussed with your researcher, in particular the thought or idea that … (to be suffixed as agreedat the baseline interview)13. At this moment, to what extent do you believe this concern is true?14. At this moment, how much does this concern upset you?15. At this moment, to what extent does this concern go round and round in your mind?16. At this moment, to what extent does this concern interfere with what you are doing?

Company The following questions concern who you are with right now.17. Who is with you right now?

Activity The following questions concern the most significant or important activity you performed since the last signal.18. Which of your activities do you feel was the most important?19. To what extent did you find this activity pleasant?20. To what extent did you find this activity stressful?

Reactivity to ESM(only in the last diary of theday)

Please give us your opinion about responding to these questionnaires today.21. To what extent did completing these questionnaires influenceyour mood today?22. To what extent did responding to these questions change your regular activities today?23. To what extent was this an ordinary day for you?

S.H.-w. So et al. / Psychiatry Research 210 (2013) 82–8884

taken to ensure the self-assessment forms were easy to complete,with each electronic interview taking no longer than 3 min tocomplete. The ESM items were organized into sections on positiveand negative affect, psychotic symptoms, and delusion dimensions(see Table 1 for the list of items). The name of the sections did notappear in the questionnaire, and labels such as delusions andhallucinations were not used in the questionnaire or interviews.Each section began with a statement introducing the generalcontent of subsequent questions. Each diary consisted of the sameset of questions, with three additional questions at the last reportof each day about the participant's reactivity to conducting theESM assessment.

The ESM items on psychotic symptoms and affect were deter-mined with reference to previous ESM studies (e.g. Kimhy et al.,2006; Myin-Germeys et al., 2001a, 2001b, 2005). Consistent withthe circumplex model of affect (Larsen and Diener, 1992; Russell,2003), mood items were selected to measure a diversity of statescomposing negative (feeling low, tense or irritated) and positive(feeling cheerful, relaxed or content) dimensions. Delusions weremeasured multi-dimensionally, as suggested by factor analyticstudies (e.g. Kendler et al., 1983). The delusion dimensions (con-viction, distress, preoccupation, disruption) chosen were the sameas those in PSYRATS and Peters et al. (2012). The exact wording ofthe items was individualized according to each participant'sdelusional belief, as agreed in the baseline interview. Only onebelief was selected and rated throughout the study period.

As suggested by Palmier-Claus et al. (2011), items on contextualinformation (e.g. ‘where are you right now?’) were also included toensure that the participant was not solely focusing on his/hersymptoms and that the diary was not only about emotionallysalient information. Both positively and negatively worded items(e.g. pleasant, stressful) were used.

The full list of items was reviewed in detail by a panel ofexperts (PG; SK; EP; JS). Wording for some items was changed and

the items were re-ordered so that the assessment began with asituational question to draw the participant's attention to thehere-and-now and ended with questions involving evaluation ofthe activity preceding the beep. The revised version was thenpiloted with two individuals from the general population, afterwhich wording of the items was further simplified to easecomprehension. The final list of items is shown in Table 1.

2.2.2. ESM protocol and equipmentIn order to obtain time-stamped data and avoid back-filling of

reports (Broderick et al., 2003; Stone et al., 2003), the assessmentwas conducted electronically on a personal digital assistant(PDA). The modified version of the Purdue Momentary Assess-ment Tool software (PMAT; version 2.1.2) (Weiss et al., 2004) wasused to present questions and collect responses on the PDAs.PMAT has been used to program PDAs in other ESM studiesof schizophrenia (e.g. Ben-Zeev et al., 2010; Granholm et al.,2008). PMAT is easy to use after training from an experiencedresearcher (JS).

A signal-contingent protocol was used where participants wereasked to complete a diary entry by a signal at various timesthroughout the day. Participants could choose between an early(9 am–9 pm) or late (11 am–11 pm) schedule. In order to capturevarious moments during the day and to avoid clustering ofresponses, the 12 target hours each day were divided into sevenblocks of approximately 90 min. One signaling time was selectedfor every block using random numbers generated by the computer,with the proviso that no signals should occur within 20 min ofeach other. The PDA program permitted responses to be providedonly within 20 min following the signal (and 2 min following anygiven question). If participants did not complete the report withinthis time window, the PDA was programmed to turn off auto-matically until the next activation.

Fig. 1. PMAT screenshots.

S.H.-w. So et al. / Psychiatry Research 210 (2013) 82–88 85

Most of the items were rated on seven-point Likert scaleswhere the participant had to indicate their responses on agraphical slider (see Fig. 1). For contextual questions, the partici-pants were asked to respond by checking one of the several boxes(see Fig. 1). In order to save the battery, all other functions/programs in the PDA were disabled during the study period.

2.2.3. Symptomatology and delusion dimensionsThe Schedule for the Assessment of Positive Symptoms

(SAPS; Andreasen, 1984), and the Positive and Negative SyndromeScale (PANSS; Kay et al., 1987) were included at the baselineand week 2 interviews. The Psychotic Symptom Rating Scales(PSYRATS; Haddock et al., 1999) was conducted at baseline, week1 and week 2. All the clinical rating scales were administered by thesame researcher (SHS), who was a Hong Kong qualified clinicalpsychologist and had received training from experts in psychiatryand clinical psychologists who had extensive experience in con-ducting clinical interviews with psychiatric patients. The assessingresearcher did not have knowledge of the participant's ESM scoreswhen conducting the interviews.

2.3. Procedures

Participants were interviewed as soon as they began antipsy-chotic treatment (baseline), 1 week later, and again 2 weeks later.During that time they were asked to complete the ESM assessmentfor 14 consecutive days. At the baseline interview, the researcheragreed with the participant on the index delusional belief (labeledas “the problem or main concern”) to be rated on the delusion-related ESM questions. After the baseline assessment, the partici-pant was trained to conduct the computerized ESM procedure,including the operation of the PDA and the meaning of allquestions and response choices. The researcher completed at leastone practice diary entry together with the participant, emphasizedthe importance of compliance, and discussed typical situations inwhich this might be difficult (e.g. in a shower).

Once the participants expressed confidence in completing theassessment on their own, the individually designed questionnairewas then programmed onto a Palm Tungsten E2 PDA (Palm OSs

version 5.2.1). Participants were given the PDA to carry with them

and the ESM measurements began immediately after. The max-imum number of potential signals a participant would receive was98 (seven signals per day for 14 days). Previous studies have setthe criterion that participants' data should be excluded from dataanalysis if they completed fewer than 33% of the experiencesampling activations (Delespaul, 1995; Myin-Germeys et al.,2001a, 2001b). In this study, the minimum compliance criterionwas rounded to 30 assessments.

To ensure that participants understood and complied with theprocedure, they were contacted by the researcher at least twice inthe first week to offer support and to remind them to charge thebattery. Individuals who demonstrated difficulty in understandingassessment questions or operating the device were given addi-tional training. Participants were encouraged to contact the re-searcher by phone if they encountered any problems during theassessment period.

2.4. Statistical analysis

Internal consistency and divergent validity of the ESM assess-ment were tested by multi-level linear regression modeling. Multi-level linear regression modeling was used because the ESM datasets contain repeated observations nested within participants, andthese analyses adjust for dependencies among observations gen-erated by each individual. Regression models were tested usingthe multi-level XTREG command in STATA 10 (StataCorp, 2007).For analysis of internal consistency, associations among thepositive affect items and among the negative affect items across14 days were analyzed respectively. For analysis of divergentvalidity, associations between items of opposite affect (i.e. Cheerfulvs. Low; Relaxed vs. Tense) across 14 days were analyzed.

To examine the convergent validity of ESM, associationsbetween the ESM scores of hallucinations and delusions and thecorrespondent clinical ratings on Day 1 were tested using multi-level linear regression modeling, with each of the ESM symptomscores as dependent variables respectively and scores on theclinical scales as independent variables.

To examine change over the 2-week period, clinical ratings ofpsychotic symptoms at baseline and at week 2 were comparedusing within-sample t-tests on the Statistical Package for theSocial Sciences (SPSS) 15.0 (SPSS, 2006), whereas ESM ratings on

Table 2Mean levels (S.D.) of symptom ratings at baseline and week 2 (N¼16).

Baseline Week 2 Paired-samplet-test

S.H.-w. So et al. / Psychiatry Research 210 (2013) 82–8886

psychotic symptoms over 14 days were analyzed using multi-levellinear regression modeling, with each of the symptoms as depen-dent variables respectively and day as an independent variable.Day was entered as a continuous variable (1–14).

PANSS positive 20.63 (4.52) 16.25 (3.72) t¼4.01, po0.01PANSS negative 11.88 (5.24) 10.00 (3.58) t¼1.57, p¼0.17PANSS general 33.25 (10.51) 24.75 (5.16) t¼3.78, po0.01PANSS total 65.75 (16.10) 51.00 (10.91) t¼4.06, po0.01PSYRAT delusions 17.25 (3.92) 13.69 (3.63) t¼3.20, po0.01SAPS delusions 4.00 (0.52) 3.13 (1.15) t¼3.22, po0.01PANSS delusions 5.38 (1.03) 4.31 (1.01) t¼3.44, po0.01PANSS suspiciousness 5.39 (1.15) 4.06 (1.70) t¼1.33, p¼0.01PANSS hallucinatorybehavior

3.19 (1.94) 2.31 (1.70) t¼3.05, p¼0.01

PSYRATS auditory hallucinations 13.38 (18.02) 9.06 (14.24) t¼2.23, p¼0.04SAPS voices 2.19 (2.29) 1.63 (2.36) t¼1.78, p¼0.10SAPS images 0.63 (1.03) 0.88 (1.71) t¼-0.85, p¼0.41

Fig. 2. Changes in severity of psychotic symptoms on ESM (N¼16).

3. Results

3.1. Feasibility of the ESM assessment

Out of the 26 consented participants, five refused to completethe ESM assessment after 1 day and discontinued from the study.Reasons for refusal were as follows: the beeps were annoying, thePDA or eye glasses went missing, physical illness, and beingparanoid about the PDA. Sixteen participants met the minimumcompliance criterion, completing 30 or more diary entries. Therewas no significant difference between the 16 participants who metthe minimum compliance requirement and the 10 participantswho did not in age, duration and dosage of medication, number ofadmissions, and all scale scores in PANSS, SAPS and PSYRATS(p40.10).

Among the 16 participants who completed ESM, the meannumber of entries per participant was 59 (range 34–89) out of apotential maximum of 98. The mean rate of compliance over 14days was 70.7% (range 40.2–94.6%). Related-samples WilcoxonSigned Rank Test revealed no difference (p40.10) in compliancerate between Day 1 (mean¼84.9%, S.D.¼30.5%) and Day 14(mean¼74.3%, S.D.¼24.1%). The total number of observationsavailable for multi-level models was 1306.

In response to the question “To what extent did completingthese questionnaires influence your mood today?”, the averagerating was 3.40 (S.D.¼2.11) on a 1–7 point scale. To the question“To what extent did responding to these questions change yourregular activities today?”, the average rating was 2.98 (S.D.¼2.23)on a 1–7 point scale. Fatigue effects were examined using aBernoulli model with the logit-link function with missing data asdependent variable, and Day as independent variable. There wasno significant change in missing data over time (β¼0.02, p40.10).

3.2. Validity of ESM

3.2.1. Internal consistencyMulti-level linear regression modeling revealed a significant

and positive association between the three positive affectitems—Cheerful and Relaxed (β¼0.60, p o0.001), Cheerful andContent (β¼0.54, p o0.001), and Relaxed and Content (β¼0.61,po0.001), and between the three negative affect items—Irritatedand Low (β¼0.51, p o0.001), Irritated and Tense (β¼0.49,po0.001), and Low and Tense (β¼0.52, p o0.001).

3.2.2. Divergent validityMulti-level linear regression modeling revealed significant

and negative associations between Cheerful and Low (β¼−0.09,po0.01) and between Relaxed and Tense (β¼−0.09, po0.01).

3.2.3. Convergent validityConvergent validity of ESM was examined using ESM data on

Day 1 and clinical ratings on the same day, an approach comparableto Kimhy et al. (2006).

Multi-level regression modeling revealed that the ESMSuspiciousness item was significantly associated with the SAPSPersecutory delusions item (β¼0.53, p ¼0.02) and the PANSSSuspiciousness item (β¼0.47, p ¼0.03). The ESM rating of Imageswas also significantly associated with the Visual hallucination itemon SAPS (β¼0.69, p o0.01). However, the ESM rating of Voiceswas not related to the Auditory hallucination items on SAPS

(β¼0.10, p ¼0.54) or PSYRATS (β¼0.02, p ¼0.44). ESM ratings ofdelusion dimensions were significantly associated with PSYRATSratings on Disruption only (β¼1.79, p ¼0.01; Conviction: β¼−0.97,p ¼0.22; Distress: β¼−0.49, p ¼0.10; Preoccupation: β¼0.15,p ¼0.80).

3.3. Changes over 2 weeks

As shown in Table 2, paired sample t-tests revealed a significantimprovement in PANSS positive, general and total scores betweenbaseline and week 2. There was an improvement in delusions(including suspiciousness) shown across ratings scales, whereasimprovement in auditory hallucinations was significant for PANSSand PSYRATS, but not SAPS. Change in visual hallucinations wasnot significant.

In contrast, multi-level regression modeling of the ESM ratings(see Fig. 2) showed a significant increase (i.e. more severe) overtime in Voices (β¼0.03, p ¼ .02) and Images (β¼0.03, p ¼ .01), butno significant change in Suspiciousness (p40.10).

4. Discussion

This study found that measuring changes in an acute psychoticepisode at the beginning of treatment using ESM was feasible andinternally consistent. The predictions that ESM and clinical ratingswould converge with each other were only partially supported.

S.H.-w. So et al. / Psychiatry Research 210 (2013) 82–88 87

ESM with acutely ill patients was challenging yet ultimatelyfeasible. This ratio of participation (26 out of 68, 38.2%) was higherthan Peters et al. (2012, 14.6%), and lower than Granholm et al.(2008, 96.4%) and Kimhy et al. (2006, 84.6%). Our relatively lowratio of participation may be partially explained by the fact that wetargeted patients who were not yet stabilized by medication in theacute psychiatric wards, whereas Granholm et al. (2008) recruitedfrom out-patient clinics. However, Peters et al. (2012) reported aneven lower participation rate in their out-patient sample. Sincethe information about the patients who refused to take part inour study was not available, it is not possible to systematicallycompare the patients who consented and those who refused.Peters et al. (2012) reported no difference in demographic orclinical characteristics between their participants and refusers.On the other hand, Peters et al. (2012) attributed their lowparticipation rate to the high demands of the study (five roundsof 1-week long ESM assessment over 1 year). Indeed, our study(seven times a day for 14 days) is less demanding than that ofPeters et al. (2012) but more demanding than those of Granholmet al. (2008, four times per day for a week) and Kimhy et al. (2006,10 times over 1 day only). Therefore, it is more likely that rate ofparticipation is affected by the demands of the study than theclinical status of the participants.

Despite the difficulty in recruitment, compliance rate (70.7%) inthis study was comparable to previous studies that involved lessdemanding assessment schedules. Responses to the positive andnegative affect items were internally consistent. The association ofopposite constructs in a predicted direction confirms that partici-pants were able to provide meaningful and reliable responses toESM questionnaires on the PDA.

Findings of convergent validity and sensitivity to change werenot unequivocal. At baseline, clinical ratings at baseline wererelated to ESM ratings on suspiciousness and visual hallucinations,but not on auditory hallucinations and most of the delusiondimensions. When changes over a 2-week period were examined,there was even less consistency between measures. Mixed resultsof convergence between ESM and clinical ratings have also beenreported in other studies—Granholm et al. (2008) found goodconvergence between ESM measures of psychotic symptoms andPANSS Positive subscale, but they did not examine correlations forspecific symptoms and the two sets of measures covered differentperiods of time. In studies where convergence between measuresfor intensity of specific symptoms was examined, weak correla-tions were reported by Kimhy et al. (2006) and Peters et al. (2012).The only other study that had examined delusion dimensions(Peters et al., 2012) reported no systematic correlation betweenESM and clinical ratings.

It is of interest to speculate why convergence between mea-sures was not strong. One possibility is that the time epochcovered by the ESM measures was different from that coveredby clinical ratings. While clinical ratings in the baseline interviewassessed symptomatology over the previous week, ESM measuredthe experience in the moment. This may have weakened thecorrelation between the two methods, especially for symptomsthat occurred infrequently and would have been captured byretrospective measures rather than assessment of the moment.In Palmier-Claus et al. (2012), where participants were asked toreport on a mobile phone whether they experienced psychoticsymptoms between two signals, they reported a higher level ofcorrelation between the momentary measures and clinical ratings.

Therefore, it is possible that the lack of congruence betweenESM and interviewer-rated clinical measures does not necessarilyindicate a lack of validity of ESM; rather, it is likely that the twoapproaches are assessing different aspects of the same psychoticpresentation. Clinical ratings require a reflection of the partici-pant's experiences over the last week, whereas ESM represents the

participant's thoughts and feelings at the time of assessment.Apart from the difference in time frame, clinical ratings drawon episodic memory extensively whereas ESM requires an instan-taneous response and minimal input from episodic memory.As individuals with schizophrenia commonly experience episodicmemory difficulties, the discrepancy between ESM and interview-based ratings may be partly attributed to such difficulties. The self-report nature of ESM also makes it different from interviewer-rated clinical scales. The use of both ESM and clinical ratings maytherefore provide a more comprehensive assessment of both theretrospective and momentary experiences and viewpoints fromthe person as well as the observer.

There were a number of limitations to this study. Firstly,changes over time were analyzed with day modeled as a contin-uous variable. The assumption of a linear effect of time was notexplicitly tested, and alternative nonlinear models may be possi-ble. Secondly, since this study was conducted during the period oftime a new medication regime began, medications were titratedup or down for some patients. However, the study design did notmanipulate the medication regime, and the effect of medicationtitration was not systematically measured or analyzed. Against thiscaveat, the present study found that it is feasible to assess acutepsychotic symptoms, including delusion dimensions, in the earlyphase of antipsychotic treatment using ESM. Its combination ofmoment-by-moment symptom ratings with situational variableshas the potential for in-depth monitoring of subtle changes andcontextual factors in the acute psychotic state.

Acknowledgments

This study was supported by the Croucher Foundation Scholar-ship and the University of London Central Research Fund toSHS. PG and EP acknowledge support for some clinical sessionsfrom the National Institute of Mental Health (NIHR) BiomedicalResearch Centre for Mental Health, King's Health Partners.

References

Agid, O., Kapur, S., Arenovich, T., Zipursky, R., 2003. Delayed-onset hypothesis ofantipsychotic action: a hypothesis tested and rejected. Archives of GeneralPsychiatry 60, 1228–1235.

Agid, O., Seeman, P., Kapur, S., 2006. The “delayed onset” of antipsychotic action—an idea whose time has come and gone. Journal of Psychiatry and Neuros-ciences 31, 93–100.

Andreasen, N., 1984. Scale for the Assessment of Positive Symptoms (SAPS).Department of Psychiatry, University of Iowa College of Medicine, Iowa City.

Andreasen, N.C., Pressler, M., Nopoulos, P., Miller, D., Ho, B.C., 2010. Antipsychoticdose equivalents and dose-years: a standardized method for comparingexposure to different drugs. Biological Psychiatry 67, 255–262.

Barrett, L., Barrett, D., 2001. An introduction to computerized experience samplingin psychology. Social Science Computer Review 2, 175–185.

Ben-Zeev, D., Ellington, K., Swendsen, J., Granholm, G., 2010. Examining a cognitivemodel of persecutory ideation in the daily life of people with schizophrenia: acomputerized experience sampling study. Schizophrenia Bulletin 37,1248–1256.

Ben-Zeev, D., Young, M.A., Depp, C.A., 2012. Real-time predictors of suicidalideation: mobile assessment of hospitalized depressed patients. PsychiatryResearch 197, 55–59.

Broderick, J., Schwartz, J., Shiffman, S., Hufford, M., Stone, A., 2003. Signaling doesnot adequately improve diary compliance. Annals of Behavioral Medicine 26,139–148.

Collins, R., Morsheimer, E., Shiffman, S., Paty, J., Gnys, M., Papandonatos, G., 1998.Ecological momentary assessment in a behavioral drinking moderation trainingprogram. Experimental and Clinical Psychopharmacology 6, 306–315.

Conner Christensen, T., Barrett, L., Bliss-Moreau, E., Lebo, K., Kaschub, C., 2003.A practical guide to experience-sampling procedures. Journal of HappinessStudies 1, 53–78.

Cranford, J.A., Shrout, P.E., Iida, M., Rafaeli, E., Yip, T., Bolger, N., 2006. A procedurefor evaluating sensitivity to within-person change: Can mood measures indiary studies detect change reliably? Personality and Social Psychology Bulletin32, 917–929.

Csikszentmihalyi, M., Larson, R., Prescott, S., 1977. The ecology of adolescentactivity and experience. Journal of Youth and Adolescence 6, 281–294.

S.H.-w. So et al. / Psychiatry Research 210 (2013) 82–8888

Delespaul, P., 1995. Assessing Schizophrenia in Daily Life. Universitaire PersMaastricht, Maastricht.

Granholm, E., Loh, C., Swendsen, J., 2008. Feasibility and validity of computerizedecological momentary assessment in schizophrenia. Schizophrenia Bulletin 34,507–514.

Haddock, G., McCarron, J., Tarrier, N., Faragher, E., 1999. Scales to measuredimensions of hallucinations and delusions: the psychotic symptom ratingscales (PSYRATS). Psychological Medicine 29, 879–889.

Hintzen, A., Delespaul, P., van Os, J., Myin-Germeys, I., 2010. Social needs in daily lifein adults with Pervasive Developmental Disorders. Psychiatry Research 179,75–80.

Kay, S., Opler, L., Fiszbein, A., 1987. Positive and Negative Syndrome Scale (PANSS)Rating Manual. Social and Behavioral Sciences Documents, San Rafael, CA.

Kendler, K., Glazer, W., Morgenstern, H., 1983. Dimensions of delusional experience.American Journal of Psychiatry 140, 466–469.

Kimhy, D., Corcoran, C., 2008. Use of Palm computer as an adjunct to cognitive-behavioural therapy with an ultra-high-risk patient: a case report. EarlyIntervention in Psychiatry 2, 234–241.

Kimhy, D., Delespaul, P., Ahn, H., Cai, S., Shikhman, M., Lieberman, J., Malaspina, D.,Sloan, R., 2010. Concurrent measurement of “real-world” stress and arousal inindividuals with psychosis: assessing the feasibility and validity of a novelmethodology. Schizophrenia Bulletin 36, 1131–1139.

Kimhy, D., Delespaul, P., Corcoran, C., Ahn, H., Yale, S., Malaspina, D., 2006.Computerized experience sampling method (ESMc): assessing feasibility andvalidity among individuals with schizophrenia. Journal of Psychiatric Research40, 221–230.

Lane, S.P., Shrout, P.E., 2011. Measuring the reliability of within-person change overtime: a dynamic factor analysis approach. Technical Report No. 11.1. Availablefrom New York University Couples Lab website at ⟨www.psych.nyu.edu/couples/Reports/11.01_Lane_&_Shrout.pdf⟩.

Larsen, R.J., Diener, E., 1992. Promises and problems with the circumplex model ofemotion. In: Clark, M.S. (Ed.), Review of Personality and Social Psychology, vol.13. Sage, Newbury Park, CA, pp. 25–59.

Le, B., Choi, H., Beal, D., 2006. Pocket-sized psychology studies: exploring dailydiary software for palm pilots. Behavior Research Methods 38, 325–332.

Leucht, S., Busch, R., Hamann, J., Kissling, W., Kane, J., 2005. Early-onset hypothesisof antipsychotic drug action: a hypothesis tested, confirmed and extended.Biological Psychiatry 57, 1543–1549.

Myin-Germeys, I., 2012. Psychiatry. In: Mehl, M.R., Conner, T.S. (Eds.), Handbook ofResearch Methods for Studying Daily Life. The Guilford Press, New York,pp. 636–650.

Myin-Germeys, I., Birchwood, M., Kwapil, T., 2011. From environment to therapy inpsychosis: a real-world momentary assessment approach. Schizophrenia Bul-letin 37, 244–247.

Myin-Germeys, I., Delespaul, P., van Os, J., 2003. The experience sampling methodin psychosis research. Current Opinion in Psychiatry 16 (Suppl. 2), S33–S38.

Myin-Germeys, I., Delespaul, P., van Os, J., 2005. Behavioural sensitization to dailylife stress in psychosis. Psychological Medicine 35, 733–741.

Myin-Germeys, I., Nicolson, N., Delespaul, P., 2001a. The context of delusionalexperiences in the daily life of patients with schizophrenia. PsychologicalMedicine 31, 489–498.

Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespau, P., van Os, J., 2009.Experience sampling research in psychopathology: opening the black box ofdaily life. Psychological Medicine: A Journal of Research in Psychiatry and theAllied Sciences 39, 1533–1547.

Myin-Germeys, I., van Os, J., Schwartz, J., Stone, A., Delespaul, P., 2001b. Emotionalreactivity to daily life stress in psychosis. Archives of General Psychiatry 58,1137–1144.

Oorschot, M., Kwapil, T., Delespaul, P., Myin-Germeys, I., 2009. Momentary assess-ment research in psychosis. Psychological Assessment 21, 498–505.

Palmier-Claus, J.E., Ainsworth, J., Machin, M., Barrowclough, C., Dunn, G., Barkus, E.,Rogers, A., Wykes, T., Kapur, S., Buchan, I., Salter, E., Lewis, S.W., 2012.The feasibility and validity of ambulatory self-report of psychotic symptomsusing a smartphone software application. BMC Psychiatry 12, 172.

Palmier-Claus, J., Myin-Germeys, I., Barkus, E., Bentley, L., Udachina, A., Delespaul, P.,Lewis, S., Dunn, G., 2011. Experience sampling research in individuals withmental illness: reflections and guidance. Acta Psychiatrica Scandinavica 123,12–20.

Peters, E., Lataster, T., Greenwood, K., Kuipers, E., Scott, J., Williams, S., Garety, P.,Myin-Germeys, I., 2012. Appraisals, psychotic symptoms and affect in daily life.Psychological Medicine 42, 1013–1023.

Russell, J.A., 2003. Core affect and the psychological construction of emotion.Psychological Review 110, 145–172.

Scharer, L., Hartweg, V., Hoern, M., Graesslin, Y., Strobl, N., Frey, S., Biedermann, C.,Walser, S., Walden, J., 2002a. Electronic diary for bipolar patients. Neuropsy-chobiology 46 (Suppl. 1), 10–12.

Scharer, L., Hartweg, V., Valerius, G., Graf, M., Hoern, M., Biedermann, C., Walser, S.,Boensch, A., Dittmann, S., Forsthoff, A., Hummel, B., Grunze, H., Walden, J.,2002b. Life charts on a palmtop computer: first results of a feasibility studywith an electronic diary for bipolar patients. Bipolar Disorder 4 (Suppl. 1),107–108.

Shiffman, S., Stone, A., Hufford, M., 2008. Ecological momentary assessment. AnnualReview of Clinical Psychology 4, 1–32.

SPSS, 2006. Statistical Package for the Social Sciences (SPSS) (Version 15.0)[Computer software]. SPSS Inc, Chicago, Illinois.

StataCorp, 2007. Stata Statistical Software: Release 10 [Computer software].StataCorp LP, College Station, TX.

Stone, A., Shiffman, S., 2002. Capturing momentary, self-report data: a proposal forreporting guidelines. Annals of Behavioral Medicine 24, 236–243.

Stone, A., Shiffman, S., Schwartz, J., Broderick, J., Hufford, M., 2003. Patientcompliance with paper and electronic diaries. Controlled Clinical Trials 24,182–199.

Thewissen, V., Bentall, R.P., Lecomte, T., van Os, J., Myin-Germeys, I., 2008.Fluctuations in self-esteem and paranoia in the context of daily life. Journalof Abnormal Psychology 117, 143–153.

Thewissen, V., Bentall, R.P., Oorschot, M., à Campo, J., van Lierop, T., van Os, J., Myin-Germeys, I, 2011. Emotions, self-esteem, and paranoid episodes: an experiencesampling study. British Journal of Clinical Psychology 50, 178–195.

Trull, T., Ebner-Priemer, U., 2009. Using experience sampling methods/ecologicalmomentary assessment (ESM/EMA) in clinical assessment and clinicalresearch: Introduction to the special section. Psychological Assessment 21,457–462.

Trull, T.J., Ebner-Priemer, U.W., Brown, W.C., Tomko, R.L., Scheiderer, E.M., 2012.Clinical psychology. In: Mehl, M.R., Conner, T.S. (Eds.), Handbook of ResearchMethods for Studying Daily Life. The Guilford Press, New York, pp. 620–635.

Weiss, H., Beal, D., Lucy, S., MacDermid, S., 2004. Purdue Momentary AssessmentTool Version 2.1.2. Military Family Research Institute, Purdue University, WestLafayette, IN.

Wilhelm, P., Perrez, M., Pawlik, K., 2012. Conducting research in daily life:a historical review. In: Mehl, M.R., Conner, T.S. (Eds.), Handbook of ResearchMethods for Studying Daily Life. The Guilford Press, New York, pp. 62–86.

Wilhelm, P., Schoebi, D., 2007. Assessing mood in daily life: structural validity,sensitivity to change, and reliability of a short-scale to measure three basicdimensions of mood. European Journal of Psychological Assessment 23,256–267.