\PERGAMON Journal of Psychiatric Research 21 "0887# 250Ð257

9911Ð2845:87 ,08[99 Þ 0887 Published by Elsevier Science Ltd[ All rights reservedPII] S 9 9 1 1 Ð 2 8 4 5 " 8 7 # 9 9 9 1 0 Ð 0

Validity of the Composite International Diagnostic Interview"CIDI# psychosis module in a psychiatric setting

Lucy Coopera\ Lorna Petersa\ Gavin Andrewsa\b�a World Health Or`anization Collaboratin` Centre for Mental Health and Substance Abuse\ St Vincent|s Hospital\ Darlin`hurst\

New South Wales 1909\ Australiab School of Psychiatry\ University of New South Wales\ Australia

Received 0 September 0886^ accepted 7 April 0887

Abstract

This study aimed to test the procedural validity of the psychosis module of the Composite International Diagnostic Interview"CIDI# by comparing it with diagnostic checklists completed by experienced clinicians[ Seventy!_ve subjects were interviewed usingthe interviewer!administered version of the CIDI[ Their clinician"s# then completed diagnostic checklists for DSMIV and ICD09diagnoses of schizophrenia[ Agreement was measured at the diagnostic\ criterion and subcriterion levels[ The validity standard"diagnostic checklist# was shown to be reliable with interrater agreement between the clinicians for the diagnosis of schizophreniabeing excellent "k � 9[71 for DSMIV and k � 9[60 for ICD09#[ The agreement between the CIDI and the clinician checklists variedwith sensitivities for DSMIV subcriteria ranging from 9[07 "negative symptoms# to 9[82 "bizarre delusions# and speci_cities rangingfrom 9[27 "catatonia# to 9[84 "disorganised speech#[ A similar pattern was found for ICD09 subcriteria] sensitivity varied from 9[08"neologisms# to 9[89 "persistent delusions# and speci_city varied from 9[28 "catatonia# to 9[84 "negative symptoms#[ The poorestlevels of agreement were found for symptoms requiring interviewer judgement[ The CIDI showed good agreement with clinicianchecklist diagnoses when the criteria were based on questions asked of the subjects[ When the interviewer was required to makejudgement of behaviours\ the agreement between the CIDI and the clinician checklists was lower\ resulting in overall poor agreementbetween the CIDI and the clinician checklists[ Suggestions for improving the validity of the psychosis module of the CIDI are made[Þ 0887 Elsevier Science Ltd[ All rights reserved[

Key words] Composite International Diagnostic Interview^ Structured diagnostic interviews^ Psychosis^ Validity

0[ Introduction

The CIDI is a fully standardised structured diagnosticinterview for the assessment of mental disorders[ It wasdeveloped by the World Health Organisation "WHO# incollaboration with the former US Alcohol\ Drug Abuseand Mental Health Administration "ADAMHA#[ Unlikeits predecessor\ the NIMH!Diagnostic Interview Sched!ule "DIS\ Robins et al[\ 0870#\ it provides a range ofdiagnoses according to the de_nitions and criteria of bothICD09 Diagnostic Criteria for Research "WHO\ 0882#and the Diagnostic and Statistical Manual of MentalDisorders "DSMIV^ American Psychiatric Association\0883#[ To ensure concordance with current diagnosticsystems\ the CIDI is continually updated and revised bya team of experts from around the world[ Version 1[0\modi_ed to meet DSMIV criteria\ was released in 0886[

� Corresponding author[ Tel[ ] 99501 8221 0902 ^ fax ] 99501 82213205 ^ e!mail ] gavinaÝcrufad[unsw[edu[au

As well as a twelve!month and lifetime version\ a fullycomputerised version of the interview is also available"CIDI!Auto^ WHO\ 0886#[ CIDI!Auto faithfully repro!duces the questions from the paper and pencil version ofthe CIDI\ with the skip patterns and probe!~ow chartquestions implemented by the program[ It is availablein two versions] interviewer!administered or respondent!administered[ Studies comparing CIDI!Auto with paperand pencil CIDI show the concordance between theseinstruments to be high "Peters et al[ in press#[

A main advantage of the CIDI is that it is designedto be administered by lay interviewers with no clinicaltraining[ As all diagnoses are computed by the program\using scoring algorithms based on ICD09 and DSMIVcriteria\ no clinical judgment on the part of the inter!viewer is required[ Further\ the highly structured formatof the CIDI ensures that sources of unreliability such ascriterion variance\ information variance\ interpretationvariance and observer variance are minimised "Spitzer\0872^ Wittchen\ 0883#[ Indeed\ a number of studies haveshown the CIDI to have good to excellent reliability

L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257251

across raters\ cultures and time "e[g[ Andrews et al[\ 0884^Wittchen et al[\ 0880^ for a review\ see Wittchen\ 0883#[

While the validity of the CIDI has been less extensivelystudied\ it is acceptable "Peters and Andrews\ 0884^Wittchen\ 0883#[ The validity of the psychosis module\however\ is less clearly established[ Studies investigatingearlier versions of the module from the DIS "e[g[ Burnamet al[\ 0872^ Hendricks et al[\ 0872^ Pulver and Carpenter\0872^ Spengler and Wittchen\ 0877^ Wittchen et al[\ 0874#were disappointing with the DIS tending to under!estimate the prevalence of schizophrenia when measuredagainst a range of clinical standards including chart diag!nosis "Hendricks et al[\ 0872^ DIS!DSM!III!R^ Erdmanet al[\ 0876#^ LEAD "Longitudinal\ Expert\ All Data#standard "Wittchen et al[\ 0874# and the Present StateExamination "Pulver and Carpenter\ 0872#[ The mostrecent study was of CIDI!Auto 0[0 and reported a poorresult with a sensitivity for the diagnosis of schizophreniaof 9[35 "Rosenman et al[\ 0886#[ Methodological prob!lems\ such as the use of an ureliable validity standard"unstructured diagnosis of a single clinician# and the useof the respondent administered version of CIDI!Autoin a severely disturbed population\ warrants caution indrawing _rm conclusions from this result[

With the poor performance of the psychosis moduleand the lack of research investigating current versions ofthe module\ the aim of the present research was to exam!ine the procedural validity "cf[ Spitzer and Williams\0879# of the psychosis module of CIDI 1[0[ This is par!ticularly crucial given the current debate surrounding thevalue of using lay administered\ structured diagnosticinstruments for the assessment of more severe mentaldisorders such as psychosis "Pulver and Carpenter\ 0872^Spengler and Wittchen\ 0877#[ Validity was tested byexamining the concordance of the CIDI with the struc!tured diagnoses of two independent clinicians[

1[ Method

1[0[ Subjects

A total of 64 subjects "14 females\ 49 males^ age]mean�24[62\ S[D�02[82# were administered the psy!chosis module of CIDI!Auto 1[0[ A clinical sample wasprocured in order to overcome the low base rate of schizo!phrenia in the general population[ Subjects were selectedwho met the following criteria] "a# they could attend a29!min interview and "b# they could give coherentresponses to questions[ Interviews from two subjects hadto be discarded from the _nal analysis because of com!prehension di.culties[ There were 59 patients from anacute psychiatric ward\ but due to the slow turnover ofpatients on the ward\ additional subjects were recruitedfrom other clinical services[

1[1[ Desi`n

ICD09 and DSMIV diagnoses of schizophrenia madeby the computerised version of CIDI 1[0 were comparedto ICD09 and DSMIV diagnoses made by clinicians usingchecklists for schizophrenia[ It was expected that addingstructure to the diagnostic process would substantiallyimprove the reliability of clinicians| diagnoses by limitingsources of observer\ information and criterion variance"cf[ Dohrenwend\ 0889^ McGorry et al[\ 0881^ Robins etal[\ 0877^ Spitzer\ 0872^ Wittchen\ 0883#[ Further\explicitly listing the diagnostic criteria allowed a directcomparison at the criterion level between the CIDI andclinician diagnoses[

1[1[0[ CIDIÐauto psychosis moduleThe psychosis module from the 01!month version of

CIDI!Auto 1[0 was used[ The 01!month and lifetimeversions of the CIDI are identical except for the timeframes in which questions are phrased[ While the lifetimeCIDI asks the respondent if they have ever experienced asymptom in their lifetime\ the 01!month CIDI asks if theyhave experienced the symptom in the past 01 months[In other words\ in the 01!month CIDI a diagnosis isestablished only on the basis of the individuals| statusduring 01 months prior to interview[ CIDI!Auto is acomplete computerised replication of the paper and pen!cil CIDI "WHO\ 0886#[ By following the skip rules andtaking the interviewer through the correct route in theprobe ~ow chart\ the opportunity for interviewer error isfurther reduced[ Internal scoring algorithms within theprogram provide DSMIV and ICD09 diagnoses forschizophrenia and related disorders[ Due to the severityof mental disorders in the sample\ the interviewer admin!istered\ as opposed to the respondent administered ver!sion of CIDI!Auto was used[ The psychosis module isone of the 04 discrete modules in the CIDI[ It can eitherbe administered as part of the larger interview\ or by itself[The psychosis module consists of 35 questions relatingto ICD09 and DSMIV diagnostic criteria[ 24 of thesequestions are asked directly of the respondent and askabout delusional beliefs "e[g[ {{In the past 01 months\have you ever believed people were spying on you<||# andhallucinatory experiences "e[g[ {{In the past 01 months\have you more than once heard things other peoplecouldn|t hear\ such as a voice<||#[ Respondents are thenasked for an example which the interviewer judges for itsplausibility[ The _nal 00 questions of the module arenot asked of the respondent but are completed by theinterviewer at the end of the interview[ These questionsrelate to the presence of symptoms such as hallucinatorybehaviour\ catatonia\ brief empty speech and ~at a}ect[Judgements are made on the basis of the respondent|sbehaviour during the interview[

L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257 252

1[1[1[ ICD09 and DSMIV checklistsClinicians| diagnoses were established using two sep!

arate diagnostic checklists] one listing DSMIV diagnosticcriteria for schizophrenia and the other ICD09 diagnosticcriteria[ Next to each criterion or sub!criterion\ clinicianscircled one of three possible response options] "0#\ if thesymptom was not present^ "4#\ if the symptom was presentand "8#\ if the clinician was uncertain[ While these check!lists were generated for the purpose of the current study\criteria were listed using the exact wording from eachclassi_cation system[

1[2[ Procedure

The CIDI interviews were conducted in private[Written informed consent was obtained from all subjectsbefore the interviews began[ Subjects were paid ,19 fortheir participation[ The two CIDI interviewers were four!year trained psychology graduates who had recently com!pleted a one!week intensive training course in admin!istering the CIDI at the WHO designated training centrein Sydney[ Given the high proportion of questions requir!ing interviewer judgements\ the _rst six interviews wererated by both interviewers[ This allowed rating dis!crepancies to be resolved in the initial stages[ To enableinterrater reliability to be calculated across all the inter!views\ a further three interviews were double rated laterin the study[ Average CIDI psychosis module interviewtime was 13 min "S[D[�01 min#[

The treating clinicians were given both the ICD09 andDSMIV checklists to complete in the days after eachCIDI interview had been completed[ Clinicians were notgiven access to the information obtained from the CIDIinterview[ Where two treating clinicians were available"n�32#\ both clinicians completed the checklists[ Wherethere were discrepancies between the checklists\ the clin!icians met\ resolved the discrepancies and completed athird\ consensus checklist which was used as the validitystandard[ This procedure generated a more reliable val!idity standard while also enabling the reliability of theclinicians| diagnoses to be assessed[ Eight clinicians par!ticipated "senior psychiatry residents\ consultant psy!chiatrists\ and clinical psychologists#[ All clinicians hadextensive experience in the treatment and assessment ofpsychiatric disorders[

After all data were collected\ the WHO scoring pro!gram for the CIDI was used to generate ICD09 andDSMIV diagnoses[ As the entire CIDI was not admin!istered in the study\ some of the exclusion criteria forschizophrenia "e[g[ the presence of other mental dis!orders# could not be assessed and\ therefore\ were notincluded in the _nal diagnoses[ Clinician completedchecklists were scored according to ICD09 and DSMIVcriteria "excluding the criteria not addressed in the CIDI#[As {uncertain| was not a response option in the CIDI\ all

8s were recoded as 0 "not present# in the _nal analysis^i[e[ the more conservative estimate was used[

1[3[ Analysis

To measure interrater reliability between the clinicians\Cohen|s kappas "0859# were calculated as a measure ofchance!corrected agreement[ Interpretation of kappa wasbased on Fleiss "0870#[

To measure the concordance between the CIDI andthe checklists\ percent agreement and sensitivity andspeci_city statistics were calculated[ Statistics were cal!culated at the level of subcriterion\ criterion and diag!nostic level for both ICD09 and DSMIV diagnosticcriteria[ Sensitivity measures the proportion of subjectspositively diagnosed by the clinicians also positively diag!nosed by the CIDI[ Speci_city measures the proportionof subjects not diagnosed by the clinicians with schizo!phrenia who are also not diagnosed by the CIDI[ Whilethe kappa statistic is often calculated in studies to assessvalidity\ several factors make sensitivity and speci_citymore appropriate statistical measures for this design "seeMaclure and Willett\ 0876 for a review#[ Firstly\ kappaassumes that the same instrument is being employed bythe raters on both occasions[ In other words\ kappa treatsthe diagnostic standard "clinicians| diagnosis# as identicalto the instrument being validated "CIDI#[ This makeskappa more a measure of reliability rather than validity"Anthony et al[\ 0874^ Helzer et al[\ 0874^ Robins\ 0874#[Secondly\ kappa is sensitive to sample characteristicssuch as prevalence rates[ This limits the generalizability ofthe results of the study and renders comparisons betweenstudies uninformative and misleading "Fleiss\ 0870#[Thus\ kappa was not used as a statistical measure ofvalidity in this study[

2[ Results

2[0[ CIDI interrater reliability

The nine double rated CIDI interviews were inves!tigated for any discrepancies[ For individual criteriaacross the interviews\ only two had more than one dis!crepancy[ These were catatonic behaviour and delusions\both coded using interviewer judgements[

2[1[ Clinician checklist interrater reliability

To assess the integrity of the clinicians| diagnoses\kappas were calculated at the sub!criterion\ criterion anddiagnostic level for both ICD09 and DSMIV diagnosticcriteria[ The agreement between the clinicians rangedfrom good to excellent "see Table 0#\ with most kappasabove 9[64[ Kappa for agreement between the cliniciansfor the diagnosis of schizophrenia was 9[71 for DSMIV

L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257253

Table 0Interrater reliability between clinicians on the DSMIV and ICD09 checklists for the diagnosis of schizophrenia

DSMIV Criteria Kappa ICD!09 General criteria Kappa

184[XX Diagnosis of schizophrenia 9[71 F19[X Diagnosis of schizophrenia 9[60Criterion A 9[73 Criterion G0 9[70

A0 Delusions 9[74 0A Thought echo 9[70Bizarre delusions 9[70 0B Control delusions 9[79

A1 Hallucinations 9[76 0C Hallucinatory voices 9[73A2 Disorganised speech 9[75 0D Persistent delusions 9[67A3 Catatonia 9[64 1A Persistent hallucinations 9[80A4 Negative symptoms 9[85 1B Neologisms 9[74

Criterion B ] social:occupational 9[62 1C Catatonia 9[70Criterion C ] duration 9[89 1D Negative symptoms 9[58Criterion E ] substance:medical exclusions 9[64 Criterion G1[1 ] substance:medical exclusions 9[69

and 9[60 for ICD09[ These results indicate the validitystandard to be reliable[

2[2[ Clinician checklists vs CIDI

Amongst the 62 subjects\ the CIDI made 6 DSMIVdiagnoses of schizophrenia and 03 ICD09 diagnoses ofschizophrenia[ This was considerably fewer than the clin!icians who made 39 DSMIV diagnoses and 23 ICD09diagnoses of schizophrenia[ Table 1 presents the sen!sitivity and speci_city scores and percent agreement forthe CIDI at the subcriterion\ criterion and diagnosticlevel for ICD09 and DSMIV[ Firstly\ for DSMIV criteria\the performance of the CIDI was mixed[ Percent agree!ment was 49[6 for the diagnosis of schizophrenia\ andranged between 32[6 and 66[4 at the criterion level[Sensitivity ranged from very good "9[82 bizarredelusions#\ to very poor "9[07 negative symptoms#[ Over!all\ speci_city scores were better\ but still varied "range

Table 1CIDI vs clinician checklists ] percentage agreement\ sensitivity and speci_city for DSMIV and ICD09 at the diagnostic\ criterion and sub criterionlevel

) )DSMIV criteria agreement Sensitivity Speci_city ICD!09 criteria agreement Sensitivity Speci_city

184[XX Schizophrenia 49[6 9[04 9[86 F19[X Schizophrenia 46[7 9[15 9[75Criterion A 41[0 9[26 9[75 Criterion G0 41[0 9[24 9[73

A0 Delusions 66[4 9[82 9[44 0A Thought echo 42[4 9[71 9[37Bizarre delusions 62[1 9[66 9[56 0B Control delusions 69[3 9[56 9[61

A1 Hallucinations 56[5 9[75 9[49 0C Hallucinatory voices 69[3 9[59 9[64A2 Disorganised speech 53[7 9[10 9[84 0D Persistent delusions 66[4 9[89 9[52A3 Catatonia 34[0 9[52 9[27 1A Persistent hallucinations 48[1 9[48 9[59A4 Negative symptoms 52[3 9[07 9[82 1B Neologisms 55[1 9[08 9[82

Criterion B ] social:occupational 55[1 9[57 9[47 1C Catatonia 31[2 9[56 9[28Criterion C ] duration 32[6 9[19 9[84 1D Negative symptoms 52[3 9[19 9[84Criterion E ] substance:medical 55[1 9[57 9[52 Criterion G1[1 ] substance:medical 55[1 9[60 9[59exclusions exclusions

9[27Ð9[84#[ In general\ the sensitivity of the CIDI wasparticularly poor for the sub criteria scored at the end ofthe interview using interviewer judgements] disorganisedspeech\ negative symptoms and catatonia\ resulting inpoor sensitivity for General Criterion A[ The sensitivityfor the diagnosis of schizophrenia was very low "9[04#\with high speci_city "9[86#[

For ICD09 criteria\ the pattern of results is repeated[Agreement was 46[7) for the diagnosis of schizophreniaand ranged between 31[2 and 66[4) at the criterion level[Sensitivity varied considerably from low "9[08 neol!ogisms# to high "9[89 persistent delusions#[ Speci_cityalso varied but was higher "range 9[28Ð9[84#[ Again\ thesensitivity of the CIDI was consistently poor for thesub criteria scored at the end of the interview using theinterviewer judgements\ neologisms and negative symp!toms\ resulting in poor sensitivity for General Criterion0[ The sensitivity for the diagnosis of schizophrenia wasagain low "9[15# and speci_city\ high "9[75#[

L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257 254

3[ Discussion

This study presents data on the validity of the psychosismodule of the interviewer administered CIDI!Auto "1[0#[Validity was assessed by examining the concordancebetween the CIDI!Auto as administered by trained\ layinterviewers and diagnostic checklists completed by clin!icians[ Unlike previous validity studies on the CIDI "e[g[Janca et al[\ 0881^ Kovess et al[\ 0881#\ the validity stan!dard of this study was not contaminated by allowing theclinicians access to the information obtained in the CIDIinterview[ Further\ as can be seen from the good inter!rater reliability between the clinicians\ the structured for!mat of the checklists provided a reliable validity standard[

Overall\ results from the study suggest that comparedto clinicians| diagnoses\ the psychosis module of the CIDIhas good speci_city\ but poor sensitivity[ For DSMIVcriteria the sensitivity and speci_city for the diagnosisof schizophrenia was 9[04 and 9[86 respectively and forICD09 criteria\ the sensitivity and speci_city was 9[15and 9[75 respectively[ This suggests that while the CIDIwas accurate in the positive diagnoses it made\ it had ahigh {miss| rate\ failing to detect many of the positivediagnoses made by the clinicians[ Compared to the clin!icians\ the CIDI detected only 07) of DSMIV diagnosesof schizophrenia\ and 30) of ICD09 diagnoses of schizo!phrenia resulting in a considerable underestimation ofthe {true| prevalence rate in this sample[

These _ndings are in accordance with earlier studiesinvestigating the validity of the psychosis module of theDIS[ Results from these studies show the psychosis mod!ule to considerably underestimate the prevalence ofschizophrenia in both general and clinical populations"Wittchen et al[\ 0874#\ for patients in both active andnon!active phases of the disorder "Semler et al[\ 0876^Spengler and Wittchen\ 0877^ Wittchen et al[\ 0874#\ andfor lifetime and current diagnosis "Spengler andWittchen\ 0877#[ In the study by Wittchen et al[ "0874#\the psychosis module had the poorest concordance withstructured clinical diagnoses "LEAD standard# out of allthe DIS modules[ Similarly\ in a study which examinedthe validity of the DIS when administered by trained layinterviewers\ the DIS failed to detect one third of subjectswith a well documented history of the disorder "Pulverand Carpenter\ 0877#[ While the considerable time delayof six years between the clinicians| diagnoses and DISinterview necessitates caution in interpreting this par!ticular result\ it is consistent with the general _ndingthat the psychosis module considerably underestimatesprevalence of the disorder[ Finally\ in the more recentstudy by Rosenman et al[ "0886# using CIDI!Auto 0[0\sensitivity for the ICD09 diagnosis of schizophrenia andrelated disorders was 9[35 and speci_city 9[75[ The lowersensitivities obtained in the present study "9[15# couldre~ect the fact that the self!administered version of CIDI!Auto does not include the interviewer!rated questions at

the end of the interview which generally performed poorly"e[g[ neologisms\ negative symptoms# and this utilised aless stringent validity standard[

While variations in sample characteristics and method!ologies between these previous studies and the currentstudy necessitate caution when making comparisons\ thesimilar pattern in results suggests particular problemswith the validity of the psychosis module[ An advantageof the current study was that the validity standardenabled a more sensitive analysis of the performance ofthe CIDI at the criterion and sub criterion levels[ Theperformance of the CIDI varied considerably betweenindividual criteria[ For ICD09 criteria\ sensitivitiesranged between 9[08 "neologisms# and 9[89 "persistentdelusions#\ and for DSMIV criteria sensitivities rangedbetween 9[07 "negative symptoms# and 9[82 "bizarredelusions#[ For both diagnostic systems\ the sensitivitiesfor 5 out of the 09 criteria were 9[59 or above[ Thissuggests that the low sensitivity obtained for the diagnosisof schizophrenia may be attributable to the poor per!formance of the CIDI on a small number of criteria[ ForDSMIV\ the poorly performing criteria were dis!organised speech\ duration greater than six months\ andnegative symptoms "sensitivity 9[10^ 9[19 and 9[07 respec!tively#\ and for ICD09 they were neologisms and negativesymptoms "sensitivity 9[08 and 9[19 respectively#[ Apartfrom duration for DSMIV\ all these poorly performingcriteria are scored from the _nal 00 questions rated bythe interviewer at the end of the interview[ These requirethe interviewer to judge\ for example\ if the subject is ableto {persist in goal directed activities|\ or whether theyexhibit features of {~at a}ect|\ or {thought disorder[| Twoexplanations could account for the low concordancebetween the CIDI interviewers and clinicians for theseitems[ Firstly\ it is possible that only administering thepsychosis module of the CIDI did not allow su.cienttime for interviewers to observe the behaviours to berated and hence\ did not allow the interviewers to makeaccurate judgements[ Secondly\ the sensitivity for theseitems may have been poor because they were assessedusing the judgements of lay interviewers[ While othercriteria are assessed in the module on the basis of subjectself!report\ these criteria are assessed entirely on the basisof interviewer judgement[ Given the complex clinical nat!ure of these criteria\ it is probable that interviewers whodo not have a clinical background do not possess thenecessary clinical skills to make these judgements[

In support of this second explanation\ the psychosismodule is the only module in the CIDI for whichadequate validity has not yet been established "seeWittchen\ 0883# and it is the only module which dependson interviewer judgement to establish diagnoses[ For allother modules\ diagnoses are made solely on the basis ofsubject self!report[ A problem unique to the psychoticdisorders\ however\ is that diagnostic criteria relating tofacial expression and speech patterns can only be assessed

L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257255

by an outside observer[ In light of this restriction\ onemeans of improving the validity of this module would beto extend the current interviewer training program tofocus speci_cally on the identi_cation and assessment ofthese criteria[ The better sensitivities obtained for thecriteria relating to the more obvious symptoms ofdelusions and hallucinations would suggest that lay inter!viewers are able to distinguish between {normal| and clini!cally signi_cant experiences when provided with self!reported symptoms[

Alternatively\ how these criteria are currentlydescribed and operationalised in the CIDI could beimproved[ Sensitivities for these items may be low simplybecause they have poor construct validity[ In otherwords\ interviewers are not coding these items incorrectlybecause they lack the necessary skills\ but because theyare poorly conceptualised and described by the CIDItext[ A way to investigate this hypothesis would be tohave trained\ experienced clinicians administer theseitems[ If poor sensitivities were still obtained for theseitems\ this would suggest problems in their oper!ationalisation as opposed to problems with the judge!ments of lay interviewers[

Other factors\ however\ are also likely to be con!tributing to the overall poor validity of this module[While the tendency for structured diagnostic interviewsin general to underdiagnose compared to clinicians hasbeen well documented in the literature "e[g[ Fennig et al[\0883^ McGorry et al[\ 0889^ Wittchen et al[\ 0874#\ fea!tures of psychosis are considered to make its assessmentvia structured interviews particularly tenuous[ Notably\the episodic nature of the illness is believed to makeobtaining an accurate assessment from a single interviewvery di.cult\ with results ~uctuating depending upon theillness {phase| in which the interview takes place"Wittchen et al[\ 0878#[ In the non!active phase of thedisorder\ fear of social stigmatisation associated with psy!chotic disorders is believed to lead subjects to either deny\or minimise experiencing symptoms "Anthony et al[\0874^ Erdman et al[\ 0876^ Pulver and Carpenter\ 0877#[This is evident in the particularly poor recall of symptomsshown by subjects with no ~orid symptoms at the timeof interview "Wittchen et al[\ 0874#[ Then\ in the {active|phase of the disorder\ features of psychosis such as de_!cits in a}ect\ insight and reasoning interfere with thesubject|s ability to give coherent and accurate responses"Farmer et al[\ 0876^ McGorry et al[\ 0878^ Pulver andCarpenter\ 0887#[ In support of this\ it has been shownthat the accuracy of CIDI diagnoses notably decreasesfor cases in an active phase of schizophrenia "Semler etal[\ 0876#[ This is likely to have been particularly prob!lematic in the current study\ given that the sample camepredominantly from an acutely ill population[

These di.culties associated with psychosis have oftenbeen raised as arguments against the use of structuredinterviews such as the CIDI for assessment of more

chronic disorders such as schizophrenia[ The problemwith these self!report instruments is that their reliabilityand validity is ultimately dependent on the lucidity andcompliance of the subject[ In the case of psychoses\however\ this cannot be depended upon and testi_es tothe need to include sources of information other thansubject self!report to establish accurate diagnoses[ Forinstance\ information collected by the CIDI could besupplemented with information obtained from medicalrecords or interviews with close relatives of the subject[

More generally\ however\ it should be noted that theendeavour to achieve perfect agreement with diagnosesmade by clinicians is unrealistic[ Firstly clinicians are ableto draw upon multiple sources of information and theirown clinical experience in reaching diagnoses[ As well\structured diagnostic interviews are more stringent intheir interpretation and application of the diagnostic cri!teria meaning that compared to clinicians\ they willalways tend to underdiagnose {true| prevalence rates[Secondly\ the less than perfect agreement found betweenclinicians in the diagnosis of schizophrenia creates a {ceil!ing|] the validity of the instrument can only be as goodas that of the diagnostic standard it is compared with[Kappas of between 9[58 and 9[60 for example\ are gen!erally reported for the diagnosis of schizophrenia by twoclinicians "Robins\ 0874^ Winokur et al[\ 0877#[ In thisstudy\ while the structure of the checklists ensured theclinicians applied the criteria\ kappas for the diagnosis ofschizophrenia were still only 9[71 for ICD09 criteria\and 9[60 for DSMIV[ Hence\ until a gold standard inpsychiatry is obtained\ the validity of instruments suchas the CIDI will always be less than perfect[

Finally\ it should also be noted that while the CIDIwas designed primarily as an epidemiological instrumentfor use in general populations\ as with most previousvalidity studies on the psychosis module\ the currentstudy was conducted within a psychiatric setting[ Giventhe nature of psychosis\ it is likely that the way in whichthis module in particular performs in a clinical populationwill be di}erent to the way in which it performs in thegeneral population[ Indeed\ a recent study by Kendler etal[ "0885# using a modi_ed University of Michigan ver!sion of the CIDI "UM!CIDI# to assess lifetime prevalencerates of disorders in the general population showed justthis[ While the lifetime prevalence rate for schizo!phrenia:schizophreniform disorder was estimated to be0[2) by the CIDI\ the equivalent prevalence rate basedon clinicians| diagnoses was 9[1)[ Only 09) of the posi!tive diagnoses made by the CIDI were con_rmed by theclinicians[ In other words\ the psychosis module whenused in the community considerably overdiagnosed theprevalence of schizophrenia and related disorders\thereby reversing the trend found in clinical populations[One explanation given for the high false positives foundin the Kendler et al[ "0885# study was that the thresholdfor judging if a symptom is {psychotic| is too low in the

L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257 256

CIDI[ This would indicate a problem with the plausibilityjudgements made by interviewers to assess the clinicalsigni_cance of {unusual| experiences[ The good sensitivityand speci_city found for these items in the current study\however\ suggests that these judgements are only di.cultto make in general populations in which the symptomsare likely to be less obvious[

In conclusion\ the following directions for futureresearch are recommended in order to improve the poorvalidity of the psychosis module[ Firstly\ the module|scurrent reliance on interviewer judgement at the end ofthe interview\ particularly in relation to the assessmentof negative symptoms\ needs to be addressed[ Secondly\using other sources of information to supplement theCIDI in establishing diagnoses\ particularly in a psy!chiatric setting\ needs to be considered[ This would alsohelp to identify additional reasons for the module|s cur!rent high false negative rate[ Finally\ the way in which themodule performs in less severely disturbed populationsneeds to be determined\ particularly in relation to itsability to discriminate threshold or subclinical cases from{true| cases[ This is particularly crucial given the CIDI|sprimary role as an epidemiological instrument[

Acknowledgements

This study was supported by a grant from the Com!monwealth of Australia\ Research and DevelopmentGrants Advisory Committee[

References

American Psychiatric Association[ Diagnostic and statistical manual ofmental disorders[ 3th ed[ Washington\ DC] American PsychiatricAssociation\ 0883[

Andrews G\ Peters L\ Guzman A\ Bird K[ A comparison of two struc!tured diagnostic interviews] CIDI and SCAN[ Australian and NewZealand Journal of Psychiatry 0884^18]013Ð21[

Anthony JC\ Folstein M\ Romanowski AJ\ von Kor} MR\ NestadtGR\ Chadal R\ Merchant A\ Brown H\ Shapiro S\ Kramer M\Gruenberg EM[ Comparison of the lay Diagnostic Interview Sched!ule and a standardized psychiatric diagnosis[ Experiences EasternBaltimore[ Archives of General Psychiatry 0874^31]556Ð64[

Burnam MA\ Karno M\ Hough RL\ Escobar JI\ Forsyth AB[ TheSpanish Diagnostic Interview Schedule[ Reliability and comparisonwith clinical diagnoses[ Archives of General Psychiatry0872^39]0078Ð85[

Cohen\ J[ A coe.cient of agreement for nominal scales[ EducationalPsychological Measurement 0859^19]26Ð35[

Dohrenwend BP[ {The problem of validity in _eld studies of psycho!logical disorders| revisited[ Psychological Medicine 0889^19]084Ð197[

Erdman HP\ Klein MH\ Greist JH\ Bass SM\ Bires JK\ MachtingerPE[ A comparison of the Diagnostic Interview Schedule and clinicaldiagnosis[ American Journal of Psychiatry 0874^033]0366Ð79[

Farmer AE\ Katz R\ McGu.n P\ Bebbington P[ A comparison betweenthe present state examination and the Composite International

Diagnostic Interview[ Archives of General Psychiatry 0876^33]0953Ð8[

Fennig S\ Craig T\ Lavelle J\ Kovasznay B\ Bromet EJ[ Best!estimateversus structured interview!based diagnosis in _rst!admission psy!chosis[ Comprehensive Psychiatry 0883^24]230Ð7[

Fleiss JL[ Statistical methods for rates and proportions[ 1nd ed[ NewYork] John Wiley and Sons\ 0870[

Helzer JE\ Robins LN\ McEvoy LT[ A comparison of clinical andDiagnostic Interview Schedule diagnoses] physician reexaminationof lay!interviewed cases in the general population[ Archives of Gen!eral Psychiatry 0874^31]546Ð55[

Hendricks LE\ Bayton JA\ Collins JL\ Mathura CB\ McMillan SR\Montgomery TA[ The NIMH Diagnostic Interview Schedule] a testof its validity in a population of black adults[ Journal of NationalMedical Association 0872^64]556Ð60[

Janca A\ Robins LN\ Cottler LB\ Early TS[ Clinical observation ofassessment using the Composite International Diagnostic Interview"CIDI#] an analysis of the CIDI _eld trials*wave II at the St Louissite[ British Journal of Psychiatry 0881^059]704Ð7[

Kendler KS\ Gallagher TJ\ Abelson JM\ Kessler RC[ Lifetime preva!lence\ demographic risk factors\ and diagnostic validity of non!a}ective psychosis as assessed in a US community sample[ Archivesof General Psychiatry 0885^42^0911Ð20[

Kovess V\ Sylla O\ Fournier L\ Favigny V[ Why discrepancies existbetween structured diagnostic interviews and clinicians| diagnoses[Social Psychiatry and Psychiatric Epidemiology 0881^16]074Ð80[

Maclure M\ Willett WC[ Misinterpretation and misuse of the kappastatistic[ American Journal of Epidemiology 0876^015]050Ð8[

McGorry PD\ Copolov DL\ Singh BS[ Royal Park MultidiagnosticInstrument for Psychosis] part I[ Rational and review[ Schizo!phrenia Bulletin 0878^05]490Ð04[

McGorry PD\ Singh B\ Connell S\ McKenzie D\ Van Riel R\ CopolovDL[ Diagnostic concordance in functional psychosis revisited] astudy of inter!relationships between alternative concepts of psy!chotic disorder[ Psychological medicine 0881^11]256Ð67[

McGorry PD\ Singh B\ Copolov DL\ Kaplan I\ Dossetor CR\ vanRiel RJ[ Royal Park Multidiagnostic Instrument for Psychosis II[Development\ reliability and validity[ Schizophrenia Bulletin0889^05]406Ð25[

Peters L\ Andrews G[ Procedural validity of the computerized versionof the Composite International Diagnostic Interview "CIDI!Auto#in anxiety disorders[ Psychological Medicine 0884^14]0158Ð79[

Peters L\ Clark D\ Carroll F[ Are computerised interviews equivalent tohuman interviewers< CIDI!Auto vs CIDI in anxiety and depressivedisorders[ Psychological Medicine\ in press[

Pulver AE\ Carpenter WT[ Lifetime psychotic symptoms assessed withthe DIS[ Schizophrenia Bulletin 0872^8]266Ð71[

Pulver AE\ Carpenter WT[ Accuracy of the diagnoses of a}ective dis!orders and schizophrenia in public hospitals[ American Journal ofPsychiatry 0877^034]107Ð19[

Robins LN[ Re~ections on testing the validity of psychiatric interviews[Archives of General Psychiatry 0874^31]807Ð13[

Robins LN\ Helzer JE\ Croughan J\ Ratcli} KS[ National Institute ofMental Health Diagnostic Interview Schedule[ Archives of GeneralPsychiatry 0870^27]270Ð8[

Robins LN\ Wing J\ Wittchen H!U\ Helzer JE\ Babor TF\ Burke J\Farmer A\ Jablenski A\ Pickens R\ Regier D\ Sartorius N\ TowleLH[ The Composite International Diagnostic Interview[ Archivesof General Psychiatry 0877^34]0958Ð66[

Rosenman SJ\ Korten AE\ Levings CT[\ Computerised diagnosis inacute psychiatry] validity of CIDI!Auto against routine clinicaldiagnoses[ Journal of Psychiatry Research 0886^20]470Ð81[

Semler G\ Wittchen H!U\ Joschke K\ Zaudig M\ von Geiso T\ KaiserS\ von Cranach M\ P_ster H[ Test!retest reliability of a standardizedpsychiatric interview "DIS:CIDI#[ European Archives of Psychiatryand Neurological Sciences 0876^125]103Ð11[

Spengler PA\ Wittchen H!U[ Procedural validity of standardized symp!

L[ Cooper et al[:Journal of Psychiatric Research 21 "0887# 250Ð257257

tom questions for the assessment of psychotic symptoms] A com!parison of the DIS with two clinical methods[ Comprehensive Psy!chiatry 0877^18]298Ð11[

Spitzer RL[ Psychiatric diagnosis] are clinicians still necessary< Com!prehensive Psychiatry 0872^13]288Ð300[

Spitzer RL\ Williams JBW[ Procedural validity] validity of the diag!nostic process[ In Kaplan HI\ Freedman AM\ Sadock BJ\ editors[Comprehensive Textbook of Psychiatry 2rd ed[ Baltimore] Williamsand Wilkins Company\ 0879[

Winokur G\ Zimmerman M\ Cadoret R[ Cause the bible tells me so[Archives of General Psychiatry 0877^34]572Ð3[

Wittchen\ H[ [ Reliability and validity studies of the WHO!CompositeInternational Diagnostic Interview "CIDI#] a critical review[ Journalof Psychiatric Research 0883^17]46Ð3[

Wittchen H!U\ Semler G\ von Zerssen D[ A comparison of two diag!nostic methods[ Clinical ICD diagnosis vs DSM!III and RDC usingthe DIS "version II#[ Archives of General Psychiatry 0874^31]566Ð73[

Wittchen H!U\ Burke JD\ Semler G\ P_ster H\ von Cranach M\ ZaudigM[ Recall and dating of psychiatric symptoms] test!retest reliabilityof time!related symptom questions in a standardized psychiatricinterview[ Archives of General Psychiatry 0878^35]326Ð32[

Wittchen H!U\ Robins LN\ Cottler LB\ Sartorius N\ Burke JD\ RegierD[ Participants in the multicentre WHO:ADAMHA _eld trials[Cross!cultural feasibility\ reliability and sources of variance of theComposite International Diagnostic Interview "CIDI#[ British Jour!nal of Psychiatry 0880^048]534Ð42[

World Health Organization[ The ICD09 classi_cation of mental andbehavioral disorders] diagnostic criteria for research[ Geneva\WHO\ 0882[

World Health Organization[ CIDI!Auto "1[0# administrator|s guideand reference[ Sydney] Fast Books\ 0886[

Zimmerman M[ Why are we rushing to publish DSM!IV[ Archives ofGeneral Psychiatry 0877^34]0024Ð7[