Developing an African Youth Psychosocial

Assessment: An Application of Item Response

Theory

Theresa S. Betancourt, Sc.D., M.A.Associate Professor, Department of Global Health and Population

Director, Research Program on Children and Global Adversity

Department of Global Health and Population

Harvard T.H. Chan School of Public Health

Co-Authors: Frances Yang, Paul Bolton, & Sharon-Lise Normand

Overview

□ Measuring Mental Health

Constructs Cross-Culturally

□ Challenges

□ Methods

□ Northern Uganda: Context

□ Qualitative and other studies

□ Refining the AYPA using IRT

□ Implications for Future

Research

Psychosocial Impact of Armed Conflict

□ More than 1billion children worldwide live in areas affected by armed conflict

□ War-affected children experience direct and indirect exposure to violence, disrupted family functioning, damaged social structures, etc. all of which increase risks for mental health problems

□ LMICS lack of monetary and human resources devoted to accurate measurement of mental health problems in children and adolescents

□ Result: Limited data to support claims about the burden of mental health disorders in young people, or the outcomes of intervention research

Culture in Assessment/Measurement and

Intervention Development

“Ethnographic studies

demonstrate convincingly

that concepts of emotions,

self, and body, and general

illness categories differ so

significantly in different

cultures that it can be said

that each culture’s beliefs

about normal and

abnormal behavior are

distinctive”

(Kleinman 1988, p.49)

Typical Use of Questionnaires in Assessment

and Evaluation

1. Select or create questionnaire/select standard measure to adapt

• Usually developed outside the local culture/situation

2. Translate into local language (no validity tests)

3. Individual interviews with survey

4. Determine need based on frequency of responses

5. Choice of problem and therefore intervention is based on quantitative results

6. Repeat individual surveys before and after intervention to assess program impact

Problems with Relying on Western Measures in Cross-

Cultural Research

□ Cultural validity: How closely concepts in a questionnaire match local concepts; Western/outside concepts may not apply locally

□ Unknown local concepts: Are there important local issues/concepts unknown to us? How to include questions we don’t know we should be asking?

□ Translation problems: Who translates? Translation- back translation methods inadequate, can result in semantic equivalence but real-world insignificance (i.e. lighting fires)

□ RISK: Evaluations don’t accurately measure impact

Photo courtesy of Laurie Wen

Use a Mixed Methods Approach:

(qualitative + quantitative methods)

A Model for Designing and Evaluating Mental Health and

Child Development Programs in Diverse Cultural Settings

Qualitative data informs assessment and intervention

Apply lessons learned to new settings and intervention

adaptations

Use qualitative data to select,

adapt, and create

mental health measures and interventions;

conductvalidity study

Implement culturally relevant

intervention;evaluate with

rigorous design

Identify important mental health,

child development etc. constructs relevant

to the context(qualitative

inquiry)

Northern Uganda

Affected Districts

Northern Uganda: Background

□ More than 20 years of

war

□ Joseph Kony (also

Acholi) leader of Lords

Resistance Army (LRA)

seeks to overthrow

government and

establish rule based

on the 10

Commandments

□ LRA involved in

countless atrocities

incl. rape, mutilation,

forced recruitment

(esp. children)

IDP Camps : More than 1.8 Million

People Displaced In IDP camps

Formerly Abducted Children:> 66,000 Children Abducted By LRA

Research: Northern Uganda

□ Derluyn (2004) : 97% PTSD in former LRA

abductees

□ 80% > 6 Traumatic events

□ 77% witnessed killing

□ 39% had been forced to kill

□ SWAY Survey (Annan, 2006): young men aged

14-30

□ Experienced avg. 11 serious violence exposures

□ Lower than expected levels of emotional distress

using adapted Western mental health measures

Northern Uganda:The Mental Health Response

□ Non-governmental organizations launching psychosocial interventions

□ Lack of assessment & tools for program evaluation; Many standardized mental health measures not meaningful

□ Goals: to develop locally meaningful and valid measures of mental health constructs (syndromes, symptoms and function)

□ To use these measures in a RCT of mental health interventions for young people

Unyama IDP Camp

Awer Camp

Phases of the Study

□ PHASE I: Qualitative Study

□ PHASE II: Validity Study & Measures Refinement

□ PHASE III: Randomized Control Trial of mental health interventions

□ PHASE IV: Measures refinement to create African Youth Psychosocial Assessment

□ (AYPA)

Phase I: Qualitative Study

Studying Local Perceptions of

Mental Health

1. Free Listing

2. Key informant interviews

Free List Exercise

□ Respondents are asked a question

designed to elicit responses in the form of

a list.

□ Responses, and a short description of

each, are recorded verbatim in the

respondent’s own language by a trained

local interviewer.

“What are the problems of children in

this camp?”

Free Listing: Analysis

□ Collapse all lists into single composite list

of all problems mentioned

□ Ordered according to frequency

□ Order gives some indication of priority

Free List Exercise

□ Participants: N=15 adults, N=31 10-17 year olds

□ Problems facing children in the IDP camps

□ Function/tasks of young people in family and community

Main Problem Themes Emerging from Free

Listing Exercises (N=45)

Theme Number reporting Percentage

Lack food 34 74%

Lack clothing 31 67%

Lack school fees, uniforms, books, etc. 30 65%

Insecurity/fear of abduction 18 39%

Diseases (sexually transmitted, due to poor

hygiene, malaria) 14 30%

Poor hygiene (latrines, bathing, soap etc) 13 28%

Lack parents 9 20%

Lack of safe housing/shelter 9 20%

Males disrupting females/girls staying with

soldiers/rape 9 20%

Lack money (general) 8 17%

Dropping out of school 7 15%

Stubborn, don't listen to parents 7 15%

Fighting 6 13%

Rude or spoilt (children) 6 13%

□ Prolonged interviews with local experts

□ N=32 adults, N=25 10-17 year olds

□ Investigate selected free list issues in detail (nature, causes, treatments)

□ Look for other syndromes missed on free lists

□ Ability to conduct repeat interviews

Key Informant Interviews

Results – Northern UgandaMood problems (Two Tam, Par and Kumu)

□ sad, cries continuously, sits with cheek in palm,

□ constant worries, forget what they are thinking, loses interest in school

□ think they are of no use, thinks about suicide, don’t care whether they live or die, talks about problems constantly, sits alone, don’t feel like talking to others

□ pain all over body, headache, loses appetite, weak

□ doesn’t sleep

Conduct problems (Gin Lugero/Kwo Maraco)

□ loses interest in school, sexual misbehavior, fights, use bad language, drinks alcohol, disrespectful, misbehaves, disobedient

Anxiety-like problems (Ma Lwor)

□ Clings to elders, wants to be alone, doesn’t greet people,

□ constantly running around, doesn’t sleep, thinks people are chasing them,

□ fast heart rate, loss of appetite, think they have no future,

□ doesn’t like loud noise

□ Developed from the qualitative study

□ Categorized them into 5 local syndromes similar to 3 DSM-IV/ICD-10 domains:1. Depression : Par, Kumu, Two Tam2. Anxiety : Ma Lwor3. Conduct Problems: Kwo maraco/Gin Lugero

□ Additional Qualitative work added prosocial subscale

□ 60 items, 6 subscales, caregiver report & youth self report

Result: Acholi Psychosocial Assessment Instrument (APAI)

□ Boys:

□ Fetching water

□ Digging

□ Sweeping compound

□ Playing football

□ Playing games

□ Girls:□ Fetching water

□ Washing clothes

□ Digging

□ Washing utensils

□ Sweeping house

□ Smearing floor

□ Cooking food

□ Traditional dance

□ Playing ball

Items Comprising the Assessment of Functioning Scales

Phase II: Validity Study

□ Validity: whether instrument really

measures the construct of interest

□ Major problem with much cross-cultural

research is criterion validity

□ Comparison of results against a gold

standard

□ Accepted gold standards usually not

available for these populations

Testing Validity

Diagnosis

□ Local appraisal of illness present or absent

as judged by local people can be used as

an alternative ‘gold standard.’

□ Requires an understanding of local

perceptions of mental disorders or illness

□ Triangulation of informant necessary when

gold standard is lacking

□ Multiple-informants used to identify ‘cases’

and ‘non-cases’ and look for agreement

To Compare The Locally Derived Measure

(The APAI) To Several Validity Criteria:a) Self Report

b) Caregiver Report

c) Standardized Western Measure of Emotional and Behavioral Problems in Children and Adolescents: The Strengths and Difficulties Questionnaire (Goodman, 1997)

Phase II: Validity Study

14-17 year-olds

1. Generate cut-off score for study eligibility (N=178 in validity study)

2. Evaluate the psychometric properties of the instrument using full sample screened (N=667)

Purpose of Instrument Validation Study

□ Three levels of stringency in defining a local case of two tam, par, kumu, gin lugero/kwo maracoand ma lwor:

1. Caregiver reported their child having the syndrome for at least one month (least stringent)

2. Child reported having the syndrome for at least one month

3. Both child and caregiver agreed on presence of syndrome for at least one month (most stringent).

□ All cases as defined above also demonstrated some degree of functional impairment.

Determination of Local “Caseness”

*1 out of 3 depression syndromes**1 out of 5 APAI syndromes

Syndrome Specific Analyses

Concordance Discordance

Norm Presence of

Syndrome

Absence of

Syndrome

Possible

Syndrome

Apai Subscale Range Mode M (SD) M (SD) M (SD) p M (SD)

Two Tam 1-43 14 16.38 (7.69) 21.36 (7.87) 13.25 (6.42) *** 17. 02 (7.25)

Kumu 0-35 10 12.58 (6.76) 15.77 (6.29) 7.67 (5.06) *** 11.52 (6.17)

Par 0-40 10 13.58 (7.17) 16.24 (6.86) 7.75 (5.78) *** 11.36 (6.09)

Ma lwor 0-29 6 8.62 (4.79) 10.47 (4.88) 6.49 (4.05) *** 7.90 (4.45)

Kwo Maraco 0-21 0 2.34 (3.13) 7.03 (5.15) 1.79 (2.35) *** 2.99 (3.63)

APAI Total

Depression*2-87 30 32.07 (15.55) 37.20 (14.75) 16.64(10.05) *** 26.67 (13.65)

APAI Total Score** 3-125 31 39.79 (19.19) 45.54(18.70) 20.89(11.88) *** 32.15 (15.71)

Prosocial

Behaviors24-5 16 14.64 (3.48)

□ Internal Consistency: Cronbach’s alpha coefficients:

• Combined APAI depression-like syndrome scale (35 items for two tam, par and kumu syndromes) α=0.92; total APAI problem score =0.93 (N=662)

• Two tam (16 items) α=0.84

• Par (17 items) α=0.85

• Kumu (13 items) α=0.83

• Ma Lwor (12 items) α=0.67

• Kwo Maraco (11 items) α=0.77

• Pro-social scale (8 items) α =0.75

□ Inter-rater & Test-retest Reliability:

• Test-retest: r=.84 for the total depression score/ total APAI score (N=30)

• Inter-rater: r=.84 for the total depression score/ r=.74 for total APAI score (N=19)

* Similar findings by gender and in parent report

Psychometric Properties of the APAI

Convergent Validity of the APAI with the SDG (Goodman, 1997)

APAI Syndrome Corresponding SDQ scale

Correlation

between APAI &

SDQ scales

Two tam Emotional Problems 0.63***

Kumu Emotional Problems 0.61***

Par Emotional Problems 0.60***

Malwor Hyperactivity 0.46***

Kwo Conduct problems 0.46***

Total Depression

SyndromesSDQ Emotional Problems 0.69***

Discriminant Validity of the APAI

Area under curve

(SE), [CI] using APAI

Optimal cut offs

(abnormal range)SENS SPEC

Two tam 0.79, (0.02), [.74-.84] 16 (16-43) 75.00 71.21

Kumu 0.85, (0.02), [.81-.89] 10 (10-35) 77.61 73.98

Par 0.84, (0.03), [.78-89] 10 (10-40) 77.58 68.57

Malwor 0.74, (0.03), [.69-.80] 7 (7-29) 70.46 67.26

Kwo Maraco 0.84, (0.04), [.77-.92] 3 (3-21) 76.47 82.10

1 out of Depression

Syndromes0.74, (0.02), [.70-.78] 25 (26-87) 77.63 54.48

1 out of 5

Syndromes 0.74, (0.02), [.70-.78] 34 (35-125) 70.10 61.74

Refining the AYPA

Item Response Theory (IRT)

□ In psychometrics, item response theory (IRT)

(also known as latent trait theory, strong true

score theory, or modern mental test theory) is a

paradigm for the design, analysis, and scoring

of tests, questionnaires, and similar instruments

measuring abilities, attitudes, or other variables

□ it does not assume that each item is equally

difficult or good at capturing an underlying trait like depression

IRT

□ generally regarded as superior to classical test

theory, it is the preferred method for developing

and refining scales

□ The name item response theory is due to the focus

of the theory on the item, as opposed to the test-

level focus of classical test theory

□ IRT models the response of each examinee of a

given ability (or severity of underlying distress) to

each item in the test. (Chan, Griffiths, Gao, Chan, & Fok, 2008; Yu et al., 2010)

Advantages of IRT

□ Has been applied to mental health

measurement research to estimate the

probability that a given individual characterized

by higher or lower levels of latent psychological distress will endorse each possible response

option

□ Additionally, IRT models can estimate item bias or differential item functioning across different subgroups (such as age, race, gender, differing syndrome types or severity).

(Chan, Griffiths, Gao, Chan, & Fok, 2008; Yu et al., 2010)

Demographic characteristics of Acholi Luo youth

study sample

Item summary frequency of full APAI questions (N= 667)

Item summary frequency of full APAI questions (N= 667), continued

Goal of the APYA Study

□ Applied Item Response Theory (IRT)-based

analyses were used to shorten and refine

the original APAI scale into a more

generalizable AYPA, and to determine:

□ The dimensionality of its scales

□ Item parameters for each item in each subscale

□ Accuracy of the refined measure for assessing

underlying constructs of distress as well as

prosocial behavior/attitudes

□ Total information provided by each item on its

relevant subscale (items not all assumed to be

equal)(Betancourt, Bass, et al., 2009; Betancourt, Speelman, et al., 2009),

Item Parameters

(a) item difficulty - In mental health measurement it can be

thought of as item severity since the latent trait is distress rather than ability

(b) item discrimination

– the accuracy with which an item measures the latent trait (how well an item of a given difficulty sorts people at high and low levels of the trait)

Examination of these parameters can lead to scale calibrations to ensure that items are scored according to their relation to the latent trait.

(Deng, 2010; Wang et al., 2000)

),

Refinement of the APAI to create the APYA

□ Used full sample of youth screened using

APAI for the RCT of depression

interventions for Acholi youth aged 14-17

(N = 667)

□ Participants also completed a

demographic questionnaire and a mea-

sure of functional impairment

Progression of the IRT Analysis

Results

□ Following EFA we examined the fit of three, four or five factor solutions

□ After elimination of 19 APAI items with loadings less than 0.4, the fit for the four-factor CFA model was the best and most clinically useful

□ IRT analyses confirmed that a four factor solution best fit the data and validated broad domains manifest in the qualitative data:

□ Internalizing problems

□ Externalizing problems

□ Prosocial attitudes/behaviors

□ Somatic complaints without medical cause

Analysis of Concurrent Validity

□ Functional Impairment: □ Subscale for internalizing emotional and

behavioral problems was correlated with impairment in functioning at 0.36 (p<0.001)

□ Subscale for externalizing problems was correlated with impairment in functioning at 0.28, (p<0.001)

□ Subscale for somatic complaints without medical cause was correlated with impairment in functioning at 0.3 (p < 0.001)

□ Subscale for prosocial attitudes and behavior was not significantly correlated with impairment in functioning (0.07, p = 0.15).

Test Information Curves

□ Items in the internalizing factor contribute high

discrimination &difficulty parameters/ high information content (wide distribution across different levels of

severity, compared to the other subscales)

□ Externalizing/conduct items have next highest

discrimination parameters, but cover a narrower, more

skewed range based on the difficulty parameter.

□ Prosocial items cover a large range of the trait, but

total information lower than the previous two traits

mentioned.

□ Somatic items also cover a wide range of the trait, but

yield the lowest information overall when compared to

the other traits.

□

Test Information Curves

for the APYA Subscales

Alignment with other SSA data

□ 72% of items on the internalizing and

externalizing subscales matched to total

problem items in qualitative data collected

in Rwanda (Betancourt, Rubin-Smith, et al.,

2011);

□ 68% of items on the internalizing subscale

were matched to internalizing problems

identified in qualitative data from Sierra

Leone (Betancourt, 2010).

Item Performance

□ Internalizing: lowest discriminating item was “I

feel I can do nothing to help myself” (a=0.40)--means that responses to this question do less well

in distinguishing between those with low and

high levels of underlying internalizing problems.

□ Item was retained because in a camp, options

for helping oneself are limited by circumstances,

and because modification indices suggest that

dropping it did not improve the scale.

□ Item’s performance needs to be tested in other

settings to determine if the item performance is

improved in situations where children and youth

can exert greater self-efficacy.

□

Item Performance□ Internalizing: highest discriminating item “I have lots

of worries” (a=0.78) -- this item distinguishes well

between youth with low versus high levels of

underlying internalizing problems.

□ Externalizing: highest discriminating item “I insult

friends” (a=0.90), whereas items such as “I deceive”

were less discriminating (a=0.60).

□ Prosocial attitudes and behavior, the most

discriminating item was “I cooperate with others”

(a=0.73), while items like “I play together with others”

showed poorer discrimination (a=0.40).

□ Somatic complaints: “I have pain all over my body”

showed the best discrimination (a=0.76), while the

item “I get headaches” was less discriminating

(a=0.60).

Item response theory parameters (a = discrimination)

(b = thresholds) for the 41 AYPA items

Chronbach’s Alpha of subscales

in this sample

□ Internalizing problems (α=0.88)

□ Externalizing problems (α=0.83)

□ Prosocial attitudes/behaviors (α=0.72)

□ Somatic complaints without medical cause

(α=0.74)

Limitations

□ Reliance on youth self-reports remains an important

limitation.

□ Finding that the subscale for somatic complaints

without medical cause was weakest must be

considered in the context of a setting with high disease

burden and limited services --even with adequate

instructions, participants may find it challenging to

distinguish somatic complaints not due to a medical

cause from those due to illnesses such as malaria. No

independent medical evaluation was able to be

conducted to make this determination

□ In future research on the AYPA in SSA, it would be

valuable to assess validity using comparison to the

ratings of a child and adolescent psychiatrist familiar

with the culture and setting.

Broader Applicability of APYA?

□ Final refined and validated AYPA measure demonstrates potential for broader applicability to other African settings

□ Qualitative data on expressions of child mental health problems and prosocial behaviors in Rwanda and Sierra Leone share many similarities with items in the AYPA’s internalizing and externalizing problem scales.

□ Phrasing of items in the AYPA is more comparable to how mental health problems of children and adolescents may be expressed in SSA□ Expressions of sadness (i.e. having “pain in the heart” and “sitting

with cheek in palm”)

□ Social withdrawal (“staying away from others”) and anxiety or rumination (“thinking too much” )

□ Conduct (“being a rough person”)

□ Prosocial attitudes/behaviors (“welcoming others”)

□ AYPA was used successfully in other RCTs among war-affected youth in the Democratic Republic of Congo (O’Callaghan & McMullen, 2013)

Conclusions

□ Refined and shortened AYPA measure is a promising tool for assessing emotional and behavioral and prosocial attitudes and behaviors among youth in SSA

□ IRT-driven analyses can be applied to the refinement of instruments on emotional and behavioral problems in children derived from qualitative data in a war-affected and LMIC setting

□ Approach taken to develop APYA and examine its psychometric properties can be replicated in future research to expand measures available for use in LMICs

Thank you

Acknowledgements

□ Analysis was funded by the National Institute of Mental

Health, the National Center for Minority Health and Disparities, and by the Francois-Xavier Bagnoud Center

for Health and Human Rights

□ Primary data collection was supported by World Vision

Uganda and War Child Holland

□ The authors are grateful to the children, youth and

families of the Awer and Unyama IDP camps who

participated in this research

References

□ Betancourt T.S. (2010) A longitudinal study of psychosocial adjustment and community reintegration among former child soldiers in Sierra Leone. International Psychiatry, 7(3), 60–62.

□ Betancourt T.S., Bass J., Borisova I., Neugebauer R., Speelman L., Onyango G., Bolton P. (2009a) Assessing local instrument reliability and validity: a field-based example from northern Uganda. Social Psychiatry and Psychiatric Epidemiology, 44(8), 685–692.

□ Betancourt T.S., Speelman L., Onyango G., Bolton P. (2009b) A qualitative study of mental health problems among children displaced by war in northern Uganda. Journal of Transcultural Psychiatry, 46(2), 238–256. DOI: 10.1177/ 1363461509105815

□ Betancourt T.S., Meyers-Ohki S., Stevenson A., Ingabire C., Kanyanganzi F., Munyana M.C., Mushashi C., Teta S., Fayida I., CyamatareF.R., Stulac S.N., Beardslee W. (2011a) Using mixed-methods research to adapt and evaluate a family strengthening intervention in Rwanda. African Journal of Traumatic Stress, 2(1), 32–45.

□ Betancourt T.S., Rubin-Smith J., Beardslee W.R., Stulac S.N., Fayida I., Safren S.A. (2011b) Understanding locally, culturally, and contextually relevant mental health problems among Rwandan children and adolescents affected by HIV/AIDS. AIDS Care, iFirst, 1–12. DOI: 10.1080/09540121.2010.516333

□ Bolton P. (2001) Cross-cultural validity and reliability testing of a standard psychiatric assessment instrument without a gold standard. Journal of Nervous and Mental Disease, 189(4), 238–242. DOI: 10.1097/00005053-200104000-00005

□ Bolton P., Tang A.M. (2002) An alternative approach to cross-cultural function assessment. Social Psychiatry and Psychiatric Epidemiology, 37(11), 537–543. DOI: 10.1007/s00127-002-0580-5

□ Bolton P., Wilk C.M., Ndogoni L. (2004) Assessment of depression prevalence in rural Uganda using symptom and function criteria. Social Psychiatry and Psychiatric Epidemiology, 39(6), 442–447. DOI: 10.1007/s00127-004-0763-3

□ Bolton P., Bass J., Betancourt T.S., Speelman L., Onyango G., Clougherty K.F., Neugebauer R., Murray L., Verdeli H. (2007) Interventions for depression symptoms among adolescent survivors of war and displacement in northern Uganda: a randomized controlled trial.Journal of the American Medical Association, 298(5), 519–527. DOI: 10.1001/jama.298.5.519

□ Chan E.Y.Y., Griffiths S., Gao Y., Chan C.W., Fok T. F. (2008) Addressing disparities in children’s health in China. Archives of Disease in Childhood, 93, 346–352. DOI: 10.1136/adc.2007.130146

□ Cheng T.A., Williams P. (1986) The design and development of a screening questionnaire (CHQ) for use in community studies of mental disorders in Taiwan. Psychological Medicine, 16(02), 415–422. DOI: 10.1017/S0033291700009247

□ Child Rights International Network. (2001) Rwanda Country Report, New York, Better Care Network.

□ Chisholm D., Flisher A.J., Lund C. Patel V., Saxena S., Thornicroft G., Tomlinson M. (2007) Scale up services for mental disorders: a call for action. Lancet, 370(9594), 1241–1252.

□ Deng Q. (2010) Natural disasters, migration and urban insecurity in China. In Interconnection Among Global Problems in NortheastAsia, San Francisco, CA, Nautilus Institute for Security and Sustainability.

□ Dorit R.-S. (2001) Reducing risk for children in changing cultural contexts: recommendations for intervention and training. Child Abuse & Neglect, 25(2), 231–248. DOI: 10.1016/s0145- 2134(00)00242-8

References

□ Guha-Sapir D., van Panhuis W.G., Degomme O., Teran V. (2005) Civil conflicts in four African countries: a five-year review of trends in nutrition and mortality. Epidemiologic Reviews, 27(1), 67–77. DOI:

□ 10.1093/epirev/mxi010

□ Jacob K.S., Sharan P., Mirza I., Garrido-Cumbrera

□ I.M., Seedat S., Mari J.J., Saxena S. (2007) Mental health systems in countries: where are we now? Lancet, 370, 1061–1077. DOI: 10.1016/S0140-6736(07)61241-0

□ Newman D.A. (2003) Longitudinal modeling with randomly and systematically missing data: a simulation of ad hoc, maximum likelihood, and multiple imputation techniques. Organiza- tional Research Methods, 6(3), 328–362. DOI: 10.1177/1094428103254673

□ Newnham E.A, Doyle E.L., Sng A.A.H., Hooke G.R., Page A.C. (2012) Improving clinical outcomes in psychiatric care with touch-screen technology. Psychological Services, 9(2), 221–223.

□ Patel V. (2003) Cultural issues in measurement and research. In Prince M., et al. (eds) Practical Psychiatric Epidemiology, pp. 43–64, Oxford, Oxford University Press.

□ Patel V. (2007) Mental health in low- and middle- income countries. British Medical Bulletin, 81, 81–96. DOI: 10.1093/bmb/ldm010

□ Patel V., Simunyu E., Gwanzura F., Lewis G., Mann A. (1997) The Shona Symptom Questionnaire: the development of an indigenous measure of common mental disorders in Harare. Acta Psychiatrica Scandinavica, 469–475. DOI: 10.1111/j.1600-0447.1997.tb10134.x

□ Patel V., Araya R., Chatterjee S., Chisholm D., Cohen A., De Silva M., Hosman C., McGuire H., Rojas G., van Ommeren M. (2007a) Treatment and prevention of mental disorders in low-income and middle-income countries. The Lancet, 370(9591), 991–1005. DOI: 10.1016/S0140-6736(07)61240-9

□ Patel V., Flisher A.J., Hetrick S., McGorry P. (2007b) Mental health of young people: a global public health challenge. Lancet, 369, 1302–1313. DOI: 10.1016/S0140-6736(07)60368-7

□ Prince M., Patel V., Saxena S., Maj M., Maselko J., Phillips M.R., Rahman A. (2007) No health without mental health. The Lancet, 370(9590), 859–877. DOI: 10.1016/S0140-6736(07) 61238-0

□ Psychosocial Working Group. (2003) Psychosocial Intervention in Complex Emergencies: A Conceptual Framework, Edinburgh, Centre for Inter- national Health Studies, Queen Margaret University College.

□ Pynoos R.S., Nader K. (1988) Psychological first aid and treatment approach to children ex- posed to community violence: research implications. Journal of Traumatic Stress, 1(4), 445–473. DOI: 10.1002/jts.2490010406

□ Saraceno B., van Ommeren M., Batniji R., Cohen A., Gureje O., Mahoney J., Sridhar D., Underhill C. (2007) Barriers to improvement of mental health services in low-income and middle-income coun- tries. The Lancet, 370(9593), 1164–1174. DOI: 10.1016/S0140-6736(07)61263-X

□ Saxena S., Thornicroft G., Knapp M., Whiteford H. (2007) Resources for mental health: scar- city, inequity, and inefficiency. The Lancet, 370(9590), 878–889. DOI: 10.1016/S0140- 6736(07)61239-2

References

□ Srinivasan T.N., Suresh T.R. (1990) Non-specific symptoms and screening of non-psychotic morbidity in primary care. Indian Journal of

Psychiatry, 32(1), 77–82.

□ UNICEF. (2009) Machel Study 10-Year Strategic Re- view: Children and Conflict in a Changing World, New York, UNICEF.

□ Van Ommeren M. (2003) Validity issues in transcultural epidemiology. The British Journal of Psychiatry, 182(5), 376–378. DOI:

10.1192/bjp.182.5.376

□ Wang X., Gao L., Shinfuku N., Zhang H., Zhao C., Shen Y. (2000) Longitudinal study of eathquake-related PTSD in a randomly selected

community sample in North China. American Journal of Psychiatry, 157, 1260–1266. DOI: 10.1176/appi.ajp.157.8.1260

□ Yang H., Stanton B., Cottrel L., Kaljee L., Galbraith J., Li X., Cole M., Harris C., Wu Y. (2006) Pa- rental awareness of adolescent risk

involvement: implications of overestimates and underestimates. Journal of Adolescent Health, 39(3), 353–361. DOI: 10.1016/j.

jadohealth.2005.12.008

□ Yu X., Lau J.T.F., Zhang J., Mak W.W.S., Choi K. C., Lui W.W.S., Zhang J., Chan E.Y.Y. (2010) Posttraumatic growth and reduced suicidal

ideation among adolescents at month 1 after the Sichuan earthquake. Journal of Affective Disorders, 123, 327–331. DOI: 10.1016/j.

jad.2009.09.019