The Relevance of Multilevel Statistical Methods for Identifying

8/11/2019 The Relevance of Multilevel Statistical Methods for Identifying

1/7

Social Science & Medicine 58 (2004) 19611967

Commentary

The relevance of multilevel statistical methods for identifying

causal neighborhood effects

S.V. Subramanian*

Harvard School of Public Health, Department of Society, Human Development and Health, 677 Huntington Avenue,

KRESGE, 7th floor, Boston, MA 02115-6096, USA

Introduction

Our current understanding of, and continued interest

in, the social determinants of health is in large part due

to the important empirical contributions that in the last

10 years have consistently shown an association between

neighborhood factors and individual health (for an

excellent review of these works, see Diez Roux, 2001;

Ellen, Mijanovich, & Dillman, 2001; Pickett & Pearl,

2001; Sampson & Morenoff, 2002; OCampo, 2003).

Three aspects underscore the importance of these

empirical studies. First, the impact of neighborhood

characteristics, independent of individual factors, has

been shown to exist across a range of public health

outcomes (e.g., all-cause mortality, cardiovascular mor-

tality, infant and child health, womens health, chronic

diseases, mental health, health behaviors, health percep-

tions, delinquency, violence). Second, multiple aspects of

neighborhood (e.g., deprivation, inequality, neighbor-

hood ties, social control, institutional resources) have

been shown to be associated with public health out-

comes. Finally, much of this research utilizes some form

of multilevel statistical methodsan improved quan-

titative methodology especially suited to modeling

neighborhood clustering and variationin order to

estimate neighborhood effects. Does the multilevel

empirical evidence for independent neighborhood ef-fects, accumulated over the last 10 years across a range

of public health outcomes measuring a range of

neighborhood factors, provide a basis to conclude that

neighborhoods matter for health?

In this issue, Oakes (2003) reviews the capability of

multilevel regression models to identify and estimate

neighborhood effects. Notably, Oakes presents four

conceptual/methodological issues that threaten the

ability of multilevel models to detect the causal effects

of neighborhoods on health, thereby casting doubt onthe trustworthiness of existing multilevel empirical

evidence. The alternative strategy (to multilevel model-

ing), Oakes argues, lies in community trials. Oakes

concludes that multilevel methods, while having en-

riched the etiological discussion on the linkages between

neighborhoods and health, are incapable of producing

valid estimates of neighborhood effects: we do not

know yet (whether there is an independent effect of

neighborhoods on health or not), but now we know that

our methodology must change if we wish to find out.

In this commentary, I argue that Oakes position is

rather extreme and overly pessimistic. In the interest of a

more vigorous discussion, this commentary scrutinizes

the claims and conclusions of Oakes essay. In the spirit

of this timely reflection on the usefulness of multilevel

methods, the commentary makes an attempt to raise the

level of awareness on issues related to designing,

conducting and interpreting a multilevel analysis.

The (mis)estimation of neighborhood effects: a critique

In an essay that otherwise raises key methodological

challenges for research on neighborhoods and health,

the motivation and background to the discussion ismisplaced. At the outset, the essay fails to recognize that

issues of causality are one of substantive subject matter

and not one of statistical methodologies. To add to this,

Oakes mixes up the issues that are intrinsic to study

design as compared to those that pertain to metho-

dological/modeling strategies in order to make a

confusing case against the application of both multilevel

methods and observational data. The confusing nature

of the claim is evident in his proposal for randomized

community trials as an alternative to multilevel ana-

lysis. Since the alternative is proposed after painstak-

ingly rehearsing the multilevel regression methodology

ARTICLE IN PRESS

*Tel.: +1-617-432-6299; fax: +1-617-432-3123.

E-mail address: [email protected]

(S.V. Subramanian).

0277-9536/$- see front matterr 2003 Elsevier Ltd. All rights reserved.

doi:10.1016/S0277-9536(03)00415-5


2/7

as it relates to identifying neighborhood effects, followed

by a passionate criticism of it (and not after critiquing

the limits of cross-sectional observational data-design-

based inferences), Oakes seems to imply that rando-

mized community trials are an alternative to multilevel

methods. In order to evaluate Oakess claims (especially

as they relate to multilevel methods), the distinctionbetween methodological strategies and study design

is critical.

The framework of randomized community trials,

which is not without it merits, is completely independent

of the advantages or shortcoming of multilevel modeling

strategies. Importantly, even when we have data from

randomized community trials, a multilevel analytical

framework would still provide the most appropriate

statistical strategy to analyze the data for detecting and

describing neighborhood effects, besides offering other

substantive advantages (Subramanian, Jones, & Dun-

can, 2003). Analysis of cluster trials (regardless of thelevel of randomization) either involving schools or

clinics or workplaces are good analogy to Oakes

alternative of neighborhood-based community trials;

and the relevance of multilevel methods to identify

causal cluster effectsbe it schools (De Vries et al.,

1994) or clinics (Bach et al., 1995) or workplaces

(Hedeker, McMahon, Jason, &Salina, 1994)are well

recognized (Donner, 1998). Indeed, an acknowledgment

of this would make any potential basis to discourage the

use of multilevel methods less compelling. The proposal

for community trials, therefore, is not only compatible

with multilevel methods, rather multilevel methods with

its ability to model treatment-based clustering and

treatment heterogeneity would arguably be the most

appropriate methodology to analyze community trial

data.

Moreover, randomized study designs are probably

best suited when we have a single exposure of interest.

While Oakes recognizes the limitations to community

trials paradigm, he is almost silent about certain

fundamental drawbacks (Sorensen, Emmons, Hunt, &

Johnston, 1998). For instance, often community-level

interventions are beset with contamination problems

(e.g., changing secular trends in background factors, or

a moving target in the control community, or theprocess of what and who defines communities and

neighborhoods). Conversely, with community-based

interventions, the more specific the intervention (e.g.,

building a playground to encourage exercise), the less

generalizable it is and the less it represents the

constellation of societal determinants that shape popu-

lation distribution of health, including its geographic

variability. Surely, it would not be a practicable social

epidemiology to design and test every potential com-

munity variable in this manner. For Oakes to state that

the observational evidence on neighborhood effects is

not trustworthy, but still suggest that we go ahead and

conduct randomized experiments across communities is

ethically questionable both from a research and from a

policy intervention perspective.

The core issue of Oakes essay is about the limits of

observational data designs as they relate to the

intellectual and practicable project of identifying causal

neighborhood effects. Crucially, the central tenets ofOakes essay, therefore, stand independent of the rather

exhaustive (and well-known) discussion of multilevel

methods and as such Oakes claim that no one appears

to haveconceptually developedthe multilevel model with

an eye on causal inference is incorrect (see among

othersJones, 1991;Duncan, Jones,&Moon, 1993, 1996;

Jones & Moon, 1993; Jones & Duncan, 1995, 1996;

Subramanian, Kawachi, & Kennedy, 2001; Subrama-

nian et al., 2003). While researchers may not have

specifically chosen to use the term causal, a reading of

these would reveal that the motivation for the metho-

dological exposition of multilevel models is indeed tounderstand the causal role of contexts. The appropriate

framework for Oakes to discuss the important issues

would have been a detailed background critique on the

limits of cross-sectional observational data-design-based

inferences. Notwithstanding this shortcoming, Oakes

identifies four obstacles to detecting causal neighbor-

hood effects. The remainder of this commentary

scrutinizes each of the claims and will argue that

there is nothing in these claims that would force us

to abandon a sensible application of multilevel statis-

tical techniques, especially for a practicable social

epidemiology.

The first obstacle, according to Oakes, is the

feasibility of identifying true neighborhoods differ-

ences. Applied multilevel research relies on partitioning

the individual-compositional effects while estimating

the true contextual differences between neighbor-

hoods. The notion of partitioning, however, contradicts

the foundational idea that people make places and

places make people; an idea germane to neighborhoods

and health, and much of social epidemiological,

research. Oakes argues that the process of social

stratification will generate complete confounding

between the background attributes of persons in a given

neighborhood and (approximately) complete separationbetween the background attributes of people in other

neighborhoods. Given this, in a perfectly specified

individual-compositional model (or selection model

to use Oakes phrase) the neighborhood differences

would move towards zero and as such eliminate the

possibility of explaining the neighborhood differences.

While social stratification complicates neighborhood

comparisons, the extent to which they confound the

comparisons is an interesting empirical question. More-

over, for this claim to hold, the process through which

people select neighborhoods would have to be comple-

telynon-random atalltimes. Indeed, if we believe this to

ARTICLE IN PRESS

S.V. Subramanian / Social Science & Medicine 58 (2004) 196119671962


3/7


4/7

1. Importantly, the structure weights the amount of time

that individual 25 spent in her current neighborhood

residence (20% between times 1 and 2) and 80% in her

previous neighborhood residence. Importantly, such a

repeated measure, multiple membership should allow

an estimation of changing neighborhood effects, con-trolling for the changing population composition and

thus offer some mileage in addressing the endogeneity in

neighborhood exposures. Other such creative modifica-

tions can be adopted for analyzing causal neighborhood

effects. The general issue of endogeneity has certain

unique implications for multilevel models and it is useful

to take cognizance of some of the methodological work

underway to address this challenge (Spencer, 2002). The

problem of endogeneity while by no means entirely

surmountable to everybodys satisfaction can none-

theless be addressed in some practicable manner within

a multilevel framework.

The third obstacle identified by Oakes is the issue of

extrapolation. Oakes suggestion that contexts (and

individuals living in them) are unique and hence not

comparable clearly cannot be a basis for invalidating the

method. Instead, the uniqueness of neighborhoods raises

empirical challenges for measuring these contexts and to

understand why they are different. Also, at issue is

preponderance of effect; if, in general, people in poor

areas have the worst health, then even though each poor

area may be unique in some way, there is still an over-

riding effect of impoverishment.

It is also worth underscoring that multilevel models

are not about modeling specific neighborhoods (asOakes describes) even though they do allow posterior

predictions for specific neighborhoods. Rather, the

neighborhood-specific differences are viewed as a ran-

dom sample of differences that are derived from a

population of neighborhoods that can be modeled as

a function (constant, linear, quadratic) of individual as

well as neighborhood exposures (Subramanian et al.,

2003). Quite clearly then, researchers need to consider

the assumption of exchangeability not simply in

relation to who lives in these neighborhoods, but more

importantly whether the sampled neighborhoods are

representative of the population of the neighborhoods

for which we wish to make inferences (Stoker& Bowers,

2002). It is of course critical that extrapolations and

predictions are not made outside the range of possible

predictor variables and to that extentallstatistical models

need to ensure that both individuals and neighborhoods

are drawn from a population to which we wish to makeinference and predictions and as such have exchange-

able properties with sufficient sample size.

The final obstacle of disequilibria, Oakes argues, is a

consequence of failing to recognize that a treatment

given to one person does not affect (the treatment given

to) another person. Such an assumption appears to be

fundamentally inconsistent with the very basis of

investigating a community effect, which assumes that

diseases and health outcomes are clustered in the

population (not just in the data sample). One can see

the partial relevance of this assumption in conditions

where the intervention involves moving (poor) in-

dividuals to (rich) neighborhoods and thereby chan-

ging the equilibrium of the recipient neighborhoods.

Even here, from a causal perspective our interest is in the

underlying association and it is hard to see how that

might change; while the rich neighborhood may no

longer be rich but that does not mean that the

underlying association between area deprivation and

health has changed. If we shift our focus to neighbor-

hood-level treatment (rather than moving individuals),

arguably, the Stable Unit Treatment Value Assumption

should hold. For instance, the health effects on residents

of one neighborhood to having a playground will not be

different depending on whether another neighborhooddoes or does not have a playground. Indeed, considering

the complex issues of proximity and access that may not

respect neighborhood boundaries raises challenges for

estimating valid neighborhood effects, in general. How-

ever, a fair discussion of this issue is contingent on

access to a host of details through which such

interventions are designed and without knowledge of

these conditions it is hard to comment on the relevance

of this issue for observational attempts to detect

neighborhood effects. Moreover, this issue does not

demonstrate anything that invalidates the multilevel

methodology.

ARTICLE IN PRESS

1 25

1

1 50

25

1 30

50

0.40.8

Level-1: Time

Level-2: Individuals

Level-3:Neighborhoods

0.60.2 0.6 0.4

21 21 21 21 2121

Fig. 1. Multilevel structure of repeated measurements of individuals over time across neighborhoods with individuals having multiple

membership to different neighborhoods across the time span.



5/7

Challenges in multilevel analysis

The preceding commentary was not intended to

present an unconditional defense of multilevel models.

Indeed, echoing Harvey Goldstein, multilevel models

are not a panacea and like all statistical methods, need

to be used with care and understanding (Goldstein,2003). Given the complexity in the parameterization of

multilevel models, caution must be exhibited at both

design and analysis stage in the form of conceptual

justification and rigorous diagnostics, respectively.

Meanwhile, research on neighborhoods and health,

fueled by the relatively easy access to multilevel

modeling procedures, clearly risk making questionable

inferential leaps. Indeed, Oakes essay serves this general

purpose rather well. In the interest of raising the level of

awareness of multilevel applications in public health

research, I outline three critical issues for multilevel

analysis.The first relates to issues of multilevel designs and

structures. This relates to the issue of delimiting

neighborhood and identifying neighborhood boundaries

for empirical research (Sampson & Morenoff, 2002),

and importantly recognizing other multiple spatial and

non-spatial contexts within and around which neighbor-

hoods operate (Subramanian et al., 2003). Indeed,

current applications have failed to recognize this multi-

plicity of neighborhood contexts and as such do not go

beyond the two-level conceptualizations of individuals

at lower level hierarchically nested within neighbor-

hoods at a higher level. While it is now possible to

implement complex hierarchical as well as non-hier-

archical structures (Leyland & Goldstein, 2001) under

substantially realistic and sophisticated assumptions

(Browne, 2002), there is a need to develop clearer

methodological and applied understanding on combin-

ing multilevel data designs (e.g., multivariate, repeated

measures and cross-classified with multiple member-

ships) and thus reducing the knowledge gap between

statistical advances and applied quantitative research.

An immediate concern, meanwhile, for interpreting

the multilevel structures is the identification of appro-

priate levels (such as households, or other larger macro-

levels within which neighborhoods operate) or conver-sely, the implication of missing levels. Identifying true

neighborhood differences also requires identifying

true neighborhoods; an aspect on which much of

the applied work, including Oakes essay, is entirely

silent.

The second relates tomultilevel model specificationfor

making causal inferences. The flexibility to specify and

estimate complex parameters also means that multilevel

models, by definition, are highly conditional and

sensitive to model specification. Much of the existing

application of multilevel research continues to focus

largely on the fixed parameters; stated differently, the

average effects of a particular exposure (be it individual

or neighborhood) on the individual outcome. However,

in multilevel models the same fixed part of the statistical

model can be estimated under a range of random part

specifications (Subramanian et al., 2003). Consequently,

a clear understanding and justification for specifying the

within-neighborhood (level 1) and the between-neighborhood (level 2) model, especially in the random

part, is completely lacking in the current applications.

Specifically, this relates to the variancecovariance

structures that can be specified at each of the desired

level. While the notion of complex variance structures is

recognized, the assumption of homoskedastic variances

continues to prevail (at levels 1 and 2) and this has

critical implications in terms of making inferences about

neighborhood differences as well as about average

predictive role of exposures. In addition, of course,

there is also the issue of specifying the explanatory

(substantive) within-neighborhood or the individual-level model in the fixed part of the multilevel statistical

model. Typically, the tendency is to over-specify or

exhaustively control the individual model in order to

either ensure a perfect specification of the within-

neighborhood model (Oakes, 2003) or to be conservative

and cautious while estimating neighborhood differences.

Either way, there are substantive issues that relate to

model specifications, especially so in multilevel models.

One general framework would be to conduct sensitivity

analysis to ascertain the extent to which findings are

robust to alternate model specifications.

Finally, there are critical issues to interpreting multi-

level coefficients. Researchers rarely report or discuss

any diagnostic testing of the models fitted. While

diagnostic procedures are being implemented and

methodological work in this area is underway (Langford

& Lewis, 1998), applications of these methods need to

routinely report and discuss the extent to which different

models satisfy the statistical assumptions underlying

these models. Specifically, for multilevel neighborhoods

research, these could include: testing for the assumptions

of normality for random coefficients at higher and lower

levels; and testing of the assumptions related to

independence of residuals at different levels and of the

random part to the fixed part. A related concern whileinterpreting multilevel neighborhood studies is the issue

of power (Snijders& Bosker, 1993; Snijders, 2001). For

instance, how many neighborhoods and individuals

within neighborhoods do we need in order to model

the average effects of a neighborhood exposure that is

hypothesized to have a differential effect on the outcome

depending upon individual SES. Or, how many neigh-

borhoods are required to estimate the differential effect

of individual SES that is seen to vary across neighbor-

hoods. Such questions, and others, compel researchers

to consider power issues while designing, modeling and

interpreting multilevel coefficients.

ARTICLE IN PRESS

S.V. Subramanian / Social Science & Medicine 58 (2004) 19611967 1965


6/7

Concluding remarks

The clear strength of multilevel models for a causal

analysis of neighborhood effects lies in its ability to

model complex heterogeneities such that individual as

well as neighborhood exposures are not simply con-

ceptualized in terms of their average effect but rather interms of their true population heterogeneity. Modeling

heterogeneity is not only more realistic, and therefore a

better basis for a practicable social epidemiology, but

also provides important feedback loop to reframe our

questions related to average causal effects. Indeed, as

mentioned at the outset, the issue of validating causal

effects are essentially one of subject matter and less so of

the statistical methods employed.

The aim of this commentary was to balance Oakes

pessimism with a message that realistically complex

multilevel models (Best, Spiegelhalter, Thomas, &

Brayne, 1996) are crucial to not only answering theoriginal research questions but also to motivate new

causal questions, the empirical answers to which are less

well understood. While multilevel applications on

observational data-sets must be grounded in substantive

theories with careful consideration of what to measure,

specify and how to be critical of findings, it is clear that

the multilevel modeling approach can bring extra

predictive power, description and precision to our

efforts to understand causal neighborhood effects. We

have little evidence, as yet, to believe otherwise.

Acknowledgements

I am grateful to Harvey Goldstein and William

Browne for their insights on issues related to modeling

and interpreting higher-level variances. I thank Kim

Lochner, Stephen Gilman, Maria Glymour and Nancy

Krieger for their helpful comments on an earlier version

of this commentary.

References

Bach, D., Van den Berg-Segers, A., Hubner, A., Van Breukelen,

G., Cesana, M., & Pletan, Y. (1995). Rufloxacin once daily

versus ciprfloxacin twice daily in the treatment of patients

with acute uncomplicated pyelonephritis. Journal of Urol-

ogy, 154, 1924.

Best, N. G., Spiegelhalter, D. J., Thomas, A., & Brayne, C. E.

G. (1996). Bayesian analysis of realistically complex models.

Journal of Royal Statistical Society A, 159, 232342.

Browne, W. J. (2002). MCMC estimation in MLwiN. London:

Centre for Multilevel Modelling, Institute of Education.

De Vries, H., Backbier, E., Dijkstra, M., Van Breukelen, G.,

Parcel, G., & Kok, G. (1994). A Dutch social influence

smoking prevention approach for vocational school stu-

dents.Health Education Research, 9, 365374.

Diez Roux, A. V. (2001). Investigating neighborhood and area

effects on health.American Journal of Public Health,91(11),

17831789.

Donner, A. (1998). Some aspects of the design and analysis of

cluster randomization trials. Applied Statistics, 47, 95113.

Duncan, C., & Jones, K. (2000). Using multilevel models to

model heterogeneity: Potentials and pitfalls. GeographicalAnalysis, 32(4), 127.

Duncan, C., Jones, K., & Moon, G. (1993). Do places matter?

A multilevel analysis of regional variations in health-related

behavior in Britain. Social Science & Medicine,37, 725733.

Duncan, C., Jones, K., & Moon, G. (1996). Health-related

behaviour in context: A multilevel modelling approach.

Social Science & Medicine, 42(6), 817830.

Ellen, I. G., Mijanovich, T., & Dillman, K.-N. (2001).

Neighborhood effects on health: Exploring the links

and assessing the evidence. Journal of Urban Affairs, 23,

391408.

Goldstein, H. (2003). Multilevel statistical models. London:

Arnold.

Hedeker, D., McMahon, S. D., Jason, L. A., & Salina, D.(1994). Analysis of clustered data in community psychology:

With an example from a worksite smoking cessation project.

American Journal of Community Psychology, 22, 595615.

Jones, K. (1991). Specifying and estimating multilevel models

for geographic research. Transactions of the Institute of

British Geographers, 16, 148160.

Jones, K., & Duncan, C. (1995). Individuals and their ecologies:

Analyzing the geography of chronic illness within a multi-

level modeling framework. Health and Place, 1(1), 2740.

Jones, K., & Duncan, C. (1996). People and places: The

multilevel model as a general framework for the quantitative

analysis of geographical data. In P. Longley, & M. Batty

(Eds.), Spatial analysis: Modelling in a GIS environment(pp. 79104). Cambridge: Geoinformation Group, Ely.

Jones, K., & Moon, G. (1993). Medical geography: Taking

space seriously. Progress in Human Geography, 17(4),

515524.

Langford, I., & Lewis, T. (1998). Outliers in multilevel data.

Journal of the Royal Statistical Society A, 161, 121160.

Leyland, A. H., & Goldstein, H. (Eds.). (2001). Multilevel

modelling of health statistics. Wiley Series in Probability and

Statistics. Chichester: Wiley.

Macintyre, S., & Ellaway, A. (2000). Ecological approaches:

Rediscovering the role of physical and social environment.

In L. F. Berkman, & I. Kawachi (Eds.), Social epidemiology

(pp. 332348). New York: Oxford Press.

Macintyre, S., Ellaway, A., & Cummins, S. (2002). Placeeffects on health: How can we conceptualise, operationalise

and measure them. Social Science & Medicine, 55,

125139.

Oakes, M. (2003). The (mis)estimation of neighborhood effects:

Causal inference for a practicable social epidemiology.

Social Science & Medicine, forthcoming doi:10.1016/j.

socscimed.2003.08.004.

OCampo, P. (2003). Invited commentary: Advancing theory

and methods for multilevel models of residential neighbor-

hoods and health. American Journal of Epidemiology,

157(1), 913.

Pickett, K. E., & Pearl, M. (2001). Multilevel analyses of

neighborhood socioeconomic context and health outcomes:

ARTICLE IN PRESS

http://localhost/var/www/apps/conversion/tmp/scratch_6/10.1016/j.socscimed.2003.08.004http://localhost/var/www/apps/conversion/tmp/scratch_6/10.1016/j.socscimed.2003.08.004http://localhost/var/www/apps/conversion/tmp/scratch_6/10.1016/j.socscimed.2003.08.004http://localhost/var/www/apps/conversion/tmp/scratch_6/10.1016/j.socscimed.2003.08.004


7/7

A critical review. Journal of Epidemiology and Community

Health, 55, 111122.

Sampson, R. J., & Morenoff, J. D. (2002). Assessing neighbor-

hood effects: Social processes and new directions in

research.Annual Review of Sociology, 28, 443478.

Snijders, T. A. B. (2001). Sampling. In A. H. Leyland, & H.

Goldstein (Eds.), Multilevel modelling of health statistics(pp. 159174). Chichester: Wiley.

Snijders, T. A. B., & Bosker, R. J. (1993). Standard errors and

sample sizes for two-level research. Journal of Educational

Statistics, 18, 237259.

Sorensen, G., Emmons, K., Hunt, M. K., & Johnston, D.

(1998). Implications of the results of community interven-

tion trials. Annual Review of Public Health, 19 , 379416.

Spencer, N. H. (2002). Combining modelling strategies to

analyse teaching styles data. Quality and Quantity, 36,

113127.

Stoker, L., & Bowers, J. (2002). Designing multi-level studies:

Sampling voters and electoral contexts. Electoral Studies,

21, 235267.

Subramanian, S. V., Jones, K., & Duncan, C. (2003). Multilevelmethods for public health research. In I. Kawachi, & L. F.

Berkman (Eds.), Neighborhoods and health (pp. 65111).

New York: Oxford University Press.

Subramanian, S. V., Kawachi, I., & Kennedy, B. P. (2001).

Does the state you live in make a difference? Multilevel

analysis of self-rated health in the US. Social Science &

Medicine, 53, 919.

ARTICLE IN PRESS

S.V. Subramanian / Social Science & Medicine 58 (2004) 19611967 1967

The Relevance of Multilevel Statistical Methods for Identifying

Documents

Transcript of The Relevance of Multilevel Statistical Methods for Identifying