When does Multilevel Modeling come in...

4/30/2012

1

1

When does Multilevel Modeling come in handy?

Data and research questions which benefit from MLM

Alan D. Mead (Scott B. Morris)

Illinois Institute of Technology

2

MLM: The tool that may be missing?

Practitioners and researchers may not even realize that MLM is missing from their methodological toolbox

MLM is a relatively new topic

The data can typically be analyzed by ignoring the hierarchical structure

BUT, this risks bad conclusions

Which might be worse than no analysis at all!

4/30/2012

2

3

One example

Assume customer service orientation (CSO) is, overall, weakly related to sales, but:

In stores with high volumes CSO is negatively related to sales, and

In stores with low volumes CSO is positively related to sales

It would be unfortunate to miss this critical distinction

You need to use MLM to find this distinction

4

Nested data

MLM methods are appropriate whenever the data have multiple “levels” so that “lower” level date are nested under a “higher” group that (may have) an influence

For example,

It is common to measure individual-level data (employee satisfaction, customer satisfaction, sick days, etc.) and unit-level data (productivity, profitability, etc.)

Leader and subordinate data

Longitudinal data (nested under person)

4/30/2012

3

5

Cross-level effects

In our CSO → Sales example, in stores with high (low) volumes CSO is negatively (positively) related to sales

This is a “cross-level” effect because we want to use a store-level variable as a moderator of an individual level relationship

HLM shines here

Other analyses may produce misleading results

6

RQ's well-suited for HLM

Do the groups in which individuals work affect their individual outcomes?

For example,

Is marketing more or less satisfied than the five other business units?

Are individual performance appraisals affected by the manager who performs the appraisal?

4/30/2012

4

7

RQ's well-suited for HLM (cont.)

What is the effect of individual differences across work groups?

For example, is conscientiousness related to organizational citizenship behaviors across workgroups that differ in cohesion (Kidwell, et al., 1997)

8


Are individual variables affected by group-level variables?

For example, are accidents less likely in groups with high/strong safety climate?

Safety climate is a group-level variable

HLM can estimate differences in individual variables using group-level predictors

4/30/2012

5

9


Is there an interaction between individual variables with group-level variables?

For example, we might suppose that conscientiousness is negatively related to accidents...

but in groups with strong safety climate, this relationship may be weaker because workers conform to the strong climate of safety

HLM allows group-level moderators

10


Am I justified in aggregating individual responses?

When no cross-level effects are detected within nested data, they may be collapsed

For example, HLM may demonstrate that the individual perceived LMX scores of direct reports of managers may be averaged within each manager without loss of data

(Alternatively, each employee has a unique perceived LMX score)

4/30/2012

6

11

HLM is NOT well-suited for this RQ

How do individual-level variables affect group-level variables?

For example, HLM is not well suited to addressing the question of how individual differences in moral reasoning affect team moral climate

The variable being predicted must be an individual-level variable

12

Nomenclature

Multilevel modeling goes by a number of aliases (e.g., hierarchical linear modeling, random coefficients modeling)

WABA is a related analysis focused on different research questions (about level of analysis)

Regardless of what it is called, MLM has been recognized as an important advance in the way organizational researchers conceptualize phenomena and analyze data

4/30/2012

7

13

Thanks!

For a copy of the paper, email:

[email protected]

Christopher J. L. Cunningham

The University of Tennessee at Chattanooga

4/30/2012

8

Key MLM Design Issues Multilevel theory

Multilevel sampling

Multilevel operationalization & measurement

Main learning objective: Be able to identify and list

important research design considerations that

incorporate ML perspectives

Origins of ML phenomena Theory: Reason/explanation for a relationship or effect

spanning lower and higher levels?

We need theory expansion and development that considers ML

phenomena

Reality: Work environments are more complex than we

recognize in most I-O research

Nearly every p-o phenomenon can be seen as ML in some way

People function within groups, groups within organizations, etc.

Knowledge of higher-level phenomena provides context in which

interpreting lower-level phenomena may make more sense

Multi-staged or multi-source data collections may also benefit

from a ML approach (see Snijders & Bosker, 1999)

4/30/2012

9

Key ML Theory Questions • At what level do the relationships you are studying

exist?

• Do the constructs you are measuring mean the same

thing at a lower level that they do at a higher level?

• Does your research address a phenomenon in its

entirety, as holistically as possible?

• If so, you probably need to use ML methods

• If not, and you are isolating a specific aspect of such a

phenomenon, uni-level methods may be appropriate

Common Levels of Analysis Emergent: phenomenon exists at higher level, but

can be indicated by information from lower level (cf., Glick, 1985)

Team or collective efficacy

Cross level/composition: phenomenon exists at both the higher and lower levels, and direction of influence operates both ways (cf., Mossholder & Bedeian, 1983)

Average individual satisfaction represents group morale

Individual-level: phenomenon exists only at the lower- or person-level

Personality performance

?

Xp Yp

Xg Yg

Xp Yp

Xg Yg

Xp Yp

4/30/2012

10

ML Sampling and Power See Scherbaum & Ferreter (2009) for helpful, practical

guidance ML power estimation requires optimization

and balancing of multiple factors

Are you studying the right sample(s) and units to test your

hypotheses/research questions?

Are they well-defined, representative, sufficiently variable,

etc.?

What parameters are you examining (simple fixed effects,

cross-level interactions)?

What estimation method will you use (RML/FML)?

ML Sampling and Power Do you have enough participants/groups/data points?

For aggregation and between-group studies: # of groups and #

of people/group matters

Depends on variability within and between groups

Generally increasing # of groups will improve power more than

increasing # of people within groups (but this = $)

With covariates, the optimal balance may be fewer groups with more

individuals in each group

For longitudinal, repeated measure, diary studies: # of

observations within-person is important

For cross-level interactions, see Mathieu et al. (in press) JAP

4/30/2012

11

ML Measurement Level of measurement depends on the ML theory you

are applying and the phenomenon you are studying

If emergent, then the higher-level phenomenon you are

studying is best indicated by the average of lower-level

data

If cross-level, then the person-level data and the group-

level averages are needed

If individual-level, then the person-level data is needed

Keep in mind, that not all higher-level measures are

aggregated lower-level data (e.g., group size)

Aggregation? When supported by theory, it may be necessary or desirable to indicate a

higher-level construct from data gathered at a lower-level of analysis

Aggregated lower-level data are typically viewed as more stable/reliable

snapshot than single measurement (e.g., average vs. single score)

Options include: summing, averaging, indices of variability/consensus

Facilitates identification of important higher-order constructs

Emergent constructs = Group-level construct > sum of each person-level

construct

Emergence can occur by:

Composition: Additive, averaging, or consensus approaches; group- and

person-level variables share meaning (isomorphism)

Compilation: Variability as index of dispersion within a group; group- and

person-level variables not identical

4/30/2012

12

Aggregation “Theory” Chan’s (1998) 5 forms of composition:

(1) *Direct consensus: depends on within-group agreement, group norms

Staff happiness as average of members’ perceptions of own happiness

(provided sufficient within-group agreement)

(2) *Referent-shift: referent of construct shifts from individual to group level

Collective efficacy as aggregate of members’ perceptions of group’s/team’s

efficacy (provided sufficient within-group agreement)

(3) Additive: aggregated phenomenon = sum/avg of individual level components

(after all, is it true that something like organizational climate doesn’t exist if

the perceptions of the individual members you collect data from don’t agree?)

(4) Dispersion: level of agreement as a construct of its own

(5) Process composition: describe function or structure of constructs across

multiple levels

Why Not Aggregate? Aggregation does not always make sense – Theory and data should

support

Why does it make sense to aggregate lower level information to indicate a

higher level construct?

Can you demonstrate adequate within-group agreement on a particular

construct? rwg

Are group means reliably differentiated? ICC(2), ICC(1)

Are individual scores influenced by group membership (i.e., non-

independent)? ICC(1)

Is there variation between higher-level groups? ICC(1)

Do not aggregate just because it is convenient/practical

Aggregation may “wash out” meaningful individual-level variance

4/30/2012

13

ML Design Recommendations Specify all design decisions clearly so that others can understand your

ML approach as it fits your hypotheses, sample, measures, and analyses

Early in designing, consider possible ML linkages among your variables

Seriously consider theory to gauge support for potential cross-level or

other ML relationships

Operationalize constructs clearly and consider the levels of analysis

that are supported by the theoretical background you have developed

Use lower-level information to make inferences about higher-level

constructs (van Mierlo et al., 2009)

Measure at lower and higher levels if you expect there to be cross-level

relationships to be tested

ML Design Recommendations Aim high (if needed)

If theory supports, plan to aggregate lower-level data by

ensuring that the questions asked of lower-level respondents

keep the higher-level target as the primary referent

(otherwise what would the aggregated score indicate?)

Ask questions that are descriptive and target well-defined

groups

Plan sampling to ensure adequate power for the effects you

are studying

Use multiple methods and sources (see Glick, 1985)

Best way to clarify phenomena by building convergent validity

4/30/2012

14

Learning Objective: Understand how to analyze most common multilevel models

Lisa M. Kath

San Diego State University

Multilevel Modeling Alan: why we might consider multilevel models

Chris: what to consider when designing studies

which means…

I get the fun part!

(analyses)

4/30/2012

15

Analyses Draw your model

Convert picture to equations

Special issues with centering

Software packages

Data preparation

Draw your model

Predictor 1: Safety Motivation

Predictor 2: Safety Policy

Moderator: Safety Climate

Outcome: Safety Behavior

L2: Group level

L1: Individual level

At lowest level

4/30/2012

16

Y = 0 + 1X1 + 2X2 + e

0 = γ00 + u0j

1 = γ10 + u1j

2 = γ20 + u2j

Gammas are the new betas!

Level 2 equations

It’s OK – it’s just REGRESSION

Level 1 equation

Single-level model





L2: Group level


4/30/2012

17

Single-level model

(Safety Beh.) = 0 + 1(Safety Mot.) + e

0 = γ00 + u0j

1 = γ10 + u1j Level 2 equations

Level 1 equation

Safety Motivation

Safety Behavior

Cross-level direct effects model





L2: Group level


4/30/2012

18

Cross-level direct effects model


0 = γ00 + γ01 (Safety Policy) + u0j

1 = γ10 + u1j

Safety Motivation

Safety Behavior

Safety Policy

Cross-level moderation model





L2: Group level


4/30/2012

19

REGULAR moderation model

(Safety Beh.) = 0 + 1(Safety Mot.) + 2(Safety Clim.) + 3(SM x SC) + e

Main effect terms

Interaction term

Safety Motivation

Safety Behavior

Safety Climate



0 = γ00 + γ01 (Safety Climate) + u0j


But wait, there’s more…

Safety Motivation

Safety Behavior

Safety Climate

4/30/2012

20

Centering Usually for Level 1 predictors

Analytically = subtraction

Single-level model: group-mean center

X-level direct effects: grand-mean center

X-level moderation:

group-mean center L1

grand-mean center L2

add in bonus terms

Safety Motivation

Safety Behavior

Safety Policy Safety Climate





Safety Motivation

Safety Behavior

Safety Climate

4/30/2012

21



0 = γ00 + γ01 (Safety Climate) + γ02 (Safety Mot.) +

γ03 (SM x SC) + u0j


Enders & Tofighi (2007) – Psych. Methods

Safety Motivation

Safety Behavior

Safety Climate

Software packages HLM 6 ($495, $75/6 months, or free student version)

Mplus ($745 academic, $895 other, $240 student)

R (free)

SAS Proc Mixed

SPSS Mixed

Stata .xtmixed

4/30/2012

22

Data preparation Level 1 dataset – each row is an individual

Level 2 dataset – each row is a group

Linking Level 2 ID number

Sort by ID number

Check missing data handling with your software

Multilevel Modeling Summary Just enough information

Use powers for good not evil

Read up and get it straight:

In your head: Level of Theory

In your items: Level of Measurement

In your equations: Level of Analysis

In your specialty area: check for norms

4/30/2012

23

For copies of slides and full citations

please visit:

http://www.utc.edu/faculty/chris-cunningham

“Recent Conference Presentations”

When does Multilevel Modeling come in...

Documents

Transcript of When does Multilevel Modeling come in...