joint prevention seminarMultinomial logistic regression with fixed effects for school Differences...

25
Donna L. Coffman Joint Prevention Methodology Seminar

Transcript of joint prevention seminarMultinomial logistic regression with fixed effects for school Differences...

Page 1: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Donna L. CoffmanJoint Prevention Methodology Seminar

Page 2: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

The purpose of this talk is to illustrate how to obtain propensity scores in multilevel data and use these to strengthen causal inferences about mediation.

Page 3: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Many prevention programs randomly assigned at school level and hypothesized to affect mediators at individual level.

Although intervention is randomly assigned, the mediator is not.

Need to take into account confounders of the mediator and outcome.◦ e.g., poverty, individual’s prior delinquency

Page 4: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

One solution is to use propensity scores, which is based on potential outcomes framework.

One assumption of this framework is that there is no interference, such that one individual’s assignment to treatment level does not influence another individual’s outcome.

Page 5: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

No-interference is different from a nested data structure.

Nested data does not necessarily mean that we need propensity scores at both the school and individual level.

Our motivating problem is what is sometimes referred to as 2-1-1 or lower level mediation.

Page 6: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Three-level treatment: control, usual intervention, intervention adapted for rural◦ Cluster-randomized at the school level

Outcome- Alcohol use:◦ Binary because highly skewed. ◦ 0 = no use and 1 = any use

Page 7: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Wave 2 refusal confidence about drinking Wave 2 peer alcohol use norms Wave 2 positive expectancies about drinking Scores of 1 are good - indicate greater

refusal confidence, less positive expectancies, and lower substance use norms.

Page 8: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Demographics: age, gender, race/ethnicity Lifetime and wave 1 measures of recent

substance use (including alcohol)◦ Also age at first use

Wave 1 measures of mediators Wave 1 measures of delinquency◦ Stealing, destroying property, school expulsion

Coping, anxiety, rebelliousness, family relationships & rules, parental knowledge & monitoring

Page 9: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Imputation model included a lot of variables due to all the confounders in the propensity score model.

Many of the variables were not normally distributed so joint model (e.g., NORM) would not work.

We used chained equations. Even then, this step was computationally intensive and we decided to impute only one data set for illustration purposes.

Page 10: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Propensity score model for treatment◦ Multinomial logistic regression with fixed effects for

school◦ Differences are not that large initially but we

wanted to reduce variance. Computed stabilized weights◦ Predicted probability from intercept-only model

divided by predicted probability from model of treatment regressed on all confounders.◦ Helps to reduce variability of weights and extreme

weights.

Page 11: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized
Page 12: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized
Page 13: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Propensity score model for each mediator◦ Logistic regression of each mediator on all

confounders and fixed effects for school and treatment level

Computed weights◦ Numerator of weights was predicted probability

from a logistic regression of each mediator on the treatment level

Page 14: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized
Page 15: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized
Page 16: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Assess effect of intervention on mediators◦ Logistic regression for each mediator◦ Use only weights for intervention

Page 17: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Assess effect of intervention and mediators on alcohol use.◦ Logistic regression models for each mediator◦ Using product of intervention and mediator

weights.

Page 18: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Intervention did not have statistically significant effect on refusal confidence or positive expectancies.

Mediators did have statistically significant effect on alcohol use.

Intervention did not have a statistically significant direct effect on alcohol use.

Intervention did not have a statistically significant total effect on alcohol use.

Page 19: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Refusal Confidence◦ Not statistically significant when either weighted or

unweighted Peer Alcohol Use Norms◦ Significant effect: For the usual DRS intervention there is

a 19% reduction in lower norms (-.214, odds ratio =.807) when unweighted and a 20% reduction in lower norms (-.228, odds ratio = .796) when weighted.

◦ p = .025 and .035 for unweighted and weighted Positive Expectancies◦ Not statistically significant when either weighted or

unweighted

Page 20: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Refusal Confidence◦ Statistically significant effect: -.363, p = .038◦ When refusal confidence is high, alcohol use decreases –

there is a 30% reduction in alcohol use. Odds Ratio = .696

Peer Alcohol Use Norms◦ Statistically significant effect: -1.10, p < .001◦ When norms are low, alcohol use decreases – there is a

67% reduction in alcohol use. Odds Ratio = .333 Positive Expectancies◦ Statistically significant effect: -1.07, p < .001◦ When positive expectancies are low, alcohol use

decreases – there is a 66% reduction in alcohol use. Odd Ratio = .343

Page 21: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Effect of treatment on alcohol use using logistic regression without the mediators in the model.◦ No significant effect either with or without the

weights.

Page 22: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

We recognize that the mediators may not be independent. We performed an analysis that took this into consideration but results were not different.

We did not multiply coefficients together for the effect of the intervention on the mediators and the effect of mediators on alcohol use because with binary mediators and outcomes, this value is not interpretable as an indirect effect although the indirect effect can be tested for significance (Coffman & Zhong, 2012).

Page 23: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

It is important to proceed with mediation analysis even in the absence of a total effect.

It is important to look at the effects separately.◦ We learned that the mediators do indeed have

strong effects on alcohol use, even when controlling for many confounders.◦ But the intervention did not have statistically

significant effects on the mediators, perhaps due to implementation issues.

Page 24: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Mediation analyses should consider that individuals and/or groups are not randomly assigned to levels of the mediator(s) and there may be confounders of the mediator and outcome even if the intervention is randomly assigned.

This is an active area of research and future work when examine under what conditions propensity score models should include random effects, when propensity scores are needed at each level of nested data, and how to handle multiple mediators simultaneously.

Page 25: joint prevention seminarMultinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized

Bethany Bray Wanghuan Chu Michael Hecht and John Graham NIH NIDA P50 DA10075