Versions of Treatment - Princeton University...Android push received No push received Takes...

101
Versions of Treatment Formalizing the Problem When It Matters What To Do Versions of Treatment A causal inference debate that sociologists have ignored Ian Lundberg Princeton University Sociology Statistics Reading Group 20 February 2019

Transcript of Versions of Treatment - Princeton University...Android push received No push received Takes...

Page 1: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA causal inference debate that

sociologists have ignored

Ian LundbergPrinceton University

Sociology Statistics Reading Group20 February 2019

Page 2: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Introductory note for those finding these slides online

These slides were prepared for the Sociology Statistics ReadingGroup at Princeton. Everyone read the following paper in advance:

Hernan, Miguel A. 2016. Does water kill? A call for lesscasual causal inferences. Annals of Epidemiology,26(10):674–680. [link]

At times, these slides intentionally emphasize alternative positionsto those presented by Hernan (2016), such as the possibility thatconsistency is not an assumption but is rather a consequence ofthe assumptions embedded in a causal DAG (Pearl, 2010). Iemphasize this alternative view not because my personal position isstrongly one way or the other, but because it will promote betterdiscussion among a group that read the former but not the latter.See references at the end for further reading.

Page 3: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Hernan (2016): Does Water Kill?

London cholera epidemic, 1854.John Snow deduced that the water was the cause of death.

Source: Wikimedia Commons

Page 4: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Hernan (2016): Does Water Kill?

Does drinking water kill?

The definition of the causal effect is unclear without details.

Recommendation: Specify versions“until no meaningful vagueness remains,”

(Hernan, 2016)

Page 5: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Hernan (2016): Does Water Kill?

Does drinking fresh water kill?

The definition of the causal effect is unclear without details.

Recommendation: Specify versions“until no meaningful vagueness remains,”

(Hernan, 2016)

Page 6: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Hernan (2016): Does Water Kill?

Does drinking a swig of fresh water kill?

The definition of the causal effect is unclear without details.

Recommendation: Specify versions“until no meaningful vagueness remains,”

(Hernan, 2016)

Page 7: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Hernan (2016): Does Water Kill?

Does drinking a swig of fresh water from the Broad Streetpump kill?

The definition of the causal effect is unclear without details.

Recommendation: Specify versions“until no meaningful vagueness remains,”

(Hernan, 2016)

Page 8: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Hernan (2016): Does Water Kill?

Does drinking a swig of fresh water from the Broad Street pumpbetween August 31 and September 10 kill?

The definition of the causal effect is unclear without details.

Recommendation: Specify versions“until no meaningful vagueness remains,”

(Hernan, 2016)

Page 9: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Hernan (2016): Does Water Kill?

Does drinking a swig of fresh water from the Broad Street pumpbetween August 31 and September 10 kill

compared with drinking all your water from other pumps?

The definition of the causal effect is unclear without details.

Recommendation: Specify versions“until no meaningful vagueness remains,”

(Hernan, 2016)

Page 10: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Hernan (2016): Does Water Kill?

Does drinking a swig of fresh water from the Broad Street pumpbetween August 31 and September 10 and not initiating arehydration treatment if diarrhea starts kill

compared with drinking all your water from other pumps?

The definition of the causal effect is unclear without details.

Recommendation: Specify versions“until no meaningful vagueness remains,”

(Hernan, 2016)

Page 11: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Hernan (2016): Does Water Kill?

Does drinking a swig of fresh water from the Broad Street pumpbetween August 31 and September 10 and not initiating arehydration treatment if diarrhea starts kill

compared with drinking all your water from other pumps?

The definition of the causal effect is unclear without details.

Recommendation: Specify versions“until no meaningful vagueness remains,”

(Hernan, 2016)

Page 12: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Hernan (2016): Does Water Kill?

Does drinking a swig of fresh water from the Broad Street pumpbetween August 31 and September 10 and not initiating arehydration treatment if diarrhea starts kill

compared with drinking all your water from other pumps?

The definition of the causal effect is unclear without details.

Recommendation: Specify versions“until no meaningful vagueness remains,”

(Hernan, 2016)

Page 13: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Hernan (2016): Does Water Kill?

Does drinking a swig of fresh water from the Broad Street pumpbetween August 31 and September 10 and not initiating arehydration treatment if diarrhea starts kill

compared with drinking all your water from other pumps?

The definition of the causal effect is unclear without details.

Recommendation: Specify versions“until no meaningful vagueness remains,”

(Hernan, 2016)

Page 14: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

But some vagueness is

unavoidable

in both experimental and observationalsocial science.

Page 15: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

A D

U

Y

Targetpopulation

Pushnotification

sent

iPhone push receivedAndroid push received

No push received

Takes10-minute

walk

Page 16: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Col

lap

sed

byre

sear

cher

Col

lap

sed

byre

spo

nd

ent

Continuous treatment Categorical treatment

OverworkResearcher collapses continuous D

D Y

A = C(D) = I(D > 50)

Employmenthours

Hourlywage

OccupationsResearcher collapses categorical D

D Y

Class scheme: A = C(D)

Occupation Status

Self-rated healthRespondent collapses continuous D

D Y

5-point scale: A = C(D)

Health Lifespan

Returns to collegeRespondent collapses categorical D

D Y

Completed college: A = C(D)

College degree(institutionand major)

Earnings

Page 17: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Col

lap

sed

byre

sear

cher

Col

lap

sed

byre

spo

nd

ent

Continuous treatment Categorical treatment

OverworkResearcher collapses continuous D

D Y

A = C(D) = I(D > 50)

Employmenthours

Hourlywage

OccupationsResearcher collapses categorical D

D Y

Class scheme: A = C(D)

Occupation Status

Self-rated healthRespondent collapses continuous D

D Y

5-point scale: A = C(D)

Health Lifespan

Returns to collegeRespondent collapses categorical D

D Y

Completed college: A = C(D)

College degree(institutionand major)

Earnings

Page 18: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Col

lap

sed

byre

sear

cher

Col

lap

sed

byre

spo

nd

ent

Continuous treatment Categorical treatment

OverworkResearcher collapses continuous D

D Y

A = C(D) = I(D > 50)

Employmenthours

Hourlywage

OccupationsResearcher collapses categorical D

D Y

Class scheme: A = C(D)

Occupation Status

Self-rated healthRespondent collapses continuous D

D Y

5-point scale: A = C(D)

Health Lifespan

Returns to collegeRespondent collapses categorical D

D Y

Completed college: A = C(D)

College degree(institutionand major)

Earnings

Page 19: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Col

lap

sed

byre

sear

cher

Col

lap

sed

byre

spo

nd

ent

Continuous treatment Categorical treatment

OverworkResearcher collapses continuous D

D Y

A = C(D) = I(D > 50)

Employmenthours

Hourlywage

OccupationsResearcher collapses categorical D

D Y

Class scheme: A = C(D)

Occupation Status

Self-rated healthRespondent collapses continuous D

D Y

5-point scale: A = C(D)

Health Lifespan

Returns to collegeRespondent collapses categorical D

D Y

Completed college: A = C(D)

College degree(institutionand major)

Earnings

Page 20: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 21: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 22: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 23: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Causal effect = Yi(a′)− Yi(a)

Potential outcomes

{Yi(a)}︸ ︷︷ ︸Potential outcomes:

Deterministic consequence of a

Y Obsevedi = Yi(Ai)︸ ︷︷ ︸Observed outcome:

Random because Ai is random

Imbens and Rubin (2015, p. 10):

. . . for each unit, there are no different forms or versionsof each treatment level which lead to different potentialoutcomes.

Page 24: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

A D Y

Push notification iPhone push receivedAndroid push received

No push received

Walks10 minutes

Yi (a) is deterministic under either:

Deterministic detailed treatment assignment D given A

P(Di = d | Ai = a) =

{1 for one value of d

0 for all other values of d∀a

Treatment variation irrelevance (adapted from VanderWeele 2009)

Yi (d) = Yi (d′) ∀ {d , d ′} such that

P(Di = d | Ai = a) > 0 and P(Di = d ′ | Ai = a) > 0

Page 25: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 26: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 27: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Stochastic counterfactuals1 allow a more plausible assumptionof treatment variation irrelevance.

Under fixed counterfactuals

Treatment-variation irrelevance:

Yi (a, da) = Yi (a, d′a) ∀ {da, d ′a} ∈ Da

Thus can define Yi (a) ≡ Yi (a, da) for any da.

Consistency:

If Ai = a, ∃ da ∈ Da such that Y Observedi = Yi (a, da)

VanderWeele 2009, Imbens and Rubin 2015 p. 12

Page 28: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Stochastic counterfactuals1 allow a more plausible assumptionof treatment variation irrelevance.

Under stochastic counterfactuals

Treatment-variation irrelevance:

Yi (a, da)D∼Yi (a, d

′a) ∀ {da, d ′a} ∈ Da

Thus can define Yi (a) ∼Yi (a, da) for any da.

Consistency:

If Ai = a, ∃ da ∈ Da such that Y Observedi = Yi (a, da)

VanderWeele 2009, Imbens and Rubin 2015 p. 12

Page 29: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 30: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 31: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

In causal graphs, the absence of hidden versions is atheorem rather than an assumption (Pearl, 2010).

Treatment effects are defined by the DAG.

A correct DAG implies a well-defined effect.

Page 32: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

In causal graphs, the absence of hidden versions is atheorem rather than an assumption (Pearl, 2010).

Treatment effects are defined by the DAG.

A correct DAG implies a well-defined effect.

Page 33: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

In causal graphs, the absence of hidden versions is atheorem rather than an assumption (Pearl, 2010).

Treatment effects are defined by the DAG.

A correct DAG implies a well-defined effect.

Page 34: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

DeathSwallowed from theBroad Street pump

Lives in London Date

Went toBroad Street pump

Pumped handle

Water exited at velocity v

Raised cup to mouth

?

Rehydrationtherapy

E(

Death | do(Swallowed from Broad Street pump),Living in London on August 31 – September 10

)− E

(Death | do(Did not swallow from Broad Street pump),

Living in London on August 31 – September 10

)Things are vague only if the graph is insufficiently precise(and thus wrong).

Page 35: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

DeathSwallowed from theBroad Street pump

Lives in London Date

Went toBroad Street pump

Pumped handle

Water exited at velocity v

Raised cup to mouth

?

Rehydrationtherapy

E(

Death | do(Swallowed from Broad Street pump),Living in London on August 31 – September 10

)− E

(Death | do(Did not swallow from Broad Street pump),

Living in London on August 31 – September 10

)Things are vague only if the graph is insufficiently precise(and thus wrong).

Page 36: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

DeathSwallowed from theBroad Street pump

Lives in London Date

Went toBroad Street pump

Pumped handle

Water exited at velocity v

Raised cup to mouth

?

Rehydrationtherapy

E(

Death | do(Swallowed from Broad Street pump),Living in London on August 31 – September 10

)− E

(Death | do(Did not swallow from Broad Street pump),

Living in London on August 31 – September 10

)Things are vague only if the graph is insufficiently precise(and thus wrong).

Page 37: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

DeathSwallowed from theBroad Street pump

Lives in London Date

Went toBroad Street pump

Pumped handle

Water exited at velocity v

Raised cup to mouth

?

Rehydrationtherapy

E(

Death | do(Swallowed from Broad Street pump),Living in London on August 31 – September 10

)− E

(Death | do(Did not swallow from Broad Street pump),

Living in London on August 31 – September 10

)Things are vague only if the graph is insufficiently precise(and thus wrong).

Page 38: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

DeathSwallowed from theBroad Street pump

Lives in London Date

Went toBroad Street pump

Pumped handle

Water exited at velocity v

Raised cup to mouth

?

Rehydrationtherapy

E(

Death | do(Swallowed from Broad Street pump),Living in London on August 31 – September 10

)− E

(Death | do(Did not swallow from Broad Street pump),

Living in London on August 31 – September 10

)Things are vague only if the graph is insufficiently precise(and thus wrong).

Page 39: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

DeathSwallowed from theBroad Street pump

Lives in London Date

Went toBroad Street pump

Pumped handle

Water exited at velocity v

Raised cup to mouth

?

Rehydrationtherapy

E(

Death | do(Swallowed from Broad Street pump),Living in London on August 31 – September 10

)− E

(Death | do(Did not swallow from Broad Street pump),

Living in London on August 31 – September 10

)Things are vague only if the graph is insufficiently precise(and thus wrong).

Page 40: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

DeathSwallowed from theBroad Street pump

Lives in London Date

Went toBroad Street pump

Pumped handle

Water exited at velocity v

Raised cup to mouth

?

Rehydrationtherapy

E(

Death | do(Swallowed from Broad Street pump),Living in London on August 31 – September 10

)− E

(Death | do(Did not swallow from Broad Street pump),

Living in London on August 31 – September 10

)Things are vague only if the graph is insufficiently precise(and thus wrong).

Page 41: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

DeathSwallowed from theBroad Street pump

Lives in London Date

Went toBroad Street pump

Pumped handle

Water exited at velocity v

Raised cup to mouth

?

Rehydrationtherapy

E(

Death | do(Swallowed from Broad Street pump),Living in London on August 31 – September 10

)− E

(Death | do(Did not swallow from Broad Street pump),

Living in London on August 31 – September 10

)

Things are vague only if the graph is insufficiently precise(and thus wrong).

Page 42: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

DeathSwallowed from theBroad Street pump

Lives in London Date

Went toBroad Street pump

Pumped handle

Water exited at velocity v

Raised cup to mouth

?

Rehydrationtherapy

E(

Death | do(Swallowed from Broad Street pump),Living in London on August 31 – September 10

)− E

(Death | do(Did not swallow from Broad Street pump),

Living in London on August 31 – September 10

)

Things are vague only if the graph is insufficiently precise(and thus wrong).

Page 43: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Consistency: Y Observedi = Yi (ai ). Is this an assumption?

Hernan (2016): Potential death Y under weight a depends onwhether weight is set by smoking or by moderate exercise.

But we could just label these as confounding variables.Not clear that consistency is an assumption.

Weight Death

Smoking

Moderate exercise

Page 44: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Consistency: Y Observedi = Yi (ai ). Is this an assumption?

Hernan (2016): Potential death Y under weight a depends onwhether weight is set by smoking or by moderate exercise.

But we could just label these as confounding variables.Not clear that consistency is an assumption.

Weight Death

Smoking

Moderate exercise

Page 45: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Consistency: Y Observedi = Yi (ai ). Is this an assumption?

Hernan (2016): Potential death Y under weight a depends onwhether weight is set by smoking or by moderate exercise.

But we could just label these as confounding variables.Not clear that consistency is an assumption.

Weight Death

Smoking

Moderate exercise

Page 46: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 47: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 48: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 49: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Experiments identify causal effects with minimal assumptions,

but they often seek to generalize to a target population.

Versions of treatment make generalization difficult.(Hernan and VanderWeele, 2011)

Page 50: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Experiments identify causal effects with minimal assumptions,

but they often seek to generalize to a target population.

Versions of treatment make generalization difficult.(Hernan and VanderWeele, 2011)

Page 51: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

If there is contextual variation in A→ Y , then the effect maynot generalize to new contexts.

If this arises from variation in A→ D, then measuring D maypromote transportability of inference.

A Y

Context

Page 52: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

If there is contextual variation in A→ Y , then the effect maynot generalize to new contexts.

If this arises from variation in A→ D, then measuring D maypromote transportability of inference.

A Y

Context

D

Page 53: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

If there is contextual variation in A→ Y , then the effect maynot generalize to new contexts.

If this arises from variation in A→ D, then measuring D maypromote transportability of inference.

A Y

A→ Y may differ across hospitals

Surgery Survival

Hospital

Page 54: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

If there is contextual variation in A→ Y , then the effect maynot generalize to new contexts.

If this arises from variation in A→ D, then measuring D maypromote transportability of inference.

A Y

A→ Y may differ across hospitals

Surgery Survival

Hospital

Surgeon

D

Page 55: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

If there is contextual variation in A→ Y , then the effect maynot generalize to new contexts.

If this arises from variation in A→ D, then measuring D maypromote transportability of inference.

A Y

A→ Y may differ by time of day

Randomized torestaurant adon Facebook

Clickson ad

Time of day

Page 56: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

If there is contextual variation in A→ Y , then the effect maynot generalize to new contexts.

If this arises from variation in A→ D, then measuring D maypromote transportability of inference.

A Y

A→ Y may differ by time of day

Randomized torestaurant adon Facebook

Clickson ad

Time of day

Ad appearsbelow postsabout dinner

D

Page 57: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

If there is contextual variation in A→ Y , then the effect maynot generalize to new contexts.

If this arises from variation in A→ D, then measuring D maypromote transportability of inference.

A Y

A→ Y may differ by rates of compliance

Givenaspirin

Headachegone

Complier

Page 58: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

If there is contextual variation in A→ Y , then the effect maynot generalize to new contexts.

If this arises from variation in A→ D, then measuring D maypromote transportability of inference.

A Y

A→ Y may differ by rates of compliance

Givenaspirin

Headachegone

Complier

TakesAspirin

D

Page 59: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Generalizing experimental evidence

Generalization impossible

A

U

Y

Targetpopulation

Pushnotification

sent

Health

Generalization possibleby randomizing D

A D

U

Y

Targetpopulation

Pushnotification

sent

iPhone pushAndroid push

No push

Health

Generalization impossible

A D

X U

Y

Targetpopulation

Pushnotification

sent

iPhone pushAndroid push

No push

Health

Generalization possibleby randomizing D and observing X

A D

U X

Y

Targetpopulation

Pushnotification

sent

iPhone pushAndroid push

No push

Health

Page 60: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 61: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 62: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Col

lap

sed

byre

sear

cher

Col

lap

sed

byre

spo

nd

ent

Continuous treatment Categorical treatment

OverworkResearcher collapses continuous D

D Y

A = C(D) = I(D > 50)

Employmenthours

Hourlywage

OccupationsResearcher collapses categorical D

D Y

Class scheme: A = C(D)

Occupation Status

Self-rated healthRespondent collapses continuous D

D Y

5-point scale: A = C(D)

Health Lifespan

Returns to collegeRespondent collapses categorical D

D Y

Completed college: A = C(D)

College degree(institutionand major)

Earnings

Page 63: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 64: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

D Y

Because the collapsed C(D) is not in the causal graph, the effectof a collapsed treatment is undefined.

We might examine:(VanderWeele and Hernan (2013, Prop. 8), though notation differs.)

E(Y | C(D) = c) =∑

d∈C−1(c)

E(Y | do(D = d)

)P(D = d | D ∈ C−1(c)

)One causal contrast

E(Y | C(D) = c ′)− E(Y | C(D) = c)

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Page 65: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

D Y

Because the collapsed C(D) is not in the causal graph, the effectof a collapsed treatment is undefined. We might examine:(VanderWeele and Hernan (2013, Prop. 8), though notation differs.)

E(Y | C(D) = c) =∑

d∈C−1(c)

E(Y | do(D = d)

)P(D = d | D ∈ C−1(c)

)

One causal contrast

E(Y | C(D) = c ′)− E(Y | C(D) = c)

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Page 66: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

D Y

Because the collapsed C(D) is not in the causal graph, the effectof a collapsed treatment is undefined. We might examine:(VanderWeele and Hernan (2013, Prop. 8), though notation differs.)

E(Y | C(D) = c) =∑

d∈C−1(c)

E(Y | do(D = d)

)P(D = d | D ∈ C−1(c)

)One causal contrast

E(Y | C(D) = c ′)− E(Y | C(D) = c)

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Page 67: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

D Y

Because the collapsed C(D) is not in the causal graph, the effectof a collapsed treatment is undefined. We might examine:(VanderWeele and Hernan (2013, Prop. 8), though notation differs.)

E(Y | C(D) = c) =∑

d∈C−1(c)

E(Y | do(D = d)

)P(D = d | D ∈ C−1(c)

)One causal contrast

E(Y | C(D) = c ′)− E(Y | C(D) = c)

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Page 68: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

D Y

Because the collapsed C(D) is not in the causal graph, the effectof a collapsed treatment is undefined. We might examine:(VanderWeele and Hernan (2013, Prop. 8), though notation differs.)

E(Y | C(D) = c) =∑

d∈C−1(c)

E(Y | do(D = d)

)P(D = d | D ∈ C−1(c)

)One causal contrast

E(Y | C(D) = c ′)− E(Y | C(D) = c)

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Page 69: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

D Y

Because the collapsed C(D) is not in the causal graph, the effectof a collapsed treatment is undefined. We might examine:(VanderWeele and Hernan (2013, Prop. 8), though notation differs.)

E(Y | C(D) = c) =∑

d∈C−1(c)

E(Y | do(D = d)

)P(D = d | D ∈ C−1(c)

)One causal contrast

E(Y | C(D) = c ′)− E(Y | C(D) = c)

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw

Random drawDifference

Average over many reps

Page 70: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

D Y

Because the collapsed C(D) is not in the causal graph, the effectof a collapsed treatment is undefined. We might examine:(VanderWeele and Hernan (2013, Prop. 8), though notation differs.)

E(Y | C(D) = c) =∑

d∈C−1(c)

E(Y | do(D = d)

)P(D = d | D ∈ C−1(c)

)One causal contrast

E(Y | C(D) = c ′)− E(Y | C(D) = c)

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random draw

Difference

Average over many reps

Page 71: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

D Y

Because the collapsed C(D) is not in the causal graph, the effectof a collapsed treatment is undefined. We might examine:(VanderWeele and Hernan (2013, Prop. 8), though notation differs.)

E(Y | C(D) = c) =∑

d∈C−1(c)

E(Y | do(D = d)

)P(D = d | D ∈ C−1(c)

)One causal contrast

E(Y | C(D) = c ′)− E(Y | C(D) = c)

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Page 72: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

D Y

Because the collapsed C(D) is not in the causal graph, the effectof a collapsed treatment is undefined. We might examine:(VanderWeele and Hernan (2013, Prop. 8), though notation differs.)

E(Y | C(D) = c) =∑

d∈C−1(c)

E(Y | do(D = d)

)P(D = d | D ∈ C−1(c)

)One causal contrast

E(Y | C(D) = c ′)− E(Y | C(D) = c)

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Page 73: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

With covariates. (equivalent to

VanderWeele and Hernan 2013, Prop. 8)Xj D Y

E(Y | C(D) = c , ~X = ~x) =∑d∈C−1(c)

E(Y | do(D = d), ~X = ~x

)P(D = d | D ∈ C−1(c), ~X = ~x

)One causal contrast

∑~x P(~X = ~x)

(E(Y | C(D) = c ′, ~X = ~x)− E(Y | C(D) = c, ~X = ~x)

)

among ~X = ~x

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Average over ~X

Page 74: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

With covariates. (equivalent to

VanderWeele and Hernan 2013, Prop. 8)Xj D Y

E(Y | C(D) = c , ~X = ~x) =∑d∈C−1(c)

E(Y | do(D = d), ~X = ~x

)P(D = d | D ∈ C−1(c), ~X = ~x

)One causal contrast∑

~x P(~X = ~x)

(E(Y | C(D) = c ′, ~X = ~x)− E(Y | C(D) = c, ~X = ~x)

)

among ~X = ~x

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Average over ~X

Page 75: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

With covariates. (equivalent to

VanderWeele and Hernan 2013, Prop. 8)Xj D Y

E(Y | C(D) = c , ~X = ~x) =∑d∈C−1(c)

E(Y | do(D = d), ~X = ~x

)P(D = d | D ∈ C−1(c), ~X = ~x

)One causal contrast∑

~x P(~X = ~x)

(E(Y | C(D) = c ′, ~X = ~x)− E(Y | C(D) = c, ~X = ~x)

)

among ~X = ~x

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Average over ~X

Page 76: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

With covariates. (equivalent to

VanderWeele and Hernan 2013, Prop. 8)Xj D Y

E(Y | C(D) = c , ~X = ~x) =∑d∈C−1(c)

E(Y | do(D = d), ~X = ~x

)P(D = d | D ∈ C−1(c), ~X = ~x

)One causal contrast∑

~x P(~X = ~x)

(E(Y | C(D) = c ′, ~X = ~x)− E(Y | C(D) = c, ~X = ~x)

)

among ~X = ~x

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Average over ~X

Page 77: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

With covariates. (equivalent to

VanderWeele and Hernan 2013, Prop. 8)Xj D Y

E(Y | C(D) = c , ~X = ~x) =∑d∈C−1(c)

E(Y | do(D = d), ~X = ~x

)P(D = d | D ∈ C−1(c), ~X = ~x

)One causal contrast∑

~x P(~X = ~x)

(E(Y | C(D) = c ′, ~X = ~x)− E(Y | C(D) = c, ~X = ~x)

)

among ~X = ~x

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Average over ~X

Page 78: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

With covariates. (equivalent to

VanderWeele and Hernan 2013, Prop. 8)Xj D Y

E(Y | C(D) = c , ~X = ~x) =∑d∈C−1(c)

E(Y | do(D = d), ~X = ~x

)P(D = d | D ∈ C−1(c), ~X = ~x

)One causal contrast∑

~x P(~X = ~x)

(E(Y | C(D) = c ′, ~X = ~x)− E(Y | C(D) = c, ~X = ~x)

)

among ~X = ~x

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw

Random drawDifference

Average over many reps

Average over ~X

Page 79: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

With covariates. (equivalent to

VanderWeele and Hernan 2013, Prop. 8)Xj D Y

E(Y | C(D) = c , ~X = ~x) =∑d∈C−1(c)

E(Y | do(D = d), ~X = ~x

)P(D = d | D ∈ C−1(c), ~X = ~x

)One causal contrast∑

~x P(~X = ~x)

(E(Y | C(D) = c ′, ~X = ~x)− E(Y | C(D) = c, ~X = ~x)

)

among ~X = ~x

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random draw

Difference

Average over many reps

Average over ~X

Page 80: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

With covariates. (equivalent to

VanderWeele and Hernan 2013, Prop. 8)Xj D Y

E(Y | C(D) = c , ~X = ~x) =∑d∈C−1(c)

E(Y | do(D = d), ~X = ~x

)P(D = d | D ∈ C−1(c), ~X = ~x

)One causal contrast∑

~x P(~X = ~x)

(E(Y | C(D) = c ′, ~X = ~x)− E(Y | C(D) = c, ~X = ~x)

)

among ~X = ~x

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Average over ~X

Page 81: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

With covariates. (equivalent to

VanderWeele and Hernan 2013, Prop. 8)Xj D Y

E(Y | C(D) = c , ~X = ~x) =∑d∈C−1(c)

E(Y | do(D = d), ~X = ~x

)P(D = d | D ∈ C−1(c), ~X = ~x

)One causal contrast∑

~x P(~X = ~x)

(E(Y | C(D) = c ′, ~X = ~x)− E(Y | C(D) = c, ~X = ~x)

)

among ~X = ~x

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Average over ~X

Page 82: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

With covariates. (equivalent to

VanderWeele and Hernan 2013, Prop. 8)Xj D Y

E(Y | C(D) = c , ~X = ~x) =∑d∈C−1(c)

E(Y | do(D = d), ~X = ~x

)P(D = d | D ∈ C−1(c), ~X = ~x

)One causal contrast∑

~x P(~X = ~x)

(E(Y | C(D) = c ′, ~X = ~x)− E(Y | C(D) = c, ~X = ~x)

)

among ~X = ~x

Observed detailed treaments dmapping to collapsed treatment c ′

Observed detailed treaments dmapping to collapsed treatment c

Random draw Random drawDifference

Average over many reps

Average over ~X

Page 83: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 84: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 85: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

What is the effect of overwork (> 50 hours) on thehourly wage of those working at least 40 hours per week?

Suppose gender is the only source of confounding.

Employment hours

Overwork = C(Employment hours) = I(Hours > 50)

Hourly wageGender

E(Wage | do(Overwork),Man

)=∑

h>50

()

E(Wage | do(Overwork),Woman

)=∑

h>50

()Heterogeneous treatment effectsEffects of heterogeneous treatments

Page 86: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

What is the effect of overwork (> 50 hours) on thehourly wage of those working at least 40 hours per week?

Suppose gender is the only source of confounding.

Employment hours

Overwork = C(Employment hours) = I(Hours > 50)

Hourly wageGender

E(Wage | do(Overwork),Man

)=∑

h>50

()

E(Wage | do(Overwork),Woman

)=∑

h>50

()Heterogeneous treatment effectsEffects of heterogeneous treatments

Page 87: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

What is the effect of overwork (> 50 hours) on thehourly wage of those working at least 40 hours per week?

Suppose gender is the only source of confounding.

Employment hours

Overwork = C(Employment hours) = I(Hours > 50)

Hourly wageGender

E(Wage | do(Overwork),Man

)=∑

h>50

(E(Wage|do(Hours=h),Man

)×P(Hours=h|Hours>50,Man

))

E(Wage | do(Overwork),Woman

)=∑

h>50

(E(Wage|do(Hours=h),Woman

)×P(Hours=h|Hours>50,Woman

))Heterogeneous treatment effectsEffects of heterogeneous treatments

Page 88: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

What is the effect of overwork (> 50 hours) on thehourly wage of those working at least 40 hours per week?

Suppose gender is the only source of confounding.

Employment hours

Overwork = C(Employment hours) = I(Hours > 50)

Hourly wageGender

E(Wage | do(Overwork),Man

)=∑

h>50

(E(Wage|do(Hours=h),Man

)×P(Hours=h|Hours>50,Man

))

E(Wage | do(Overwork),Woman

)=∑

h>50

(E(Wage|do(Hours=h),Woman

)×P(Hours=h|Hours>50,Woman

))

Heterogeneous treatment effectsEffects of heterogeneous treatments

Page 89: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

What is the effect of overwork (> 50 hours) on thehourly wage of those working at least 40 hours per week?

Suppose gender is the only source of confounding.

Employment hours

Overwork = C(Employment hours) = I(Hours > 50)

Hourly wageGender

E(Wage | do(Overwork),Man

)=∑

h>50

(E(Wage|do(Hours=h),Man

)×P(Hours=h|Hours>50,Man

))

E(Wage | do(Overwork),Woman

)=∑

h>50

(E(Wage|do(Hours=h),Woman

)×P(Hours=h|Hours>50,Woman

))Heterogeneous treatment effects

Effects of heterogeneous treatments

Page 90: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

What is the effect of overwork (> 50 hours) on thehourly wage of those working at least 40 hours per week?

Suppose gender is the only source of confounding.

Employment hours

Overwork = C(Employment hours) = I(Hours > 50)

Hourly wageGender

E(Wage | do(Overwork),Man

)=∑

h>50

(E(Wage|do(Hours=h),Man

)×P(Hours=h|Hours>50,Man

))

E(Wage | do(Overwork),Woman

)=∑

h>50

(E(Wage|do(Hours=h),Woman

)×P(Hours=h|Hours>50,Woman

))

Heterogeneous treatment effects

Effects of heterogeneous treatments

Page 91: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

What is the effect of overwork (> 50 hours) on thehourly wage of those working at least 40 hours per week?

Suppose gender is the only source of confounding.

Employment hours

Overwork = C(Employment hours) = I(Hours > 50)

Hourly wageGender

E(Wage | do(Overwork),Man

)=∑

h>50

(E(Wage|do(Hours=h),Man

)×P(Hours=h|Hours>50,Man

))

E(Wage | do(Overwork),Woman

)=∑

h>50

(E(Wage|do(Hours=h),Woman

)×P(Hours=h|Hours>50,Woman

))

Heterogeneous treatment effectsEffects of heterogeneous treatments

Page 92: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 93: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 94: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

What To Do

In randomized experiments aiming to generalize:

Randomize a detailed treatment

Theorize context-specific versions likely to remain

In observational studies:

Estimate at the finest level of detail measured

— Promotes a simple definition of the effect— Promotes transportability— Promotes clear policy implications

If treatment remains vague, state the implied intervention.

Page 95: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

Versions of Treatment Formalizing the Problem When It Matters What To Do

Versions of TreatmentA Causal Inference Debate Sociologists Have Ignored

1. Formalizing the problemA) Potential outcomes

B) Stochastic counterfactuals

C) Causal graphs

2. When it matters: Consequences of collapsed versions

A) Experimental studies: Effects may not generalize

B) Observational studies:

— “Effects” may be an unusual average

— Heterogeneous treatment effects may really be

— the effects of heterogeneous treatments

3. Recommendations: What to do

Page 96: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

References

Appendix

Page 97: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

References

Some define the assumptions for causal inference as:

Ignorable treatment assignment

— Violated if the treated would do better even without treatment

Positivity

— Violated if P(Treated) is 0 or 1 for some units

Stable Unit Treatment Value Assumption

— Violated if there is interference— Violated if there are hidden versions of treatment

In this setup, hidden versions are the second part of SUTVA.Social scientists often focus on the first assumptions and give lessthought to this part of SUTVA.

Page 98: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

References

Why not use the Y (a, da) notation?

One could state potential outcomes as a function of bothtreatment and treatment version (VanderWeele, 2009; Hernan andVanderWeele, 2011; VanderWeele and Hernan, 2013).

Versions of treatment

A D Y

Yi (a, da) is unnecessary notation, though. Because only one aexists for any given d , Yi (d) carries the same information.In contrast, this is useful in mediation.

Mediation

A

M

Y

Yi (a,m) is valuable. Because A is not fixed given M, there existmultiple {a, a′} with Yi (a,m) 6= Yi (a

′,m).

Page 99: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

References

Why not put A on the DAG as a consequence of D?

When the researchers take a detailed treatment D and coarsen itinto an aggregate treatment A, Hernan and VanderWeele (2011)put it in the DAG as a consequence of D.

D A Y

The reasons not to do this are

1. In a DAG, it is useful to be able to conceive of an interventionto any given node. Because D → A is deterministic, it is hardto imagine an intervention to A which has no consequence forD. By the DAG, this intervention would have no consequencefor D. This seems hard to swallow.

2. Perhaps A is not deterministic: it is reported D. But thisseems like a whole different set of issues, and it is clear evenwithout the DAG that intervening to change a report wouldhave no consequence for Y .

Page 100: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

References

What about when A→ D is confounded?

When treatment precedes version, Hernan and VanderWeele(2011) also include cases like below:

U

A D Y

The reasons not to do this are

1. The edge U → A implies this is an observational study ratherthan an experiment. In observational studies, I usually do notbelieve the story that A is assigned first, followed by D. Ithink in observational studies D is typically the only variableinvolved.

2. We already have transportability issues from U → D alone.Omitting U → A helps to highlight these problems in thescenario when A is randomized.

Page 101: Versions of Treatment - Princeton University...Android push received No push received Takes 10-minute walk Versions of Treatment Formalizing the Problem When It MattersWhat To Do y

References

References

Hernan, M. A. 2016. Does water kill? A call for less casual causalinferences. Annals of epidemiology, 26(10):674–680.

Hernan, M. A. and T. J. VanderWeele 2011. Compoundtreatments and transportability of causal inference.Epidemiology, 22(3):368.

Imbens, G. W. and D. B. Rubin 2015. Causal Inference inStatistics, Social, and Biomedical Sciences. CambridgeUniversity Press.

Pearl, J. 2010. Brief report: On the consistency rule in causalinference: Axiom, definition, assumption, or theorem?Epidemiology, Pp. 872–875.

VanderWeele, T. J. 2009. Concerning the consistency assumptionin causal inference. Epidemiology, 20(6):880–883.

VanderWeele, T. J. and M. A. Hernan 2013. Causal inferenceunder multiple versions of treatment. Journal of CausalInference, 1(1):1–20.