Handout 1

K. Tatsiramos - Topics in Econometrics, University of Nottingham

Topics in EconometricsHandout 1

Konstantinos TatsiramosUniversity of Nottingham

January 2015

1 / 37


Outline

I IntroductionI The Fundamental Problem of Causal InferenceI Treatment Effects of InterestI Naive Estimation of Treatment Effects

2 / 37


Introduction

I The methods studied in this course are usually employed inmicroeconometric analysis.

I Microeconometric analysis is about the analysis of micro-leveldata on the economic behaviour of agents (individuals,households, firms).

I Data include cross-sectional surveys, longitudinal surveys andcensuses.

I Often concerned with the effect of policy interventions oneconomic behaviour.

I Our emphasis will be on methods and assumptions we need tomaintain to determine causation rather than just measurecorrelations.

3 / 37


Causality versus Correlation

I Running a regression of Y on X will in general be informativeabout patterns of correlation.

I However, correlation is not causation.

I The ceteris paribus effect (i.e. holding all other relevantfactors fixed) may be given a causal interpretation if we havefully controlled for all other relevant factors.

I In most cases this is not possible (unobserved heterogeneity)and may also not be sufficient.

I We need to develop the methods and specify the maintainedassumptions for identifying the causal effect of X on Y .

4 / 37


Classic Example - Schooling and Earnings

I Suppose schooling (x1) is the source of variation in earnings(y).

I Suppose ability (x2) is another source of variation but it isignored in the model (unobserved).

I By ignoring ability, the part of the variation of earnings that isdue to ability is incorrectly attributed to schooling.

I So the relative importance of schooling and ability inexplaining earnings is confounded.

I The leading source of confounding bias is the omission offactors that affect the outcome of interest and are correlatedwith other relevant factors.

5 / 37


Program Evaluation or Treatment Evaluation

I Program Evaluation is a new strand in the microeconometricliterature which provides a statistical framework for theestimation of causal parameters.

I In medical studies, a new drug is tested by comparing theresponse to the drug between a group which is treated andanother which is untreated.

I In economics, we are interested in evaluating the effect ofexposure of individuals, households, or firms to a program(treatment), on some outcome.

I Unlike medical studies, it is not always possible to useexperimental variation so we may need to rely onobservational data.

6 / 37


Examples of Programs/Treatments

I Labor market training programs.

I School types (e.g. catholic/private vs. public schools).

I Being a member of a trade union.

I Receipt of a transfer payment.

I Changes in regulation for receiving a transfer payment.

I Changes in rules and regulations in financial transactions.

I etc.

7 / 37


Training programs and Earnings

I Training programs for disadvantaged individuals (vocationaleducation, on-the-job training etc.).

I Participants include heterogenous individuals such as welfarerecipients, displaced workers etc.

I Evaluating such programs poses significant challenges.

I Trainees tend to experience a downward spiral in earnings justbefore receiving training (Ashenfelters Dip).

I Even in the absence of training, the wages of trainees wouldrise to some degree.

I Comparing earnings before and after program participationmay lead to significantly biased results.

8 / 37


School Type and Educational Outcomes

I Early research has shown that Catholic schools are moreeffective than public schools in teaching maths and reading.

I Criticism: Students in the two types of schools areinsufficiently comparable even after adjusting for familybackground and motivation.

I Self-Selection: those who are more likely to benefit fromCatholic schools will enroll net of all observable characteristics.

9 / 37


Outline


10 / 37


Potential Outcomes or Counterfactuals

I Consider the existence of two well-defined causal states inwhich individuals can be exposed to.

I We can think of one state as the treatment and the otheras the control.

I The key assumption of the counterfactual framework is thefollowing:I Each individual in the population of interest has a potential

outcome under each treatment state.

I However, each individual can be observed in only onetreatment state at any point in time.

I Despite this fundamental problem, the potential outcomesmodel allows to conceptualise observational studies as if theywere experimental designs.

11 / 37


Example

I Consider, for example, the effect of having a college degreerather than only a high school degree on earnings.

I Those who have high school degrees have theoretical what-ifearnings under the state have a college degree.

I Those who have completed college degrees have theoreticalwhat-if earnings under the state have only a high schooldegree.

I These what-if potential outcomes are counterfactual, theyare not actually observed.

12 / 37


The Fundamental Problem of Causal Inference I

I The evaluation is based on the comparison of outcomes underthe two possible states of treatment.

I The problem is that we can at most observe one of theseoutcomes because each unit can be exposed to only one levelof treatment.

I We cannot observe the outcome of a unit both when treatedand during the counterfactual situation of not being treated.

I Causal effects cannot be calculated at the individual level.

13 / 37


The Fundamental Problem of Causal Inference II

I To perform the evaluation we need to compare distinct unitsreceiving different levels of the treatment (treated vs.control).

I This comparison can involve different physical units or thesame physical unit at different times.

I In the training program example we could compare theoutcomes before and after participating in training.

I In the Catholic school example we could compare studentswho attend Catholic vs. public schools.

14 / 37


The Fundamental Problem of Causal Inference III

I The threat of identification is that the treated are bydefinition different from the non-treated.

I If the differences that determine treatment also influence theoutcomes, then this may invalidate the comparison ofoutcomes by treatment status.

15 / 37


The Potential Outcomes Model

I Consider program participation over a population of interest.

I We denote with the indicator Di the causal exposure variable:I Di = 1 for those exposed to the treatment stateI Di = 0 for those exposed to the control state

I For each individual i there are two potential outcomes:Yi (0) and Yi (1).

I The potential outcome Yi (0) denotes the outcome that wouldbe realized if i does not participate in the program.

I The potential outcome Yi (1) denotes the outcome that wouldbe realized if i participates in the program.

I The individual-level causal effect of the treatment is definedas:

i = Yi (1) Yi (0)

16 / 37


Treatment Groups and Observed Outcomes

I We can define the observed outcome variable Yi in terms ofthe potential outcomes and the treatment variable.

I The observed outcome is defined as follows:

Yi = Yi (1) if Di = 1

Yi = Yi (0) if Di = 0

I This can be written compactly as:

Yi = Yi (Di ) = DiYi (1) + (1 Di )Yi (0)

17 / 37


Table: The Fundamental Problem of Causal Inference

Group Yi (1) Yi (0)

Treatment Group (Di = 1) Observable as Yi CounterfactualControl Group (Di = 0) Counterfactual Observable as Yi

I Individual causal effects are defined within rows but are notobservable.

I Only the diagonal of the table is observable.I The outcome variable Yi reveals only half of the information

contained in the underlying potential outcome variables.

I Individuals contribute outcome information only from theobserved treatment state.

18 / 37


Outline


19 / 37


The Average Treatment Effect

I Because we cannot estimate individual treatment effects, wefocus on averages.

I The average treatment effect in the population is:

ATE E [] = E [Yi (1) Yi (0)]

= E [Yi (1)] E [Yi (0)]I It measures the expected causal effect of randomly assigning a

person in the population to the program.

I It is relevant if the policy under consideration would expose allunits of the population to treatment or none at all.

20 / 37


Example of ATE

I In the Catholic school example the individual treatment effecti is the what-if difference in outcomes if a person i waseducated both in a Catholic and in a public school.

I The average treatment effect (E []) is the mean valueamong all students in the population of these what-ifdifferences in outcomes.

I It is equal to the expected value of the what-if difference inoutcomes for a randomly selected student from thepopulation.

21 / 37


Conditional Average Treatment Effects - ATT

Sometimes a treatment is available to individuals without forcingthem to take the treatment. We would like to know how effectivethe treatment is for those who choose to take the treatment.

I The Average Treatment Effect for the Treated (ATT) is themean effect for those who typically take the treatment.

ATT E [Yi (1) Yi (0)|Di = 1]

= E [Yi (1)|Di = 1] E [Yi (0)|Di = 1]I ATT in many cases is the more interesting parameter: is a

program beneficial for participants?

I Catholic school example: ATT is the average effect for thosewho attend Catholic schools; not across all students whocould potentially attend.

22 / 37


Conditional Average Treatment Effects - ATC

I The Average Treatment Effect for the Controls (ATC) is themean effect for those who typically do not take the treatment.

ATC E [Yi (1) Yi (0)|Di = 0]

= E [Yi (1)|Di = 0] E [Yi (0)|Di = 0]I ATC is of interest if the goal is to determine the effect of a

potential policy intervention.

I Catholic school example: a new school voucher programwhich moves students from public to Catholic schools.

23 / 37


Taking Stock

I The potential outcomes model allows to conceptualizeobservational studies as if they were experimental designs.

I Each individual in the population of interest has a potentialoutcome under each treatment state.

I Because of the fundamental problem of causal inference wefocus on non-individual causal effects (ATE, ATT etc.).

24 / 37


Outline


25 / 37


Naive Estimation of ATE

I The naive estimator of ATE is the following:

NAIVE = E [Yi |Di = 1] E [Yi |Di = 0]= E [Yi (1)|Di = 1] E [Yi (0)|Di = 0]

I With observational data, in general, the naive estimator doesnot converge to a consistent estimate of the ATE.

I To see this, consider the following decomposition of ATE:

E [] = {piE [Yi (1)|Di = 1] + (1 pi)E [Yi (1)|Di = 0]}{piE [Yi (0)|Di = 1] + (1 pi)E [Yi (0)|Di = 0]}

where pi is the proportion of individuals in the population ofinterest that takes the treatment.

I We do not observe the counterfactual conditionalexpectations.

26 / 37


Expected Bias of Naive Estimator

I The decomposition of ATE can be also written as follows:

E [Yi (1)|Di = 1] E [Yi (0)|Di = 0] =

E []

+{E [Yi (0)|Di = 1] E [Yi (0)|Di = 0]}+(1 pi){E [|Di = 1] E [|Di = 0]}.

I The naive estimator converges to the LHS of this equation,which is not equal only to the ATE.

I There are two sources of bias: baseline bias and differentialtreatment effect bias.

27 / 37


Sources of Bias

I The baseline bias is equal to the difference in the averageoutcome in the absence of the treatment, Yi (0), betweenthose in the treatment group and those in the control group.

I The differential treatment effect bias is equal to the expecteddifference in the treatment effect, E [], between those in thetreatment and those in the control group.

28 / 37


Example for the Bias of the Naive Estimator

Consider the treatment to be college education and a labor marketoutcome.

Group E [Yi (1)|.] E [Yi (0)|.]Treatment Group (Di = 1) 10 6Control Group (Di = 0) 8 5

I The naive comparison between college (COL) and high school(HS) graduates gives a difference of 5 (10-5).

29 / 37


Example (continued)

Group E [Yi (1)|.] E [Yi (0)|.]Treatment Group (Di = 1) 10 6Control Group (Di = 0) 8 5

I COL would have done better than HS even if had not gone tocollege (6 vs. 5) - baseline bias. Note this is opposite for HS(8 vs. 10).

I ATT is higher (10 6 = 4) than ATC (8 5 = 3)differential treatment effect bias.

I If the proportion of COL is 30% then:ATE = 0.3(10 6) + (1 0.3)(8 5) = 3.3

I Total Bias=baseline bias (6 5)+differential treatmenteffect bias (1 0.3)(4 3) = 1.7

30 / 37


Maintained Assumptions and Causal Effects

I Under what assumptions the naive estimator will identify theATE?

Assumption 1 : E [Yi (1)|Di = 1] = E [Yi (1)|Di = 0]

Assumption 2 : E [Yi (0)|Di = 1] = E [Yi (0)|Di = 0]I If both hold then we know the counterfactuals and there is no

bias : ATE = ATT = ATC .

I When these assumptions are reasonable?

I When treatment is assigned randomly: Yi (1),Yi (0) D(independence assumption).

I With observational data is rarely justified.

31 / 37


Naive Estimation Assuming Independence

I Consider the observed outcome Yi :

Yi = DiYi (1) + (1 Di )Yi (0) =

Yi (0) + Di [Yi (1) Yi (0)]I Taking expectations:

E [Yi |Di ] =E [Yi (0)|Di ] + Di{E [Yi (1)|Di ] E [Yi (0)|Di ]} =

E [Yi (0)] + Di{E [Yi (1)] E [Yi (0)]}I The last equality follows by assuming independence.I Then:

E [Yi |Di = 1] E [Yi |Di = 0] =E [Yi (1)] E [Yi (0)] ATE

32 / 37


Maintained Assumptions and Causal Effects

I If only Ass1 holds then the naive estimator is consistent forATC:

NAIVE E [Yi (1)|Di = 1] E [Yi (0)|Di = 0]

= E [Yi (1)|Di = 0] E [Yi (0)|Di = 0] ATCI Use the observed outcome of the treated as the counterfactual

for the non-treated.

I If only Ass2 holds then the naive estimator is consistent forATT:

NAIVE E [Yi (1)|Di = 1] E [Yi (0)|Di = 0]

= E [Yi (1)|Di = 1] E [Yi (0)|Di = 1] ATTI Use the observed outcome of the controls as the appropriate

counterfactual for the treated.

33 / 37


The Stable Unit Treatment Value Assumption (SUTVA)

I SUTVA is a basic assumption of causal effect stability.

I SUTVA requires that the potential outcomes of individuali do not depend on the treatments received by otherindividuals.

I In economics, it is referred to as a no-macro effect or partialequilibrium assumption.

I It is violated when the underlying causal effects are a functionof the treatment assignment patterns.

I That is, when the treatment effect differs (e.g. be lesseffective) when more individuals are assigned to it.

I In the Catholic schooling example, school effectiveness shouldnot depend on the number/composition of students who enterCatholic schools.

34 / 37


SUTVA (continued)

I If SUTVA is maintained but there is some doubt about itsvalidity, then certain types of marginal effects can still bedefended.

I The estimates of ATE hold only for what-if movements of avery small number of individuals from one hypotheticaltreatment state to another.

I For more extensive interventions when SUTVA is clearlyviolated the variation of the causal effect as a function oftreatment assignment patterns need to be modeled explicitly.

I This requires to develop a full structural model and it isbeyond the scope of this course.

I In what follows we will maintain SUTVA.

35 / 37


Taking Stock

I With observational data, if we do not impose independence,then we need to find solutions for correcting the bias.

I We need to understand what determines treatment selection.

I An ideal possibility is to rely on experimental variation intreatment selection (assignment).

I If that is not available we can consider subgroups for whichthe assumptions for identification can be defended.

I This strategy involves conditioning on observed variables fromthe population.

I This leads us to the idea of the regression in the potentialoutcomes framework.

36 / 37


References

I The material in this handout follows:

Stephen L. Morgan and Christopher Winship Counterfactualsand Causal Inference. Methods and Principles for SocialResearch. 2007. Cambridge University Press.

I Required reading: Sections 1.3, 1.4, 2.1-2.6

I Suggested reading: Sections 1.1-1.2 provide a nice historicaldiscussion of the framework.

37 / 37

Handout 1

Documents

Transcript of Handout 1