28 April 2009 @ Crawford School 1 Causality and Causal Inference Semester 1, 2009 POGO8096/8196:...

28 April 2009 @ Crawford School 1

Causality and Causality and Causal InferenceCausal Inference

Semester 1, 2009Semester 1, 2009

POGO8096/8196: Research POGO8096/8196: Research MethodsMethods

Crawford School of Economics and Government Crawford School of Economics and Government

228 April 2009 @ Crawford School

This weekThis week

Research design and causal thinkingResearch design and causal thinking What is a research design?What is a research design? Conditions of causalityConditions of causality

Research designs for causal thinkingResearch designs for causal thinking True experimental designTrue experimental design Quasi-experimental designQuasi-experimental design Correlation designCorrelation design (Design without a control group)(Design without a control group)


What is a research What is a research design?design?

Narrow definition [today’s topic]Narrow definition [today’s topic] Research design refers to the logical method Research design refers to the logical method

by which we propose to test a hypothesis.by which we propose to test a hypothesis. Broad definitionBroad definition

Research design refers to a whole proposal Research design refers to a whole proposal for a research project, including the review for a research project, including the review of the literature, research questions and of the literature, research questions and hypotheses, details of data collection, hypotheses, details of data collection, methods of data analysis, expected methods of data analysis, expected outcomes, a budget proposal, etc.outcomes, a budget proposal, etc.


Why do we need it?Why do we need it? In empirical research, we often want to In empirical research, we often want to

test a test a causalcausal theory (or more specifically, theory (or more specifically, a causal hypothesis) stating that a causal hypothesis) stating that XX affects affects YY. .

But … How do we know that But … How do we know that XX causes causes YY, , but not the other way around? How do but not the other way around? How do we know that it is we know that it is XX that causes that causes YY, but , but not something else (not something else (WW, , ZZ or whatever)? or whatever)?

We need a valid research design: a We need a valid research design: a method with which we can test the method with which we can test the causalcausal hypothesis. hypothesis.


Conditions for causalityConditions for causality

Co-variationCo-variation (or (or CorrelationCorrelation): Two phenomena ): Two phenomena (i.e., variables) tend to be related. If one of the (i.e., variables) tend to be related. If one of the two variables changes, another variable two variables changes, another variable changes. [Necessary condition for causality.]changes. [Necessary condition for causality.]

Time-orderTime-order: The presumed cause (an : The presumed cause (an independent variable) happened before the independent variable) happened before the presumed consequence (a dependent presumed consequence (a dependent variable).variable).

Non-spuriousnessNon-spuriousness: The co-variation between : The co-variation between independent and dependent variables is not independent and dependent variables is not caused by other factors.caused by other factors.


Research designs for Research designs for causal inferencecausal inference

True experimental designTrue experimental design Almost always quantitativeAlmost always quantitative

Quasi experimental designQuasi experimental design Often quantitativeOften quantitative ““natural experiment” in Shivelynatural experiment” in Shively

Correlational designCorrelational design Both qualitative (intentional selection of Both qualitative (intentional selection of

observations) and quantitative (correlation and observations) and quantitative (correlation and regression analysis)regression analysis)

““natural experiment without pre-measurement” natural experiment without pre-measurement” in Shivelyin Shively


True experiment – 1True experiment – 1

Physical, biological and medical Physical, biological and medical sciences, as well as some social sciences, as well as some social sciences (e.g., psychology), have sciences (e.g., psychology), have traditionally used true experiments. It is traditionally used true experiments. It is becoming popular in other social becoming popular in other social science disciplines.science disciplines.

It is very important to understand how It is very important to understand how an experiment is set up, because the an experiment is set up, because the logic involved is highly relevant to all logic involved is highly relevant to all types of research design for causal types of research design for causal thinking.thinking.


True experiment – 2True experiment – 2 The classic (simplest) experimental The classic (simplest) experimental

designdesign1.1. Select a sample of your experimental study.Select a sample of your experimental study.2.2. Randomly divide the subjects (e.g., Randomly divide the subjects (e.g.,

students) into two groups: the students) into two groups: the “experimental (or treatment) group” and “experimental (or treatment) group” and the “control group.” the “control group.”

3.3. Give a “stimulus (or treatment)” only to the Give a “stimulus (or treatment)” only to the experimental group.experimental group.

4.4. Measure the dependent variable and Measure the dependent variable and compare the difference in the dependent compare the difference in the dependent variable between the groups.variable between the groups.


“Random Assignment”

Note: In this course, “matching” means something different (= the intentional selection of observations).


Types of true Types of true experimentsexperiments

LaboratoryLaboratory experiment experiment Small-scale experiment in artificial settingSmall-scale experiment in artificial setting The effects of negative TV campaigns on The effects of negative TV campaigns on

voter’s preference of presidential voter’s preference of presidential candidates and on voter turnout candidates and on voter turnout (Ansolabehere and Iyenger 1995).(Ansolabehere and Iyenger 1995).

FieldField experiment experiment Large-scale experiment in “real” settingLarge-scale experiment in “real” setting The effects of personal canvassing, The effects of personal canvassing,

telephone calls, and direct mail on voter telephone calls, and direct mail on voter turnout (Gerber and Green 2000).turnout (Gerber and Green 2000).


MeritsMerits

Well-structured experiments can control other Well-structured experiments can control other factors almost perfectly, because the attributes factors almost perfectly, because the attributes of experimental and control groups (observable of experimental and control groups (observable and unobservable) are, and unobservable) are, on averageon average, the same., the same.

They also satisfy the “time-order” condition of They also satisfy the “time-order” condition of causality.causality.

Thus, “only randomization [i.e., a true Thus, “only randomization [i.e., a true experiment with random assignment] provides experiment with random assignment] provides a clear enough causal interpretation to settle a clear enough causal interpretation to settle issues of social-scientific research issues of social-scientific research conclusively” (Shively, p. 87).conclusively” (Shively, p. 87).


LimitationsLimitations In both laboratory and field experimentsIn both laboratory and field experiments

Experimental research is often limited to Experimental research is often limited to investigations of political and social investigations of political and social communications.communications.

It may be ethically inappropriate to conduct It may be ethically inappropriate to conduct experiments with human beings, societies and experiments with human beings, societies and politics as subjects. politics as subjects.

In laboratory experimentsIn laboratory experiments Samples in an experiment may not represent Samples in an experiment may not represent

the population.the population. Experiments are conducted in an artificial Experiments are conducted in an artificial

setting.setting.


Quasi experimentQuasi experiment The basic quasi experimental designThe basic quasi experimental design

1.1. Measure the dependent variable of the Measure the dependent variable of the subjects.subjects.

2.2. Wait until some subjects are exposed to the Wait until some subjects are exposed to the independent variable, or observe that the independent variable, or observe that the values of the independent variable change values of the independent variable change among the subjects. (among the subjects. (NoteNote: The subjects : The subjects are are notnot assigned to groups: no random assigned to groups: no random assignment.)assignment.)

3.3. Measure the dependent variable again, and Measure the dependent variable again, and compare the difference in the dependent compare the difference in the dependent variable among the subjects.variable among the subjects.


Other examplesOther examples

““Panel” studies – the same set of Panel” studies – the same set of people are surveyed multiple times. people are surveyed multiple times.

““Before-and-after” studies in policy-Before-and-after” studies in policy-oriented research.oriented research. e.g., the effects of increasing speed limits e.g., the effects of increasing speed limits

on the number of traffic fatalities. (Not all on the number of traffic fatalities. (Not all states in the US increased the speed states in the US increased the speed limits.) limits.)

NoteNote: There should be a “control group” : There should be a “control group” in these research.in these research.


Merits and LimitationsMerits and Limitations They satisfy the “time-order” condition – They satisfy the “time-order” condition –

The value of a dependent variable The value of a dependent variable changeschanges after the value of a key independent after the value of a key independent variable is observed (or changed).variable is observed (or changed).

Important:Important: They can control “subject- They can control “subject-specific, time-invariant” variables, but not specific, time-invariant” variables, but not others.others.

You should carefully examine what other You should carefully examine what other factors might affect the causal relationship factors might affect the causal relationship and try to control them (if possible). and try to control them (if possible).


Correlational designCorrelational design

The basic correlational designThe basic correlational design1.1. Select a sample of your study.Select a sample of your study.

2.2. Measure the dependent and Measure the dependent and independent variables. (independent variables. (NoteNote: The : The values of the independent variables values of the independent variables must vary among the subjects.)must vary among the subjects.)

3.3. If the dependent variable differs If the dependent variable differs among the subjects, ascribe this to the among the subjects, ascribe this to the effect of the independent variable.effect of the independent variable.


Other examplesOther examples

Studies examining how survey respondents Studies examining how survey respondents with different attributes vote differently.with different attributes vote differently.

Studies examining how countries with Studies examining how countries with different economic conditions experience different economic conditions experience different environmental issues and conflicts.different environmental issues and conflicts.

Studies examining how Indonesian local Studies examining how Indonesian local governments with different social governments with different social conditions affect their budget performance.conditions affect their budget performance.


Merits and LimitationsMerits and Limitations The “time-order” condition is not always The “time-order” condition is not always

satisfied. We need “auxiliary” satisfied. We need “auxiliary” information (e.g., common sense, prior information (e.g., common sense, prior knowledge, etc.) to judge the direction knowledge, etc.) to judge the direction of causality.of causality.

A correlational design with just one A correlational design with just one independent variable fails to control independent variable fails to control other factors. other factors.

You should carefully examine what other You should carefully examine what other factors might affect the causal factors might affect the causal relationship and try to control them. relationship and try to control them.


Design without a control Design without a control groupgroup

A typical design without a control groupA typical design without a control group1.1. Measure a certain phenomenon or behavior Measure a certain phenomenon or behavior

(A) you want to explain. (A) you want to explain.

2.2. Observe a certain phenomenon, behavior or Observe a certain phenomenon, behavior or other exogenous shocks (B) that you think other exogenous shocks (B) that you think as a “cause”. as a “cause”.

3.3. Measure the phenomenon or behavior (A’) Measure the phenomenon or behavior (A’) you want to explain again. If the you want to explain again. If the phenomenon or behavior has changed from phenomenon or behavior has changed from A to A’, then ascribe this change to the A to A’, then ascribe this change to the occurrence of B. occurrence of B.


An ExampleAn Example

Taiwan presidential election in 2000.Taiwan presidential election in 2000. The pre-election level of support for each candidatThe pre-election level of support for each candidat

e.e. China’s military actions near Taiwan, which are intChina’s military actions near Taiwan, which are int

ended to affect the election result.ended to affect the election result. The results of the presidential electionThe results of the presidential election

The Democratic Progressive Party’s Chen Shui-bian wonThe Democratic Progressive Party’s Chen Shui-bian won The ruling Kuomingtang’s Lien Chan lost.The ruling Kuomingtang’s Lien Chan lost.

Many argue that China’s attempts were counterprMany argue that China’s attempts were counterproductive and helped Chen to win. Really? oductive and helped Chen to win. Really?


Control groupControl group

ImportantImportant: A control “group” and a : A control “group” and a control “variable” are different. If control “variable” are different. If there is no control group, your there is no control group, your independent variable does not independent variable does not varyvary. . A control group = a portion of your A control group = a portion of your

observations (the subjects of your study) observations (the subjects of your study) that are not exposed to your key that are not exposed to your key independent variable.independent variable.

A control variable = another factor A control variable = another factor explaining the dependent variable.explaining the dependent variable.


ExamplesExamples

An example of true experiments: An example of true experiments: Taking an introductory Australian Government Taking an introductory Australian Government

course (course (XX) increases political interest () increases political interest (YY). ). “Students who do not take the course.”“Students who do not take the course.”

An example of natural experiments: An example of natural experiments: Watching a presidential debate (Watching a presidential debate (XX) increases ) increases

intensity of support (intensity of support (YY). ). “Students who do “Students who do not watch the debate.”not watch the debate.”

An example of correlational designs: An example of correlational designs: Voter turnout (Voter turnout (YY) is lower in urban areas () is lower in urban areas (XX). ).

“Rural areas.” “Rural areas.”


LimitationsLimitations

Research design without a control group is very Research design without a control group is very common in historical descriptions of political, common in historical descriptions of political, social and economic events. This method may social and economic events. This method may be useful to understand historical processes be useful to understand historical processes and details, but it is and details, but it is notnot recommended for recommended for causal analysis. Why? Because without a causal analysis. Why? Because without a control group you cannot say that a variation in control group you cannot say that a variation in YY is caused by a variation in is caused by a variation in XX (because there is (because there is no variation in no variation in XX). ).

They may, however, help us identify some new They may, however, help us identify some new measures, theories, and/or puzzles.measures, theories, and/or puzzles.


SummarySummary

Random Random AssignmeAssignme

ntnt

Pre-Pre-measuremmeasurem

entent

Control Control GroupGroup

True True experimental experimental designdesign

YesYes Yes/NoYes/No YesYes

Quasi Quasi experimental experimental designdesign

NoNo YesYes YesYes

Correlational Correlational designdesign

NoNo NoNo YesYes

Design without a Design without a control groupcontrol group

NoNo YesYes NoNo


Next weekNext week

Intentional selection of observationsIntentional selection of observations How to choose observations?How to choose observations? How to avoid problematic causal How to avoid problematic causal

inference?inference? Some controversiesSome controversies

Observations vs. casesObservations vs. cases Objectives of qualitative researchObjectives of qualitative research

28 April 2009 @ Crawford School 1 Causality and Causal Inference Semester 1, 2009 POGO8096/8196:...

Documents

Transcript of 28 April 2009 @ Crawford School 1 Causality and Causal Inference Semester 1, 2009 POGO8096/8196:...