28 April 2009 @ Crawford School 1
Causality and Causality and Causal InferenceCausal Inference
Semester 1, 2009Semester 1, 2009
POGO8096/8196: Research POGO8096/8196: Research MethodsMethods
Crawford School of Economics and Government Crawford School of Economics and Government
228 April 2009 @ Crawford School
This weekThis week
Research design and causal thinkingResearch design and causal thinking What is a research design?What is a research design? Conditions of causalityConditions of causality
Research designs for causal thinkingResearch designs for causal thinking True experimental designTrue experimental design Quasi-experimental designQuasi-experimental design Correlation designCorrelation design (Design without a control group)(Design without a control group)
328 April 2009 @ Crawford School
What is a research What is a research design?design?
Narrow definition [today’s topic]Narrow definition [today’s topic] Research design refers to the logical method Research design refers to the logical method
by which we propose to test a hypothesis.by which we propose to test a hypothesis. Broad definitionBroad definition
Research design refers to a whole proposal Research design refers to a whole proposal for a research project, including the review for a research project, including the review of the literature, research questions and of the literature, research questions and hypotheses, details of data collection, hypotheses, details of data collection, methods of data analysis, expected methods of data analysis, expected outcomes, a budget proposal, etc.outcomes, a budget proposal, etc.
428 April 2009 @ Crawford School
Why do we need it?Why do we need it? In empirical research, we often want to In empirical research, we often want to
test a test a causalcausal theory (or more specifically, theory (or more specifically, a causal hypothesis) stating that a causal hypothesis) stating that XX affects affects YY. .
But … How do we know that But … How do we know that XX causes causes YY, , but not the other way around? How do but not the other way around? How do we know that it is we know that it is XX that causes that causes YY, but , but not something else (not something else (WW, , ZZ or whatever)? or whatever)?
We need a valid research design: a We need a valid research design: a method with which we can test the method with which we can test the causalcausal hypothesis. hypothesis.
528 April 2009 @ Crawford School
Conditions for causalityConditions for causality
Co-variationCo-variation (or (or CorrelationCorrelation): Two phenomena ): Two phenomena (i.e., variables) tend to be related. If one of the (i.e., variables) tend to be related. If one of the two variables changes, another variable two variables changes, another variable changes. [Necessary condition for causality.]changes. [Necessary condition for causality.]
Time-orderTime-order: The presumed cause (an : The presumed cause (an independent variable) happened before the independent variable) happened before the presumed consequence (a dependent presumed consequence (a dependent variable).variable).
Non-spuriousnessNon-spuriousness: The co-variation between : The co-variation between independent and dependent variables is not independent and dependent variables is not caused by other factors.caused by other factors.
628 April 2009 @ Crawford School
Research designs for Research designs for causal inferencecausal inference
True experimental designTrue experimental design Almost always quantitativeAlmost always quantitative
Quasi experimental designQuasi experimental design Often quantitativeOften quantitative ““natural experiment” in Shivelynatural experiment” in Shively
Correlational designCorrelational design Both qualitative (intentional selection of Both qualitative (intentional selection of
observations) and quantitative (correlation and observations) and quantitative (correlation and regression analysis)regression analysis)
““natural experiment without pre-measurement” natural experiment without pre-measurement” in Shivelyin Shively
728 April 2009 @ Crawford School
True experiment – 1True experiment – 1
Physical, biological and medical Physical, biological and medical sciences, as well as some social sciences, as well as some social sciences (e.g., psychology), have sciences (e.g., psychology), have traditionally used true experiments. It is traditionally used true experiments. It is becoming popular in other social becoming popular in other social science disciplines.science disciplines.
It is very important to understand how It is very important to understand how an experiment is set up, because the an experiment is set up, because the logic involved is highly relevant to all logic involved is highly relevant to all types of research design for causal types of research design for causal thinking.thinking.
828 April 2009 @ Crawford School
True experiment – 2True experiment – 2 The classic (simplest) experimental The classic (simplest) experimental
designdesign1.1. Select a sample of your experimental study.Select a sample of your experimental study.2.2. Randomly divide the subjects (e.g., Randomly divide the subjects (e.g.,
students) into two groups: the students) into two groups: the “experimental (or treatment) group” and “experimental (or treatment) group” and the “control group.” the “control group.”
3.3. Give a “stimulus (or treatment)” only to the Give a “stimulus (or treatment)” only to the experimental group.experimental group.
4.4. Measure the dependent variable and Measure the dependent variable and compare the difference in the dependent compare the difference in the dependent variable between the groups.variable between the groups.
928 April 2009 @ Crawford School
“Random Assignment”
Note: In this course, “matching” means something different (= the intentional selection of observations).
1028 April 2009 @ Crawford School
Types of true Types of true experimentsexperiments
LaboratoryLaboratory experiment experiment Small-scale experiment in artificial settingSmall-scale experiment in artificial setting The effects of negative TV campaigns on The effects of negative TV campaigns on
voter’s preference of presidential voter’s preference of presidential candidates and on voter turnout candidates and on voter turnout (Ansolabehere and Iyenger 1995).(Ansolabehere and Iyenger 1995).
FieldField experiment experiment Large-scale experiment in “real” settingLarge-scale experiment in “real” setting The effects of personal canvassing, The effects of personal canvassing,
telephone calls, and direct mail on voter telephone calls, and direct mail on voter turnout (Gerber and Green 2000).turnout (Gerber and Green 2000).
1128 April 2009 @ Crawford School
MeritsMerits
Well-structured experiments can control other Well-structured experiments can control other factors almost perfectly, because the attributes factors almost perfectly, because the attributes of experimental and control groups (observable of experimental and control groups (observable and unobservable) are, and unobservable) are, on averageon average, the same., the same.
They also satisfy the “time-order” condition of They also satisfy the “time-order” condition of causality.causality.
Thus, “only randomization [i.e., a true Thus, “only randomization [i.e., a true experiment with random assignment] provides experiment with random assignment] provides a clear enough causal interpretation to settle a clear enough causal interpretation to settle issues of social-scientific research issues of social-scientific research conclusively” (Shively, p. 87).conclusively” (Shively, p. 87).
1228 April 2009 @ Crawford School
LimitationsLimitations In both laboratory and field experimentsIn both laboratory and field experiments
Experimental research is often limited to Experimental research is often limited to investigations of political and social investigations of political and social communications.communications.
It may be ethically inappropriate to conduct It may be ethically inappropriate to conduct experiments with human beings, societies and experiments with human beings, societies and politics as subjects. politics as subjects.
In laboratory experimentsIn laboratory experiments Samples in an experiment may not represent Samples in an experiment may not represent
the population.the population. Experiments are conducted in an artificial Experiments are conducted in an artificial
setting.setting.
1328 April 2009 @ Crawford School
Quasi experimentQuasi experiment The basic quasi experimental designThe basic quasi experimental design
1.1. Measure the dependent variable of the Measure the dependent variable of the subjects.subjects.
2.2. Wait until some subjects are exposed to the Wait until some subjects are exposed to the independent variable, or observe that the independent variable, or observe that the values of the independent variable change values of the independent variable change among the subjects. (among the subjects. (NoteNote: The subjects : The subjects are are notnot assigned to groups: no random assigned to groups: no random assignment.)assignment.)
3.3. Measure the dependent variable again, and Measure the dependent variable again, and compare the difference in the dependent compare the difference in the dependent variable among the subjects.variable among the subjects.
1428 April 2009 @ Crawford School
1528 April 2009 @ Crawford School
Other examplesOther examples
““Panel” studies – the same set of Panel” studies – the same set of people are surveyed multiple times. people are surveyed multiple times.
““Before-and-after” studies in policy-Before-and-after” studies in policy-oriented research.oriented research. e.g., the effects of increasing speed limits e.g., the effects of increasing speed limits
on the number of traffic fatalities. (Not all on the number of traffic fatalities. (Not all states in the US increased the speed states in the US increased the speed limits.) limits.)
NoteNote: There should be a “control group” : There should be a “control group” in these research.in these research.
1628 April 2009 @ Crawford School
Merits and LimitationsMerits and Limitations They satisfy the “time-order” condition – They satisfy the “time-order” condition –
The value of a dependent variable The value of a dependent variable changeschanges after the value of a key independent after the value of a key independent variable is observed (or changed).variable is observed (or changed).
Important:Important: They can control “subject- They can control “subject-specific, time-invariant” variables, but not specific, time-invariant” variables, but not others.others.
You should carefully examine what other You should carefully examine what other factors might affect the causal relationship factors might affect the causal relationship and try to control them (if possible). and try to control them (if possible).
1728 April 2009 @ Crawford School
Correlational designCorrelational design
The basic correlational designThe basic correlational design1.1. Select a sample of your study.Select a sample of your study.
2.2. Measure the dependent and Measure the dependent and independent variables. (independent variables. (NoteNote: The : The values of the independent variables values of the independent variables must vary among the subjects.)must vary among the subjects.)
3.3. If the dependent variable differs If the dependent variable differs among the subjects, ascribe this to the among the subjects, ascribe this to the effect of the independent variable.effect of the independent variable.
1828 April 2009 @ Crawford School
1928 April 2009 @ Crawford School
Other examplesOther examples
Studies examining how survey respondents Studies examining how survey respondents with different attributes vote differently.with different attributes vote differently.
Studies examining how countries with Studies examining how countries with different economic conditions experience different economic conditions experience different environmental issues and conflicts.different environmental issues and conflicts.
Studies examining how Indonesian local Studies examining how Indonesian local governments with different social governments with different social conditions affect their budget performance.conditions affect their budget performance.
2028 April 2009 @ Crawford School
Merits and LimitationsMerits and Limitations The “time-order” condition is not always The “time-order” condition is not always
satisfied. We need “auxiliary” satisfied. We need “auxiliary” information (e.g., common sense, prior information (e.g., common sense, prior knowledge, etc.) to judge the direction knowledge, etc.) to judge the direction of causality.of causality.
A correlational design with just one A correlational design with just one independent variable fails to control independent variable fails to control other factors. other factors.
You should carefully examine what other You should carefully examine what other factors might affect the causal factors might affect the causal relationship and try to control them. relationship and try to control them.
2128 April 2009 @ Crawford School
Design without a control Design without a control groupgroup
A typical design without a control groupA typical design without a control group1.1. Measure a certain phenomenon or behavior Measure a certain phenomenon or behavior
(A) you want to explain. (A) you want to explain.
2.2. Observe a certain phenomenon, behavior or Observe a certain phenomenon, behavior or other exogenous shocks (B) that you think other exogenous shocks (B) that you think as a “cause”. as a “cause”.
3.3. Measure the phenomenon or behavior (A’) Measure the phenomenon or behavior (A’) you want to explain again. If the you want to explain again. If the phenomenon or behavior has changed from phenomenon or behavior has changed from A to A’, then ascribe this change to the A to A’, then ascribe this change to the occurrence of B. occurrence of B.
2228 April 2009 @ Crawford School
An ExampleAn Example
Taiwan presidential election in 2000.Taiwan presidential election in 2000. The pre-election level of support for each candidatThe pre-election level of support for each candidat
e.e. China’s military actions near Taiwan, which are intChina’s military actions near Taiwan, which are int
ended to affect the election result.ended to affect the election result. The results of the presidential electionThe results of the presidential election
The Democratic Progressive Party’s Chen Shui-bian wonThe Democratic Progressive Party’s Chen Shui-bian won The ruling Kuomingtang’s Lien Chan lost.The ruling Kuomingtang’s Lien Chan lost.
Many argue that China’s attempts were counterprMany argue that China’s attempts were counterproductive and helped Chen to win. Really? oductive and helped Chen to win. Really?
2328 April 2009 @ Crawford School
Control groupControl group
ImportantImportant: A control “group” and a : A control “group” and a control “variable” are different. If control “variable” are different. If there is no control group, your there is no control group, your independent variable does not independent variable does not varyvary. . A control group = a portion of your A control group = a portion of your
observations (the subjects of your study) observations (the subjects of your study) that are not exposed to your key that are not exposed to your key independent variable.independent variable.
A control variable = another factor A control variable = another factor explaining the dependent variable.explaining the dependent variable.
2428 April 2009 @ Crawford School
ExamplesExamples
An example of true experiments: An example of true experiments: Taking an introductory Australian Government Taking an introductory Australian Government
course (course (XX) increases political interest () increases political interest (YY). ). “Students who do not take the course.”“Students who do not take the course.”
An example of natural experiments: An example of natural experiments: Watching a presidential debate (Watching a presidential debate (XX) increases ) increases
intensity of support (intensity of support (YY). ). “Students who do “Students who do not watch the debate.”not watch the debate.”
An example of correlational designs: An example of correlational designs: Voter turnout (Voter turnout (YY) is lower in urban areas () is lower in urban areas (XX). ).
“Rural areas.” “Rural areas.”
2528 April 2009 @ Crawford School
LimitationsLimitations
Research design without a control group is very Research design without a control group is very common in historical descriptions of political, common in historical descriptions of political, social and economic events. This method may social and economic events. This method may be useful to understand historical processes be useful to understand historical processes and details, but it is and details, but it is notnot recommended for recommended for causal analysis. Why? Because without a causal analysis. Why? Because without a control group you cannot say that a variation in control group you cannot say that a variation in YY is caused by a variation in is caused by a variation in XX (because there is (because there is no variation in no variation in XX). ).
They may, however, help us identify some new They may, however, help us identify some new measures, theories, and/or puzzles.measures, theories, and/or puzzles.
2628 April 2009 @ Crawford School
SummarySummary
Random Random AssignmeAssignme
ntnt
Pre-Pre-measuremmeasurem
entent
Control Control GroupGroup
True True experimental experimental designdesign
YesYes Yes/NoYes/No YesYes
Quasi Quasi experimental experimental designdesign
NoNo YesYes YesYes
Correlational Correlational designdesign
NoNo NoNo YesYes
Design without a Design without a control groupcontrol group
NoNo YesYes NoNo
2728 April 2009 @ Crawford School
Next weekNext week
Intentional selection of observationsIntentional selection of observations How to choose observations?How to choose observations? How to avoid problematic causal How to avoid problematic causal
inference?inference? Some controversiesSome controversies
Observations vs. casesObservations vs. cases Objectives of qualitative researchObjectives of qualitative research
Top Related