Chapter 3The Research Design

Research Design

• A research design is a plan of action for executing a research project, specifying• The theory to be tested

• The unit of analysis

• The necessary observable data

• Data-collection procedures

• Analytical procedures

Research Design

• The choice of research design is affected by• The purpose of the research

• Resources like time, money, and skill or ethical concerns

• Every research design has positive and negative attributes.

• You should match a design with your requirements.

Research Design

• All research designs attempt to• Establish a relationship between two or more variables

• Demonstrate that the results are generally true

• Establish whether one phenomenon precedes another in time

• Eliminate as many alternative explanations as possible

Research Design• The goal of scientific research is to find causal relationships.

• Causal relationships have three characteristics:• Covariation: the alleged cause varies with the supposed effect

• Time order: the cause precedes the effect in time

• Elimination of alternative explanations to isolate causation to one factor

• Need to avoid spurious relationships (hospital example)

Research Design• When choosing a research design, it is also important to

consider• Internal validity:• Refers to a causal relationship that was not created by a spurious

relationship (a relationship in which a second independent variable influenced the dependent variable)

• Effects: history, maturation, testing, selection biases, experimental mortality, instrument decay, demand characteristics

• External validity: • Refers to the extent to which the results of an experiment can be

generalized across populations, time, and settings

Experimental Design

• Experimental research designs are especially good for isolating causal factors.

• Experimentation allows a researcher to make causal inferences with great confidence in the design through control over exposure to an experimental treatment.

• But, although experiments have great internal validity, they suffer from weaker external validity.

Experimental DesignThe classical randomized experiment has five basic

characteristics:1. At least one experimental group that will have exposure to the

treatment and one control group that will not2. Randomly assigned individuals to each group, avoiding self-

selection3. Controlled administration of the treatment, including the

circumstances under which the experimental group is exposed 4. Measurement of a dependent variable before and after the

treatment with a pre-test and a post-test; any difference between the tests can be attributed to the experimental effect of exposure to the treatment

5. Controlled environment of the experiment (time, location, and other physical aspects)

Experimental Design

• Post-test design:• Shares the characteristics of the classical randomized experiment,

except that no pre-test is used because the sample is truly random and sufficiently large that one can assume that the control and experimental group(s) are equivalent

Experimental Design

• Repeated-measurement design:• Adds to the classic example additional pre-tests, post-tests, or

both in an effort to measure longer-term effects of experimental treatments

Experimental Design

• Multigroup design:• A modification of the classic example in which more than one

experimental group is created to compare the effects of different treatments

Experimental Design

• Field experiment:• An experiment in a natural setting in which the investigator does

not have control over group membership but does have control over one or more independent variables

• Causal inferences made using this design are not as strong—but may be more practical for some situations

Nonexperimental Design

• Nonexperimental designs are characterized by at least one of the following:– Presence of a single group

– Lack of control over the assignment of subjects to groups

– Lack of control over the application of the independent variable

– Inability to measure the dependent variable before and after exposure to the independent variable occurs

Nonexperimental Design

• Small-N designs:• Also called case studies or comparative cases studies

• Involve rich, deep understanding of a small number of cases

• May be used for exploratory, descriptive, or explanatory purposes

Nonexperimental Design

• Focus groups:• Can be used to create hypotheses for testing through other

research designs

• Generally not used to establish causal relationships

Nonexperimental Design

• Cross-sectional designs (survey, aggregate analysis):• Characterized by measurements of the independent and dependent

variables at approximately the same time

• Data analysis, rather than a treatment, is necessary for making causal inferences

Nonexperimental Design

• Longitudinal designs:• Allow for the measurement of variables at different points in time

• Can model change across time; examine the time order of a causal relationship; and estimate age, cohort, and period effects

Nonexperimental Design

• Trend analysis:• Analysis of variables measured across periods of generally 20

years or more with a focus on explaining change over time

Nonexperimental Design

• Panel studies:• Cross-sectional designs that include a time element

• Rely on measurement of the same units of analysis at different points in time—creating waves of data for analysis over time

• Panel mortality

Nonexperimental Design

• Intervention analysis:• Measurements of a dependent variable before and after the

introduction of an independent variable that is observed but not controlled by the researcher

Nonexperimental Design

• Nonexperimental designs are generally characterized as having less internal reliability but better external validity than experimental designs.

• There is always a tradeoff when moving from one design to another.