Research Design

RCPT 436 Research & RCPT 436 Research & Technology Technology ApplicationsApplications

Units of Analysis

Individuals: students, participants, patients

Groups: classes, families, gangs Organizations: universities,

churches, recreation departments Social Artifacts: books, paintings,

songs, editorials, buildings

A word of warning…

On average, students from Springfield score higher on the SATs then students from Shelbyville. What can we conclude about Lisa Simpson?

NOTHING! An Ecological Fallacy results when you make conclusions about individuals based upon aggregate data.

Variables

Definition: discrete phenomenon that can be observed in at least two mutually-exclusive categories– Dependent variable– Independent variables– Intervening variables

Watching a Snickers Commercial

Buying a Snickers from Vending Machine

Hunger

Dependent Variable

Intervening Variable

Independent Variable

What other (independent) variables could explain our purchase of a Snickers?

Defining the scope

Purpose statement Significance statement Theory or conceptual roadmap

Which Hypothesis is Which?

Hypothesis– Null hypothesis– Non-directional hypothesis– Alternative hypothesis

Men will perform better than women on standardized tests.

The average scores for men and women will be different.

There will be no statistical difference between test scores

Internal Validity

Determine whether a given program is working, if it meets standards, or how it compares to other programs

Properly demonstrates a causal relationship between two variables

External Validity

Ability to generalize findings to the “real world” or beyond the sample

Results should be reproducible across different experimental settings

Threats to External Validity

1. Selection bias2. Experimental settings3. Testing4. Multiple treatment interference5. Inadequate operational definition6. Hawthorne effect7. Selection interacting with any

extraneous variables

Experimental Design

1. Random selection of sample2. Pretest3. Random assignment to

experimental and control groups4. Post-test

Variations

Quasi-experimental design – study participants not randomly assigned to control or experimental group

Pre-experimental – no control group

Non-experimental – individuals are observed or surveyed

Qualitative Designs Ethnography: holistic understanding

(participant observation) Case study: in depth study of a single

phenomenon (visionary) Content analysis: written, visual, or

recorded documents (profanity on TV) Historical study: studying the past

(primary or secondary data; be skeptical)

Rigor determined by truthfulness, applicability, and consistency