Quasi Experimental and single case

experimental designs

Chapter 11

Program Evaluation

Research on programs that are proposed and implemented to achieve some positive effect on a group of people

i.e. DARE program evaluation Applies research approaches to evaluate

types of programs

Program Evaluation

Types Needs assessment

What problems, if any, need to be addressed? Once identified researchers can plan accordingly

Assessment of program theory Used to fix problems found in a needs assessment Involves collaboration of service providers, researchers,

clientele, etc… Process evaluation

Program monitoring to determine if program is doing what it is supposed to Is the target population being reached? Are the research questions being answered?

Program Evaluation

Types Outcome evaluation

Impact assessment What did the participants like/dislike about the study? Studying the outcome and then assess the impact of the

outcome measure. Efficiency assessment

Was the program worth it? Did the benefits outweigh the costs?

Uses quasi-experimental designs to assess effectiveness and efficiency

Quasi experimental designs

1 group posttest only design Has no comparison group

Doesn’t have good internal validity 1 group pretest-posttest design

History Could confound results

Maturation People change over time and this could skew results

Testing Previous experience could change behavior

Instrument decay the deterioration of research instruments or methods during

a study

Quasi experimental designs

Instrument decay equipment may wear out, respondents may become more

casual in recording their responses. Regression toward the mean

a.k.a statistical regression The tendency for extreme scores in a distribution to move

(regress) toward the mean of the distribution w/ repeated testing.

Problem of reliability Occurs when a set of extreme scores collected at one time

point is compared to scores taken at another time point.

Quasi experimental designs Nonequivalent control group design

A separate control group Selection differences – choosing groups based on naturally

existing categories; this is why study is called nonequivalent Nonequivalent control group pretest-posttest design

Benefit: a pretest for comparison purposes Interrupted time series design

Measurement of dependent variable is interrupted by the quasi independent variable

By taking multiple measurements you can look at the effectiveness of a treatment before and after it is introduced.

Control series design Extension of the interrupted time series design with a control

group

Quasi experimental designs 1 group posttest only design

E X O 1 group pretest-posttest design

E O X O Nonequivalent control group design

E X O C O

Nonequivalent control group pretest-posttest design E O X O C O O

Interrupted time series design E O O O X O O O

Control series design E O O O X O O O C O O O O O O

E = experimental groupX = treatmentC = control groupO = observation to collect data

Single case experimental designs

A.K.A. Single subject designs or Single participant designs

Baseline control period treatment period The change in behavior during these periods could be

attributed to a variety of things Reversal designs

To determine if treatment had an effect, revert back to baseline (no treatment) and look at the differences

ABA design Baseline control period treatment period control

period Can also do a ABAB design

Single case experimental designs

Multiple baseline designs Observing behavior before and after a

manipulation under multiple circumstances Can be done

Across subjects Across behaviors Across situations