Single-Case Research: Standards for Design and Analysis Thomas R. Kratochwill University of...
-
Upload
nigel-west -
Category
Documents
-
view
217 -
download
0
Transcript of Single-Case Research: Standards for Design and Analysis Thomas R. Kratochwill University of...
Tom Kratochwill, ChairJohn Hitchcock
Rob Horner Sam Odom
David RindskopfWill Shadish
Joel Levin, Consultant
An individual “case” is the unit of intervention administration and data analysis. A case may be a single participant or a cluster of participants (e.g., a classroom or community).
Within the design, the case provides its own control for purposes of comparison. For example, the case’s series of outcome variables prior to the intervention is compared with the series of outcome variables during (and after) the intervention.
The outcome variable is measured repeatedly within and across different conditions or levels of the independent variable. These different conditions are referred to as “phases” (e.g., baseline phase, intervention phase).
Dep
ende
nt V
aria
ble
Baseline(A1 Phase)
Intervention(B1 Phase)
Baseline(A2)
Intervention(B2)
Days/Weeks/Months/Sessions
Baseline Interventions 1 and 2
Days/Sessions
Dep
ende
nt V
aria
ble
Intervention 1
Interv. 1 =
Interv. 2 =
Par
tici
pant
1P
arti
cipa
nt 2
Par
tici
pant
3P
arti
cipa
nt 4
Dep
ende
nt V
aria
ble
Baseline Intervention
Days/Weeks/Months/Sessions
Ambiguous Temporal PrecedenceSelectionHistoryMaturationTestingInstrumentationAdditive and Interactive Effects of Threats
Overarching: Which intervention is effective for this case?
Is this intervention more effective than the current “baseline” or “treatment” as “usual” condition? (e.g., does Intervention A reduce problem behavior for this case?)
Does adding B to Intervention A further reduce problem behavior for this case?
Is Intervention B or Intervention C more effective in reducing problem behavior for this case?
Evaluate the Design
Meets Evidence Standards
Meets Evidence Standards with Reservations
Does Not Meet Evidence Standards
Conduct Visual Analysis for Each Outcome Variable
Strong Evidence Moderate Evidence No Evidence
Effect-Size Estimation
Independent variable must be systematically manipulated
The outcome variable must be measured systematically
The study must include at least three attempts to demonstrate an intervention effect (replication)
The phase should typically include a minimum of five data points
Measurement occurs over time Inter-observer agreement is reported Inter-observer agreement must be assessed
on each outcome variable in every phase and there should be measurement for at least 20% of the sessions distributed across all conditions of the study
Designs that generally meet this standard include:
ABAB Design Multiple Baseline Design Alternating Intervention Design
Designs not meeting this standard include:
AB Design ABA Design BAB Design
0
10
20
30
40
50
60
70
80
90
Pro
po
rtio
n o
f 10
Sec
on
d In
terv
als
Aca
dem
ical
ly E
ng
aged
DaysFirst Demonstration of Ef fect
Second Demonstration of Ef fect
Third Demonstration of Ef fect
Baseline1A1
Intervention1(Peer Tutoring)
B1 Baseline2A2
Intervention2(Peer Tutoring)
B2
Adam
Source: Horner & Spaulding, in press
ABAB Design
Pro
po
rtio
n o
f In
terv
als
wit
h P
rob
lem
Beh
av
ior
0
20
40
60
80
1
Baseline Intervention
Days
Anna
0
20
40
60
80
Amy3
0
20
40
60
80
Fred4
0
20
40
60
80
2
Dillion
Multiple Baseline Design
Source: Horner & Spaulding, in press
Exceptions: If an ABAB or Multiple Baseline Design study has fewer than three or four data points in any one phase used to demonstrate an effect, the study may Meet Evidence Standards with Reservations
Evidence Standards Met Through Visual Analysis of Single-Case Research Data Displays
WWC Reviewers Trained in Visual Analysis of Data in Single-Case Design
Single-Case Design Visual Analysis: Training Goals
Define Six Variables used in visual analysis, and build fluency in applying those variables with attention to both main and interaction effects.
Provide a Four-Step Framework for analysis of single-case designs
Visual analysis Statistical analysis
Apply visual analysis to ABAB, Multiple Baseline Designs, and AlternatingTreatment designs.
Documenting Experimental Control◦ Three demonstrations of an “effect” at three
different points in time. A “basic effect” is a change in the dependent
variable when the independent variable is actively manipulated.
◦ To assess an “effect” Visual Analysis includes simultaneous assessment of: Level, Trend, Variability, Immediacy of Effect,
Overlap across Adjacent Phases, Consistency of Data Pattern in Similar Phases.
(Parsonson & Baer, 1978, 1992; Kratochwill & Levin, 1992)
Interpreting experimental control always involves assessment of data from the whole study (all phases), not just assessment of two adjacent phases.◦ Assessment of a “basic effect” is done with
adjacent phases.◦ Assessment of experimental control, however,
requires evaluation of all data in all phases.
Four Steps in AnalysisSix Variables for Consideration
Do Baseline data document a predictable pattern?
Do data within each phase allow documentation of a predictable pattern?
Do data between phases document basic effects?
Do data across phases document experimental control?
Level
Trend
Variability
Overlap
Immediacy of effect
Consistency across similar phases
Level Trend Variability Overlap Immediacy of Effect Consistency across similar phases Stability in non-intervened series when
effect demonstrated in one series
Magnitude of separation Greater the difference between two conditions, larger the
demonstration of a functional relation
Consistency of separation Greater consistency of separation between two conditions
(no overlap) larger the demonstration of a functional relation
Number of data points used to establish separation
The more points documenting separation to larger the demonstration of a functional relation.
The Tradition of Applied Behavior Analysis Lack of Consensus Surrounding the
Statistical Analysis of Single-Case Research Design
Use of Visual Analysis in Single-Case Design in Practice Settings
Structured Training in Visual Analysis (e.g., compare visual analysis of novices to experts)
Use Visual Analysis Protocol that Includes a Component Analysis and Judgmental Aids (e.g., Tawney & Gast, 1984)
Use Visual Analysis Criteria (e.g., Dual Criterion Method and Conservative Dual Criterion Method; Fisher, Kelly & Lomas, 2003; Swaboda, Kratochwill, & Levin, 2009)
Use of Randomization in Design [Response-Guided versus Non-Response-Guided Experimentation (e.g., Ferron & Jones, 2006; Todman & Dugard, 1999, 2001)]
Blind Visual Analysis Procedures from a “Data Analyst” (Ferron & Jones, 2006)
Use Both Visual and Statistical Analysis (e.g., Borckhardt, Nash, Murphy, Moore, Shaw, & Oneil, 2008; Brossart, Parker, Olson, & Mahadevan, 2006; Ferron & Jones, 2006;..among others)
Randomization applied to Single-Case Design Structure
Statistical Analysis of Single-Case Design to Determine Statistical Significance (e.g., randomization tests, time-series analysis, HLM )
Single-Case Design Effect Size Determination
Meta-Analysis (Single-Case Design Studies or Combined with Group Design Research)
Thomas R. Kratochwill, PhDEducational and Psychological Training
Center1025 West Johnson StreetUniversity of Wisconsin-MadisonMadison, Wisconsin 53706
E-Mail: [email protected]