Received: 9 August 2018 Revised: 27 March 2019 Accepted: 28 March 2019

DOI: 10.1002/bin.1669

R E S E A R CH AR T I C L E

Using equivalence‐based instruction to teach thevisual analysis of graphs

Bryan J. Blair1 | Jonathan Tarbox2 | Leif Albright3 |

Jacquelyn M. MacDonald4 | Lesley A. Shawler1 | Samantha R. Russo1 |

Michael F. Dorsey5

1The Van Loan School, Institute for Behavioral

Studies, Endicott College, Beverly,

Massachusetts

2College of Letters, Arts, and Sciences,

Department of Psychology, University of

Southern California, Los Angeles, California

3Department of Applied Behavior Analysis,

Caldwell University, Caldwell, New Jersey

4School of Health Sciences, Regis College,

Weston, Massachusetts

5Amego, Inc, Attleboro, Massachusetts

Correspondence

Bryan J. Blair, Ph.D., BCBA‐D, Library Learning

Center, Room 228, 1 University Plaza,

Brooklyn, NY 11201.

Email: bryan.blair@liu.edu

This manuscript was submitted to the Institute for Behavior

the seventh author in partial fulfillment of the requirements

ical and insightful feedback. Additionally, the authors thank

their assistance. The first author is now an Assistant Profe

Behavioral Interventions. 2019;1–14. wiley

Previous research has demonstrated low rater agreement

when visually inspecting trends in single‐subject design

graphs (see Wolfe, Seaman, & Drasgow, 2016). Didactic

instruction of visual analysis of practicing behavior analysts

has resulted in generally poor and unreliable performances

(Danov & Symons, 2008; Diller, Barry, & Gelino, 2016).

Therefore, a refined instructional technology to improve

the reliability among behavior analysts is warranted. Devel-

oping research has focused on the application of

equivalence‐based instruction (Brodsky & Fienup, 2018;

Rehfeldt, 2011) for a variety of complex human behaviors.

In the current study, equivalence‐based instruction was

used to train four participants to identify functional rela-

tions displayed in five different classes of graphs. Training

resulted in the formation of five equivalence classes by all

participants consisting of three members (graph, functional

relation rule, and functional relation statement). In addition,

the skills were maintained for up to 2 weeks and general-

ized to novel graphs.

KEYWORDS

match‐to‐sample, multiple‐exemplar training, stimulus equivalence,

visual analysis

al Studies in the Van Loan School at Endicott College by the first author under the supervision of

for the doctoral degree in applied behavior analysis. The authors thank Caio Miguel for his crit-

Mark Silberman, Elizabeth Kirby, staff at Hopeful Journeys, and faculty at Caldwell University for

ssor at Long Island University ‐ Brooklyn.

© 2019 John Wiley & Sons, Ltd.onlinelibrary.com/journal/bin 1

2 BLAIR ET AL.

1 | INTRODUCTION

One of the fundamental qualities of behavior analysis is its use of visual analysis of data. Unlike other fields (e.g., psy-

chology) that rely on statistical comparisons and the standardization of collected data, behavior analysts rely almost

exclusively on the visual inspection of summarized and plotted data over time. An extensive line of research has sug-

gested that teaching visual analysis of graphs using standard didactic approaches and traditional supervisory methods

might not lead to the development of fluent and reliable visual analytic repertoires in practicing behavior analysts

(Danov & Symons, 2008; DeProspero & Cohen, 1979; Diller et al., 2016; Ninci, Vannest, Willson, & Zhang, 2015;

Ottenbacher, 1993; Wolfe et al., 2016). However, a few studies have shown that when participants are directly

taught specific visual analysis rules, agreement across participants improved (Hagopian et al., 1997; Roane, Fisher,

Kelley, Mevers, & Bouxsein, 2013).

The essential skill for behavior analysts, specifically, is to identify whether graphically presented data demonstrate

a functional relation between the independent and dependent variables. This skill is necessary to determine whether

interventions are effective, and in some cases, to identify the specific components of interventions that are control-

ling the target behavior(s). However, few studies have investigated different instructional methods for teaching stu-

dents and practitioners to reliably visually analyze graphs to determine functional relations between independent and

dependent variables in more common behavior analytic experimental designs (Wolfe et al., 2016).

One promising instructional framework that focuses on the direct teaching of a few specific skills and the emer-

gence of novel responding is the application of derived stimulus relations methodologies (Hayes, Barnes‐Holmes, &

Roche, 2001) in general, of which stimulus equivalence (Sidman, 1994) appears to be an example. In several studies,

participants were exposed to direct teaching methods with multiple exemplars, resulting in the emergence of rela-

tional responding across an array of stimulus parameters, participants, training settings, and instructional protocols

(Hayes et al., 2001; Sidman, 1994).

When stimuli that share no topographical similarities become part of the same class of stimuli, an equivalence

class has formed (Green & Saunders, 1998; Sidman, 1994; Sidman & Tailby, 1982). In order to demonstrate that

an equivalence class has emerged, the properties of reflexivity (A = A), symmetry (if A = B then B = A), and transitivity

(if A = B and B = C then A = C) must be demonstrated among the stimuli that form the class. For an equivalence class

to form, a learner must consistently select a member of an equivalence class when presented with other members of

the same class. Conditional discrimination training, specifically match‐to‐sample (MTS) training, is a common method

of establishing equivalence classes. There have been a number of published studies on the establishment of stimulus

equivalence (see Sidman, 1994; Stromer, Mackay, & Stoddard, 1992, for thorough reviews). Additionally, a growing

number of investigations have been conducted using equivalence‐based instruction (EBI) to teach standard

college‐level content to typically developing adults (Brodsky & Fienup, 2018; Rehfeldt, 2011).

In a series of studies (Critchfield & Fienup, 2010; Fields et al., 2009; Fienup, Covey, & Critchfield, 2010; Fienup &

Critchfield, 2010), EBI was used to teach college student complex skills such as those required for interpreting statis-

tics, and for relating facts about brain anatomy. The studies used computer‐based training to teach and test the rela-

tions involved in the equivalence classes. In all studies, derived relational responses emerged and generalized to novel

exemplars. In another study that used EBI procedures, Walker and Rehfeldt (2012) taught graduate students in a

distance‐learning program to identify single‐subject design graphs via a computer‐based system. Participants learned

to select the definitions of research designs and graphs when presented with the names, and to select vignettes when

presented with definitions of designs. After training, participants demonstrated several emergent selection‐based and

topographical responses including selecting and writing the names of designs when presented with graphs and

vignettes, and selecting the definitions of designs when presented with vignettes. The results of the study suggested

that with direct training of specific discriminative responses, derived relational responses emerged. The authors con-

cluded that EBI was an efficient method for teaching higher‐level concepts to college students with minimal instruc-

tion (Walker & Rehfeldt, 2012). However, one limitation noted was that the training did not result in acceptable

generalization to novel stimuli for all participants.

BLAIR ET AL. 3

Two studies that used EBI to teach academic skills were recently published (Albright, Reeve, Reeve, & Kisamore,

2015; Albright, Schnell, Reeve, & Sidener, 2016). In both studies, the investigators used EBI to teach complex skills

such as the concepts of statistical variability and the interpretation of the results of functional analysis graphs to col-

lege students. Custom computer software deliveredMTS trials in the experimental sessions in a pretest‐train‐posttest‐

maintenance design. They incorporated multiple‐exemplar training (MET) in order to promote generalization. In the

first study, participants learned to select statistical category definitions, sample numbers, and standard deviation def-

initions when presented with category names. In the second study, participants learned to select graph descriptions

and graphs when presented with the function and to select a vignette when presented with a graph description. Both

studies resulted in substantial increases in scores from pretest to posttest conditions and equivalence classes formed

for all participants. Results also suggested that skills were maintained at 1 week, and generalized to novel stimuli.

The purpose of the current study was to systematically replicate and extend studies on EBI (e.g., Albright et al.,

2016; Walker & Rehfeldt, 2012) with an eminently applied skill for behavior analysts. In recent studies, EBI has been

shown to be effective in teaching relevant academic skills to college students and the current study further extends

this line of research. The specific purpose of the current study was to investigate whether a visual–visual MTS pro-

cedure with MET, in which conditional discriminations between graphs and analytic rules, and between analytic rules

and functional relation statements, were taught, resulted in the formation of equivalence classes based on the level of

functional relation, including the emergence of derived relational responding in the form of selecting a functional rela-

tion statement when presented with an ABAB design graph.

2 | METHOD

2.1 | Participants

Two undergraduate students and two direct care staff who worked at an agency that provides services to children

diagnosed with autism spectrum disorder, participated in the study. We recruited participants via direct email, posts

on an online course site, and posts on social media. The four participants (one man and three women) ranged in age

from 19‐to 25‐years‐old. A screening tool was used to determine the participants' level of experience with, and expo-

sure to, the visual analysis of graphs. No participants had any experience with studying or implementing single‐

subject designs. The Endicott College Institutional Review Board approved this research project. Prior to participa-

tion, all participants signed a consent form and they received a gift card for participating in the study, regardless of

whether the participant completed all parts of the training and testing.

2.2 | Setting and materials

Experimental sessions were conducted in a small student lounge or a conference room with desktop or laptop com-

puters. For all sessions, we used the same custom web‐based software that Albright et al. (2016) developed specif-

ically for use in EBI research to design the EBI system and to deliver and control the MTS training and testing

sessions (the software is not commercially available). The developers of the EBI training software designed the sys-

tem to automatically present visual–visual MTS training and testing trials and conditions to the participant via a web

browser based on the mastery criteria and in the order that was programmed by the researcher. In addition, the soft-

ware recorded participants' responses as data in spreadsheet table form.

2.3 | Experimental stimuli, training protocol, and training structure

The visual stimuli (see Table 1 and Figure 1) used were graphs (denoted as A), written visual analysis rule statements

(denoted as B), and written functional relation statements (denoted as C), displayed on a computer screen. Five

TABLE

1Exp

erim

entalstim

uli

Class

Labe

lNotation

12

34

5

Graph

Ax

Ay

Az

Rule

BThe

reisaclea

rch

ange

in

leve

land

tren

d

betw

eenba

selin

ean

d

trea

tmen

tco

nditions

The

reisach

ange

inleve

l

betw

eenba

selin

ean

d

trea

tmen

tco

nditions

andthe

tren

disvariab

leacross

cond

itions

The

tren

dsin

allc

ond

itions

are

toovariab

leto

makea

determ

inationofwhe

ther

thereisafunc

tiona

lrelation

Thereisaslightch

ange

inleve

l

betwee

nbaselinean

dtrea

tmen

t

conditionsan

dthetren

dis

variab

leacross

conditions

Thereisnoch

ange

in

leve

lortren

d

betwee

nbaseline

andtrea

tmen

t

conditions

Statem

ent

CFun

ctiona

lrelation

Proba

blefunc

tiona

lrelation

Unc

lear

Probab

lynofunctional

relation

Nofunctional

relation

4 BLAIR ET AL.

FIGURE 1 Trained (solid arrow) and tested (dotted arrow) relations

BLAIR ET AL. 5

equivalence classes (1. Functional Relation, 2. Probable Functional Relation, 3. Unclear, 4. Probably No Functional

Relation, and 5. No Functional Relation) were established. The graphs were 30 ABAB figures with varying behaviors,

levels, trends, and numbers of data points per condition and were created for this study and did not represent actual

clinical or research data. Fifteen graphs were used during training, and 15 novel exemplars were used for the gener-

alization probes. We chose ABAB because, “they are the most straightforward and powerful within‐subject design for

demonstrating functional relations between environmental manipulations and a behavior” (Cooper, Heron, & Heward,

2007, p. 177). To program for generalization, we included three exemplars of each graph (denoted as Ax Ay, and Az)

for each equivalence class. Prior to the study, three PhD level BCBAs (BCBA‐Ds) reliably identified the graphs by

class. For the functional relation rule (B) and functional relation statement (C) stimuli, one stimulus was included in

each class. During all sessions, all stimuli were presented as black‐and‐white images on a computer screen. The size

of the stimuli that the software displayed depended on the computer monitor size and resolution; however, for this

study, all stimuli were approximately 7.5 cm × 7.5 cm.

We used a simple‐to‐complex training protocol and linear training structure (Table 2 and Figure 2) for this exper-

iment (Arntzen, 2012; Saunders & Green, 1999). We designed the system to teach the AB baseline discriminations,

then present the BA symmetry test, then teach the BC baseline discriminations, then present the CB symmetry test.

Finally, the system presented the AC transitivity test followed by the CA equivalence test. The system automatically

presented training and testing conditions based on the programmed mastery criteria for each condition.

2.4 | Experimental design and dependent variables

We used a single‐session pretest‐posttest design, embedded in a two‐tier nonconcurrent multiple baseline across

participants design (Carr, 2005; Christ, Cooper, Heron, & Heward, 2007; Watson & Workman, 1981) to examine

experimental control and assess the effects of conditional discrimination training in the form of MTS on the forma-

tion of five three‐member classes. We exposed Participants 2 and 4 to two pretests, conducted in two successive

conditions in a single session, to control for testing effects (Griffith, Ramos, Hill, & Miguel, 2018). Finally, we con-

ducted maintenance and generalization probes for all participants. The primary dependent variable was the percent-

age of correct trials for all relations, particularly transitivity and equivalence relations, during testing conditions. We

also calculated the trials‐to‐criterion for all participants for AB and BC training conditions, and training and testing

durations.

TABLE 2 Training and testing sequence and parameters

Condition Relations trained or tested Trials per block Mastery criterion

Interface Training N/A N/A N/A

Pretest AB, BA, AC, CA 60 N/A

BC, CB 10 N/A

AB Train AB 15 15/15 in one block

BA Test BA 15 14/15 in one block

BC Train BC 5 5/5 in one block

CB Test CB 5 5/5 in one block

AC Test AC 15 14/15 in one block

CA Test CA 15 14/15 in one block

Posttest AB, BA, AC, CA 60 N/A

BC, CB 10 N/A

Generalization AC 15 N/A

Maintenance AB, BA, AC, CA 60 N/A

BC, CB 10 N/A

FIGURE 2 Training protocol (simple‐to‐complex) and structure (linear)

6 BLAIR ET AL.

2.5 | Procedural integrity and interobserver agreement

The researcher used a written checklist to ensure that the correct conditions were conducted with the correct

participants and the researcher read instructions to ensure that all participants were provided with the appropriate

BLAIR ET AL. 7

instructions for each condition. Primary and secondary observers assessed procedural integrity via recorded video

in 33% of experimental sessions. According to the primary observer, the researcher completed 100% of the steps

correctly. We calculated interobserver agreement for procedural integrity by dividing the total number of steps

with observer agreement by the total number of possible steps and multiplying by 100. Interobserver agreement

was 100%.

2.6 | Procedure

2.6.1 | Visual analysis social validity

Prior to the start of the study, 12 Master's level Board Certified Behavior Analysts® (BCBAs) enrolled in a PhD pro-

gram in ABA were asked to assess the level of functional relation for 30 ABAB graphs.

2.6.2 | Visual–visual MTS training and testing system

Prior to the start of a block of trials the system presented the following instructions to participants, “Select the stim-

ulus that matches the sample stimulus. During this block you will be immediately notified if your selection was correct

or incorrect” or, “During this block you will not be notified if your selection was correct or incorrect.” After clicking

“OK,” the system presented the first trial.

A trial consisted of the presentation of a visual sample stimulus at the top of the computer screen, the display of

the total number of trials in the training/testing block, the number of trials remaining, and a progress indicator. When

the participant clicked the sample stimulus (e.g., A1) with a computer mouse, the system presented five visual com-

parison stimuli on the screen (one correct member (e.g., B1) of the class and four incorrect members, one from each

of the other classes (e.g., B2, B3, B4, and B5) in an array (randomly located) at the bottom of the screen. Trial‐and‐

error instruction was used for the MTS training conditions. During training trials, clicking on the comparison stimulus

resulted in immediate feedback (“Correct” or “Incorrect”). If a participant selected the correct stimulus, it was identi-

fied as correct, and when the participant clicked anywhere on the screen the system presented the next trial; how-

ever, if a participant selected an incorrect stimulus, the correct stimulus was not identified and the next trial was

presented. During testing trials, clicking on any comparison stimulus resulted in no feedback and the next trial was

immediately presented. Prior to the start of sessions, participants completed a simple computer‐based visual–visual

MTS task (color name‐to‐color) in order to familiarize themselves with the computer software and matching task.

All participants responded correctly for all interface training trials.

2.6.3 | Pretest

Immediately following the interface training, the researcher initiated the pretest condition. The pretest condition ran-

domly presented all possible conditional discriminations and exemplars (graph‐to‐functional relation rule [AB], func-

tional relation rule‐to‐functional relation statement [BC], functional relation rule‐to‐graph [BA], functional relation

statement‐to‐functional relation rule [CB], graph‐to‐functional relation statement [AC], and functional relation

statement‐to‐graph [CA]) as individual trials one time in a single block of trials. At the end of the pretest, the

researcher immediately started the MTS training and testing session.

2.6.4 | MTS training and symmetry tests

During the baseline training conditions (AB and BC), the system trained participants to select the correct comparison

stimulus from an array when presented with a sample stimulus. Trials were presented in blocks until participants met

the mastery criterion (100% correct for one block). Immediately following a successful baseline training condition, the

8 BLAIR ET AL.

system automatically presented participants with the corresponding symmetry (BA or CB) test. If a participant failed a

symmetry test, the corresponding baseline relation training condition was presented again.

2.6.5 | Transitivity and equivalence probes

Immediately following successful symmetry tests, the system presented participants with transitivity (AC) and equiv-

alence (CA) probes. If a participant failed either these tests, the AB training condition was presented again and the

participant proceeded through the training and testing system until mastery criteria for all conditions were met

(Figure 2).

2.6.6 | Posttest

Immediately following a successful equivalence (CA) test, the researcher initiated the posttest. The posttest condition

was identical to the pretest condition.

2.6.7 | Generalization

We initially conducted generalization probes with BAB reversal graphs with Participants 1 and 3 immediately follow-

ing the posttest (data not shown). However, due to a lower than expected performance for Participant 3, we also

conducted generalization probes with ABAB reversal graphs with all four participants. Due to this change in method

during the study, we conducted generalization probes after maintenance probes for Participants 1 and 3 and prior to

maintenance probes for Participants 2 and 4. Probes were identical to the AC test condition except with novel

graphs.

2.6.8 | Maintenance

All participants completed maintenance probes an average of 15 days (range, 14 to 17) after the training sessions.

The maintenance test was identical to the pretest and posttest conditions.

2.6.9 | Satisfaction survey

At the conclusion of the study, all participants voluntarily completed a satisfaction survey (see Table 3).

TABLE 3 Satisfaction survey results

Question Mean Range (1–5)

Prior to this training, how confident were you in your ability to visually analyze graphs? 1.5 1–2

After this training, how confident are you in your ability to visually analyze graphs? 3.5 3–4

The computer‐based training helped me learn about visually analyzing graphs. 4.0 4–4

I felt successful during the training and testing. 4.0 3–5

Compared to other instructional methods (e.g., lecture, reading a textbook,

and study guide questions), rate your preference for this type of instruction.

2.8 2–4

The computer‐based training frustrated me. 3.3 2–4

I would use this type of instruction to learn other concepts and to help me master other material. 3.3 3–4

I would recommend this type of instruction to others. 3.3 2–4

BLAIR ET AL. 9

3 | RESULTS

The mean score for the visual analysis social validity test that was completed by 12 Master's level BCBAs enrolled in

a PhD program in ABA was 73% (range, 60% to 90%). Participant scores for the pretests were below 60% (range,

35% to 57%) for all four participants (see Figures 3 and 4). Both Participants 2 and 4 showed slight increases in over-

all scores from pretest 1 to pretest 2; however, the performance for CA relations for Participant 2 decreased. Imme-

diately after completing the training conditions, Participants 1, 3, and 4 passed symmetry (BA, CB), transitivity (AC),

and equivalence (CA) tests, whereas Participant 2 failed symmetry and transitivity tests and required remedial training

to meet the mastery criteria. The mean total number of trials (Table 4) to complete training and testing was 234

(range, 150 to 440). The mean duration of MTS training and emergent relation testing sessions was 49 min (range,

13 min to 2 hr 16 min). The posttest scores for all relations and participants showed a substantial improvement in

performance when compared to pretest scores, with little variability across participants. All participants scored

89% or higher on the posttest (range, 89% to 97%) for all relations.

There was some variation in participant responding during the ABAB generalization probes. The mean score for all

participants was 80% (range, 73% to 87%). The maintenance scores for all relations and participants showed that

trained and emergent relational responding maintained for 2 weeks for three out of four participants (the score for

Participant 3 dropped to 77% during the maintenance probe).

Table 3 shows the results of the satisfaction survey that all participants completed at the end of the research. The

results suggest that all participants were generally satisfied with and accepting of the training procedures.

4 | DISCUSSION

Research in the area of stimulus equivalence and derived relational responding (Hayes et al., 2001; Sidman, 1994) has

focused on the acquisition of basic discriminative and emergent responses in structured settings. However, a recent

FIGURE 3 Results for Participants 1 and 2

FIGURE 4 Results for Participants 3 and 4

TABLE 4 Total number of experimental trials and duration of training and testing required for mastery criterion, byparticipant

Trials to mastery criterion

Participant AB BA BC CB AC CA Remedial Duration

1 105 15 5 5 15 15 N 13 min

2 270 45 60 20 30 15 Y 2 hr 16 min

3 135 15 10 5 15 15 N 23 min

4 90 15 10 5 15 15 N 26 min

10 BLAIR ET AL.

focus on using EBI with MET has yielded promising results with a wide range of learners and across an array of com-

plex skills (Brodsky & Fienup, 2018; Rehfeldt, 2011). When compared to more traditional didactic teaching methods,

this methodology of instruction appears to be an effective alternative, especially when the formation of equivalence

classes, as opposed to simple stimulus generalization, is the goal of the instruction.

In the current study, we used a web‐based EBI system to teach visual analysis of graphical data to undergraduate

students and direct care staff who had no experience with the analysis of single‐subject data graphs. For all partici-

pants, five three‐member classes of arbitrarily related stimuli were formed using automated visual–visual MTS and

MET training of baseline discriminations (AB and BC). In addition, the skills maintained for at least 2 weeks and were

applied to several novel exemplars. All participants who completed the study learned the directly trained baseline dis-

criminations, derived relational responding immediately emerged for three out of four participants, and all partici-

pants passed the tests for the emergence of symmetrical relations (BA, CB), and transitivity and equivalence

relations (AC, CA) by the end of training and testing. These results replicate and extend recent studies (Albright

et al., 2015; Albright et al., 2016; Fienup & Critchfield, 2010; Walker & Rehfeldt, 2012) where the current study

BLAIR ET AL. 11

better controlled for testing effects when compared to previous studies, and with novel complex stimuli that are

socially relevant to behavior analysts.

The current study also replicated and extended the results fromWalker and Rehfeldt (2012) by using EBI to teach

participants to visually analyze data in single‐subject research designs. However, where the researchers trained par-

ticipants to identify and define single‐subject research designs using a stimulus equivalence paradigm and MET, the

current study extended the identification of research designs to the selection of functional relation statements

related to ABAB graphs using a similar training procedure. In contrast to some previous studies, the data from the

current study suggest that not only did equivalence classes form and participants learned to discriminate functional

relations, but the skills also generalized to novel stimuli.

In the current study, 12 BCBAs enrolled in a PhD program correctly matched a graph to a functional relation

statement in only 73% of opportunities. By comparison, the four participants who completed EBI training in the cur-

rent study accurately selected a functional relation statement when presented with a graph in 93% (range, 85% to

100%) of opportunities immediately following training, and in 81% (range, 73% to 90%) of opportunities 2 weeks

after training. Participants in the current study also accurately selected a functional relation statement when pre-

sented with a novel graph in 80% (range, 73% to 87%) of opportunities. In addition, three out of four participants

required less than 30 min of training to demonstrate this ability, which is considerably less time than is usually

required to teach this skill using more traditional methods. The results of the satisfaction survey add to previously

published literature that showed that participants are generally receptive to this type of instruction and prefer it to

other types of instruction (Albright et al., 2015; Albright et al., 2016). The results of the satisfaction survey for this

study confirmed that participants had very little confidence in their ability to visually analyze graphs prior to the train-

ing and felt fairly confident in their skills after training.

The primary limitation of this study is the use of a pretest/posttest design where only two participants were

exposed to multiple pretest probes. The design significantly limits the conclusions of the study, and future research

should choose a stronger design. Another limitation to this study was the results for generalization. After training,

one participant did not generalize the identification of functional relation statements to BAB graphs, which prompted

us to include generalization probes with ABAB reversal graphs for all participants. When presented with novel ABAB

graphs all four participants correctly identified the functional relation statement at relatively high percentages. Given

the limited number of exemplars used in the current study, future studies should investigate how reliably the rule

statements can be applied to a variety of graphs of different research design types or even actual clinical data. In

addition, future research should test how the rules learned in this study apply to clinical decision‐making behavior.

Future research should also investigate the parameters of the exemplars used in training in order to ensure the gen-

eralization of responding to novel stimuli without the need for further training.

A third limitation of the study was the high number of training trials needed to meet training criteria and pass the

emergent relations (AC, CA) tests for Participant 2. Compared to other studies on EBI (e.g., Albright et al., 2016), the

results for this participant appear to be somewhat atypical. Given that the visual analysis of graphs is a complex, pos-

sibly rule‐governed behavior that might include unobserved covert behavior (e.g., tacts and intraverbals), the slow

acquisition might be due in part to the topography of the behavior itself. The slower acquisition of the trained con-

ditional discriminations, and lack of immediate emergence of untrained responding, might have also been related to

the inherent complexity of the compound stimuli (written phrases and graphs). Anecdotal reports from participants

revealed that many had never seen single‐subject graphs before and as a result struggled with the initial AB training

blocks. However, after a participant “got it,” the participant appeared to progress through the training more quickly,

and the results of the study, particularly the immediate emergence of derived relations following training for three

out of four participants, support these self‐assessments. Future research should investigate prerequisite skills of par-

ticipants, conditional discrimination training in EBI with complex/compound visual and written stimuli, fluency with

the terms and possible technical jargon in the stimuli, and pre‐training instructions given to participants.

A fourth limitation was that it is possible that participants were exposed to further instruction or self‐initiated

study in the area of visual analysis. However, given the somewhat arbitrary nature of the stimuli included in the

12 BLAIR ET AL.

current study, the fact that participants were all full‐time undergraduate students or worked full‐time, and the fact

that they likely did not have many opportunities to be exposed to the skill of visually analyzing graphs in the

2 weeks between posttest and maintenance, it appears unlikely that any other variables influenced the perfor-

mance during maintenance.

A final limitation of the study was related to the skill itself that was taught. A functional relation is not a binary

phenomenon—there are degrees to the amount of control that an independent variable has on a dependent variable,

especially in nonlaboratory settings (e.g., classrooms, home, and community). However, the stimuli used in this study

were admittedly somewhat arbitrarily created by behavior analysts according to simplified generally accepted rules.

Future studies in the area of visual analysis should use systematic criteria and rules for classifying graphs (e.g., over-

lapping data points and slopes of trends) and not simply rely on interrater agreement for the establishment of cate-

gories of graphs.

For the current study, we probed for the maintenance of skills only once following training. Although the results

of the study suggest that participants mastered the skills, equivalence classes formed, and responses maintained for

several weeks for most participants, the long‐term effects of a single training session of EBI are unknown. Future

research should focus on the effects of EBI on long‐term maintenance as well as the effects of shorter‐duration

refresher/remedial trainings on maintenance and generalization.

Future research should also investigate using EBI to teach other applied behavior analytic skills associated with

clinical activities. For example, some applied skills might include selecting appropriate function‐based treatment,

assessment procedures, and data collection methods. The current study used EBI to teach the visual analysis of ABAB

reversal graphs; however, future studies could investigate the efficacy of EBI for teaching visual analysis skills for

other graphs, specifically multiple baseline graphs given their widespread use in ABA research.

The results of the current study confirm that typically developing adults can discriminate topographically dis-

similar visual stimuli (pictures‐to‐words) and that the trained responses, with the incorporation of MET, can lead

to the emergence of derived relational responses, or discriminations, and that the responding can generalize to

novel stimuli. However, it is possible that a participant in the current study learned the rule that describes differ-

ent types of graphs and then used that rule to identify the extent of functional relation demonstrated by other

graphs. Given the fact that the graph‐to‐functional relation statement discrimination emerged for all participants

and generalized to novel stimuli, it is reasonable to question whether some part of that performance was due

to the emergence of a relation not only among the three stimuli in the functional relation statement class, but also

among the verbal stimuli in the rule itself (Barnes‐Holmes et al., 2002; Tarbox, Tarbox, & O'Hora, 2009; Tarbox,

Zuckerman, Bishop, Olive, & O'Hora, 2011). The slight increases in pretest scores for Participants 2 and 4 might

also be explained by the use of verbal problem‐solving skills and/or the participants' history with written

selection‐based assessments (i.e., multiple choice tests). However, future research in this area, specifically with

EBI for written visual stimuli in applied settings, is warranted.

Although the results of the current study demonstrated that EBI was used to teach a fairly complex behavior ana-

lytic skill, the skill itself is not exactly the behavior that clinicians engage in when they are analyzing data graphs. Ana-

lyzing data in actual clinical settings is much more complicated and nuanced than the analysis of the graphs that was

demonstrated in the current study. Additionally, the actual verbal behavior (either covert or overt) that a participant

engaged in during the analysis was not measured or assessed. For example, at the conclusion of the training, a par-

ticipant could have been asked to describe the graph, and the participant's response could have been compared to

the rule that was directly taught. Future research should assess a participant's verbal behavior when engaging in ana-

lytic activities, not just visually analyzing graphs. In addition, future studies should test for the emergence of

untrained topographical responses (Reyes‐Giordano & Fienup, 2015) including written and vocal tacts of particular

aspects of the graph (i.e., identifying level, trend, and variability) and tacts of the level of functional relation itself. This

line of research would provide further distinction between the establishment of equivalence classes as a result of

conditional discrimination performances without verbal behavior and performances with the use of verbal behavior

(Jennings & Miguel, 2017; Petursdottir, Carp, Peterson, & Lepper, 2015).

BLAIR ET AL. 13

Complex human behavior has been under both theoretical and experimental investigation since the dawn of man-

kind. Traditional approaches tend to be based on a priori assumptions and appeals to mentalistic constructs that leave

both the scientist and practitioner with an empirically unsupported foundation upon which to build a set of teaching

principles and protocols. However, data from studies like the current investigation allow both scientist and practi-

tioner to better understand and apply tools such as EBI and MET when training complex human behavior. However,

this area is admittedly open for conceptual analysis, and the field of behavior analysis is strongly encouraged to

engage in this conversation. More research needs to be conducted, and much like the training procedures them-

selves, only with multiple exemplars of research studies will a clear and concise conceptual understanding of complex

human behavior, along with practical and empirically supported technologies to train behavior analysts, ultimately

emerge.

ORCID

Bryan J. Blair https://orcid.org/0000-0002-1787-1676

REFERENCES

Albright, L., Reeve, K. F., Reeve, S. A., & Kisamore, A. N. (2015). Teaching statistical variability with equivalence‐basedinstruction. Journal of Applied Behavior Analysis, 48(4), 883–894. https://doi.org/10.1002/jaba.249

Albright, L., Schnell, L., Reeve, K. F., & Sidener, T. M. (2016). Using stimulus equivalence‐based instruction to teach graduate

students in applied behavior analysis to interpret operant functions of behavior. Journal of Behavioral Education, 25(3),

1–20. https://doi.org/10.1007/s10864‐016‐9249‐0

Arntzen, E. (2012). Training and testing parameters in formation of stimulus equivalence: Methodological issues. European

Journal of Behavior Analysis, 13(1), 123–135. https://doi.org/10.1080/15021149.2012.11434412

Barnes‐Holmes, D., Denis, O., Roche, B., Hayes, S. C., Bissett, R. T., & Lyddy, F. (2002). Understanding and verbal regulation.

In Relational frame theory (pp. 103–117). Springer US. https://doi.org/10.1007/0‐306‐47638‐X_6

Brodsky, J., & Fienup, D. M. (2018). Sidman goes to college: A meta‐analysis of equivalence‐based instruction in higher

education. Perspectives on Behavior Science, 41(1), 1–25. https://doi.org/10.1007/s40614‐018‐0150‐0

Carr, J. E. (2005). Recommendations for reporting multiple‐baseline designs across participants. Behavioral Interventions,

20(3), 219–224. https://doi.org/10.1002/bin.191

Christ, T. J., Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Experimental control and threats to internal validity of con-

current and nonconcurrent multiple baseline designs. Psychology in the Schools, 44(5), 451–459. https://doi.org/10.1002/pits.20237

Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis. Upper Saddle River, NJ: Pearson.

Critchfield, T. S., & Fienup, D. M. (2010). Using stimulus equivalence technology to teach statistical inference in a group

setting. Journal of Applied Behavior Analysis, 43(4), 763–768. https://doi.org/10.1901/jaba.2010.43‐763

Danov, S. E., & Symons, F. J. (2008). A survey evaluation of the reliability of visual inspection and functional analysis graphs.

Behavior Modification, 32(6), 828–839. https://doi.org/10.1177/0145445508318606

DeProspero, A., & Cohen, S. (1979). Inconsistent visual analyses of intrasubject data. Journal of Applied Behavior Analysis,

12(4), 573–579. https://doi.org/10.1901/jaba.1979.12‐573

Diller, J. W., Barry, R. J., & Gelino, B. W. (2016). Visual analysis of data in a multielement design. Journal of Applied Behavior

Analysis, 49(4), 980–985. https://doi.org/10.1002/jaba.325

Fields, L., Travis, R., Roy, D., Yadlovker, E., Aguiar‐Rocha, L. D., & Sturmey, P. (2009). Equivalence class formation: A method

for teaching statistical interactions. Journal of Applied Behavior Analysis, 42(3), 575–593. https://doi.org/10.1901/

jaba.2009.42‐575

Fienup, D. M., Covey, D. P., & Critchfield, T. S. (2010). Teaching brain‐behavior relations economically with stimulus

equivalence technology. Journal of Applied Behavior Analysis, 43(1), 19–33. https://doi.org/10.1901/jaba.2010.43‐19

Fienup, D. M., & Critchfield, T. S. (2010). Efficiently establishing concepts of inferential statistics and hypothesis decision

making through contextually controlled equivalence classes. Journal of Applied Behavior Analysis, 43(3), 437–462.https://doi.org/10.1901/jaba.2010.43‐437

Green, G., & Saunders, R. R. (1998). Stimulus equivalence. In K. A. Lattal, & M. Perone (Eds.), Handbook of research methods in

human operant behavior (pp. 229–262). New York: Plenum Press.

14 BLAIR ET AL.

Griffith, K. R., Ramos, A. L., Hill, K. E., & Miguel, C. F. (2018). Using equivalence‐based instruction to teach piano skills to

college students. Journal of Applied Behavior Analysis, 51(2), 207–219. https://doi.org/10.1002/jaba.438

Hagopian, L. P., Fisher, W. W., Thompson, R. H., Owen‐DeSchryver, J., Iwata, B. A., & Wacker, D. P. (1997). Toward the

development of structured criteria for interpretation of functional analysis. Journal of Applied Behavior Analysis, 30(2),

313–326. https://doi.org/10.1901/jaba.1997.30‐313

Hayes, S. C., Barnes‐Holmes, D., & Roche, B. (2001). Relational frame theory: A post‐Skinnerian account of human language and

cognitionSpringer Science & Business Media.

Jennings, A. M., & Miguel, C. F. (2017). Training intraverbal bidirectional naming to establish generalized equivalence class

performances. Journal of the Experimental Analysis of Behavior, 108(2), 269–289. https://doi.org/10.1002/jeab.277

Ninci, J., Vannest, K. J., Willson, V., & Zhang, N. (2015). Interrater agreement between visual analysts of single‐case data: A

meta‐analysis. Behavior Modification, 39(4), 510–541. https://doi.org/10.1177/0145445515581327F

Ottenbacher, K. J. (1993). Interrater agreement of visual analysis in single‐subject decisions: Quantitative review and

analysis. American Journal on Mental Retardation, 98(1), 135–142.

Petursdottir, A. I., Carp, C. L., Peterson, S. P., & Lepper, T. L. (2015). Emergence of visual–visual conditional discriminations.

Journal of the Experimental Analysis of Behavior, 103(2), 332–348. https://doi.org/10.1002/jeab.136

Rehfeldt, R. A. (2011). Toward a technology of derived stimulus relations: An analysis of articles published in the Journal of

Applied Behavior Analysis, 1992‐2009. Journal of Applied Behavior Analysis, 44(1), 109–119. https://doi.org/10.1901/jaba.2011.44‐109. https://doi.org/10.1901/jaba.2011.44‐109

Reyes‐Giordano, K., & Fienup, D. M. (2015). Emergence of topographical responding following equivalence‐basedneuroanatomy instruction. The Psychological Record, 65(3), 495–507. https://doi.org/10.1007/s40732‐015‐0125‐4

Roane, H. S., Fisher, W. W., Kelley, M. E., Mevers, J. L., & Bouxsein, K. J. (2013). Using modified visual‐inspection criteria to

interpret functional analysis outcomes. Journal of Applied Behavior Analysis, 46(1), 130–146. https://doi.org/10.1002/jaba.13

Saunders, R. R., & Green, G. (1999). A discrimination analysis of training‐structure effects on stimulus equivalence outcomes.

Journal of the Experimental Analysis of Behavior, 72(1), 117–137. https://doi.org/10.1901/jeab.1999.72‐117

Sidman, M. (1994). Equivalence relations and behavior: A research story. Boston: Authors Cooperative.

Sidman, M., & Tailby, W. (1982). Conditional discrimination vs. matching to sample: An expansion of the testing paradigm.

Journal of the Experimental Analysis of Behavior, 37(1), 5–22. https://doi.org/10.1901/jeab.1982.37‐5

Stromer, R., Mackay, H. A., & Stoddard, L. T. (1992). Classroom applications of stimulus equivalence technology. Journal of

Behavioral Education, 2(3), 225–256. https://doi.org/10.1007/BF00948817

Tarbox, J., Tarbox, R., & O'Hora, D. (2009). Nonrelational and relational instructional control. Derived relational responding.

In R. A. Rehfeldt, & Y. Barnes‐Holmes (Eds.), Derived relational responding: Applications for learners with autism and other

developmental disabilities: A progressive guide to change (pp. 111–127). New Harbinger Publications.

Tarbox, J., Zuckerman, C. K., Bishop, M. R., Olive, M. L., & O'Hora, D. P. (2011). Rule‐governed behavior: Teaching a

preliminary repertoire of rule‐following to children with autism. The Analysis of Verbal Behavior, 27(1), 125–139.https://doi.org/10.1007/BF03393096

Walker, B. D., & Rehfeldt, R. A. (2012). An evaluation of the stimulus equivalence paradigm to teach single‐subject design to

distance education students via blackboard. Journal of Applied Behavior Analysis, 45(2), 329–344. https://doi.org/

10.1901/jaba.2012.45‐329

Watson, P. J., & Workman, E. A. (1981). The non‐concurrent multiple baseline across‐individuals design: An extension of the

traditional multiple baseline design. Journal of Behavior Therapy and Experimental Psychiatry, 12(3), 257–259. https://doi.org/10.1016/0005‐7916(81)90055‐0

Wolfe, K., Seaman, M. A., & Drasgow, E. (2016). Interrater agreement on the visual analysis of individual tiers and functional

relations in multiple baseline designs. Behavior Modification, 40, 1–22. https://doi.org/10.1177/0145445516644699

How to cite this article: Blair BJ, Tarbox J, Albright L, et al. Using equivalence‐based instruction to teach the

visual analysis of graphs. Behavioral Interventions. 2019;1–14. https://doi.org/10.1002/bin.1669