Standing for Science September 2019€¦ · Standing for Science September 2019 David Celiberti,...

Standing for Science September 2019

David Celiberti, PhD, BCBA-D 1

Standing Up for Science: Ethical Challenges and Opportunities

in the Autism Community

David Celiberti, PhD, BCBA-DAssociation for Science in Autism Treatment

[email protected]

Workshop Presented at the Kentucky Association for Behavior Analysis

September 6, 2019

I presented this workshop last year with Dr. Erin Leif from Monash University

Workshop ObjectivesAt the conclusion of the presentation, participants will be able to:• Identify and describe “red flags” in autism

treatment• Identify and describe common media

misrepresentations of autism treatment• Identify and describe diverse perspectives on

treatment selection• Explain the concerns that result from each

Workshop Objectives• Demonstrate a broader conceptualization of

how the tenets of ABA can be both a model and a framework for delivering science-based treatment regardless of discipline

• Highlight the implications conceptually and procedurally

Workshop Objectives• Describe challenges for behavior analysts

related to:– Interdisciplinary collaboration– Consumer education– Interacting with members of the media

community• Describe strategies for avoiding or reducing

the impact of these challenges

Workshop Objectives• Identify specific and sustainable efforts that

can be made to promote science in the treatment of autism:– Across disciplines– Within interactions with the media community– With consumers

Red Flags in Autism Treatment• A “red flag” signals something potentially

problematic.• Behavior analysts and consumers alike must

identify red flags when evaluating claims about autism treatment.

• The Professional and Ethical Compliance Code for Behavior Analysts addresses many of these responsibilities.

• Of course, possibility for Type I errors.

mailto:[email protected]



Red Flags in Autism Treatment• 1.01 Reliance on Scientific Knowledge:

Behavior analysts rely on professionally derived knowledge based on science and behavior analysis when making scientific or professional judgments in human service provision, or when engaging in scholarly or professional endeavors

Red Flags in Autism Treatment• 2.09 Treatment/Intervention Efficacy (a):

Clients have a right to effective treatment (i.e., based on the research literature and adapted to the individual client). Behavior analysts always have the obligation to advocate for and educate the client about scientifically supported, most-effective treatment procedures. Effective treatment procedures have been validated as having both long-term and short-term benefits to clients and society.

Red Flags in Autism Treatment• 2.09 Treatment/Intervention Efficacy (d):

Behavior analysts review and appraise the effects of any treatments about which they are aware that might impact the goals of the behavior change program, and the possible impact of the behavior change program, to the extend possible

Group Discussion Question• What are some red flags in autism treatment that you

have encountered in the media?• ……………..in your training?• ……………… in your interactions with other professionals?

Remember – a red flag is a warning that something may be problematic (much like a Type II error).

Diverse Responses to “New” Autism Treatments

• Embrace the new treatment wholeheartedly• Reject outright/ Ignore advances altogether as if

they never happened• Emphasize quantity over quality (“More is

better.”)• Add it to the mix and overidentify as “eclectic”

Diverse Responses to “New” Autism Txs: Within ABA

• Overidentification as “eclectic” (theoretical eclecticism versus technical eclecticism)

• Putting all your eggs in one basket (e.g., using ONLY DTI, ONLY PRT, Only VB, only ______)

• The complex needs of persons with autism requires a myriad of behavior analytic techniques

• No single technique is comprehensive enough to address all deficit areas of autism



The Mission of Association for Science in Autism Treatment (ASAT)

• We promote safe, effective, science-based treatments for people with autism by:– sharing accurate, timely, and scientifically sound

information;– advocating for the use of scientific methods to

guide treatment; and– combating unsubstantiated, inaccurate and false

information about autism and its treatment

………regardless of age, severity of condition, income or country of residence.

The Association for Science in Autism Treatment (ASAT)

• ASAT is a national, not-for-profit, 501(c) 3 organization.

• ASAT is committed to science as the most objective, time-tested and reliable approach to discerning between safe, effective autism treatments, and those that are harmful or ineffective.

• ASAT supports all scientifically sound research on the prevention, treatment and cure of autism, as well as all treatments for autism that are shown to be effective through solid scientific research, regardless of discipline or domain.

ASAT: The Early Years

• ASAT was formed in 1998.

• Prior to that time there was a long history of failed treatments and fads, levied on vulnerable children and adults, and on their families.

• Treatment conversation have been dominated by improbable theories about etiology.

• Treatments based on speculation and shaky evidence.

ASAT: Initiatives• Website: www.asatonline.org• Online monthly publication, Science in Autism Treatment

Please subscribe: www.asatonline.org/newsletter/• Media Watch• Initiatives to support parents, teachers, physicians,

journalists, and teaching faculty• 150 hour formalized Externship at the graduate and

undergraduate levels

ASAT’s Research Synopses• To present findings from important new published, peer-

reviewed treatment studies to a general audience and discuss the implications of these findings for providers and families.

• Our synopses are divided into sections addressing: – why the topic matters– what the researchers did– what the researchers found– what were the strengths and weaknesses of the study– what do the results actually mean

ASAT’s Research Synopses: Some areas of focus

• Connection to existing literature• Logical extension from existing literature• Population adequately described• Adequately described IVs and DVs• Clear procedures• Strong controls• Consideration of alternative explanations• Limitations addressed beyond those identified by the

authors

http://www.asatonline.org/



ASAT’s Clinical Corner• Questions about scientifically supported autism treatment,

answered by experts in the field!– Early intervention– Teaching skills– Decreasing challenging behavior– Working with families– Adolescents and adults with autism– Specific treatments and providers– Other considerations

• Responses encourage reliance on existing research, defined targets, defined intervention, ongoing data collection, data-based decision making, treatment fidelity, generalization, and social validity

ASAT’s Treatment Summaries• A description of various marketed treatments for autism,

and the level and scope of supporting research• Psychological, educational, biomedical, & therapeutic

interventions– What works– What needs more research– What doesn’t work or is untested

• Our list is not all inclusive (recall first few slides in presentation L)

ASAT’s Media Watch

• ASATs Media Watch Initiative is designed to:– Respond to both accurate and inaccurate portrayals of

treatment reported by media– Increase awareness of scientific evidence for autism

treatment– Increase appreciation for the relevance of the scientific

method for autism treatment– Share specific information about effective, autism

treatment

Autism in the Media• Fast-paced news cycle• Autism has all the ingredients for a sensational media

product!– Jagged skill profile– Fascinating to many– Emotional appeal– Celebrities eager to tell their story– Controversy

In the Media/Internet:Targets for Deceleration

• Flowery and heart wrenching testimonials• Misrepresentations of research• Fancy websites lacking substance• Google ads promising the moon (like I showed you)• Slanted Wikipedia entries• Sensationalism in the media• Reliance on celebrities who use their status (and whose

behavior may compound public health crises)

In the Media:Targets for Acceleration (Implementation)

• Responsible and accurate journalism• Articles that state important “truths” such as that:

– Science matters– Existing research is relevant– Some treatments have absolutely no research support– For others, the magnitude of scientific support follows along

a continuum– Available research support should not be exaggerated



In the Media:Targets for Acceleration (Recipients)

• Articles that state important “truths” such as that:– Raising a child with autism is challenging (and costly)– The choices that are available to parents are absolutely

overwhelming– Access to qualified providers and effective services may be

limited– Some providers are practicing unethically– Autism tx is a multi-million dollar industry and some

providers are making a ton of $$– The landscape of services for adults with autism is abysmal – Consumers suffer when given misleading information– Pseudoscience drains families of money, time, and hope

Why are Accurate Media Portrayals Important?

• A primary source of information (perhaps the only)• Information shared rapidly• Quick dissemination within a social network• Media can become one of our greatest assets in promoting a

science-based approach to autism treatment

Inaccurate Portrayals of Autism Treatment in the Media: Some themes

• Ignoring the research basis that may already exist for the treatment in focus;

• Exaggerating the research support for an intervention for which no research exists;

• Failing to acknowledge research that does NOT support a particular intervention;

• Promoting ________ as a breakthrough;• Disregarding the relevance of science; and • Disregarding position statements from various professional

organizations that warn against or discourage the use of a particular treatment.

Behavior Analysts’ Ethical Responsibility to the Profession

• 6.02 Disseminating Behavior Analysis: Behavior analysts promote behavior analysis by making information about it available to the public through presentations, discussions, and other media

How Does ASAT Respond?• Letters are written in response to current media coverage

of autism treatment, and submitted to the author or editor by our Media Watch lead coordinator

• Letters are then published on ASATs website, and shared on ASATs social media pages

• Sometimes we just do Media Alerts (2-3 lines)• Media Watch is comprised of volunteer coordinators and

contributing writers

Key Characteristics of Scientific Research

• Scientific Method—A hypothesis about what works, or how it works, is formulated and tested using a research design.

• Scientific Knowledge is Cumulative— Results are incremental and build upon each other.

• Replicated— Repeated demonstrations with several studies find the SAME result.

• Convergent Findings—Results found using different approaches all point to the same conclusion.




• Generalizable— Study findings can be applied broadly, to individuals other than the ones studied.

• Rule Out of Alternative Hypotheses— Other ways to explain results are considered and discussed.


• Peer Review — The study is scrutinized by others with documented expertise

• Meets Rigorous Standards—The study's design, measures, synthesis of findings, and interpretation of results meet rigorous standards of peer review.

Other Characteristics of Scientific Research

• Incorporates IRB approval with attention to risks, benefits, and consent

• Reports clear descriptions and objective measures• Considers alternative explanations • Shares findings tentatively • Avoids confusing correlation with causation• Avoids sweeping overgeneralizations• Proposes next steps/future directions/calls for replication• Recognizes that one published study does not warrant a

paradigm shift• Acknowledges that science is incremental

Components of Evidence-Based Practice

• Evidence-based practice (EBP) is the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences

• APA Presidential Task Force on Evidence divides evidence-based practice into three main components:

1. Best Research Evidence2. Clinical Expertise & Judgment3. Values

Best Available Research Evidence

• Sizable body of research• Variety of research designs used• Interventions are demonstrated to be effective and safe

for diverse populations with diverse clinical problems• Comparative studies show the intervention is equal to or

superior when compared to other interventions

Clinical Expertise and JudgementTreatment Efficacy• Scientific studies

designed to evaluate whether a treatment works

Clinical Utility • The application,

practicality, and usefulness of a specific treatment in a specific setting or context

• How well to scientific findings generalize to the wider community



Clinical Expertise and Judgement

• Is it just opinion?...not exactly • It is the integration of research findings with individualized

data collected for a client in a clinical setting over the course of treatment

• Context is considered with the input of key stakeholders (and re-solicited frequently)

• BCBAs should be “analysts” and “strategic scientists,” and not simply technicians who deliver scripted treatments or protocols

Back to our Code• 2.09 Treatment/Intervention Efficacy (a): Clients have a

right to effective treatment (i.e., based on the research literature and adapted to the individual client). Behavior analysts always have the obligation to advocate for and educate the client about scientifically supported, most-effective treatment procedures. Effective treatment procedures have been validated as having both long-term and short-term benefits to clients and society.

Clinical Expertise and Judgement• There are a number of required competencies:

– Assessment, diagnostic judgement, and treatment planning (3.01 Behavior-Analytic Assessment)

– Progress monitoring and clinical decision-making (2.09 Treatment Efficacy)

– Continual self-reflection and professional development (1.03 Maintaining Competence)

– Evaluation and use of research in clinical practice (1.01 Reliance of Scientific Knowledge)

– Understanding of client characteristics that may influence treatment (4.02 Involving Clients in Planning)

– Seeking available resources (e.g., supervision, consultation) (1.02 –Boundaries of Competence)

– Having a clear rationale for treatment strategies (4.01 Conceptual Consistency)

Client Characteristic, Culture, and Preferences

• What works for whom, it what context?• Important to consider:

– Values– Goals– Preferences– Quality of life/happiness– Life stage– Development– Gender/gender identity– Religious beliefs/spirituality– Ethnicity– Family considerations

New York’s 1999Department of Health Report

– ABA was the only treatment substantiated to be effective based on the scope and quality of research.

– This does not mean that ABA is the only intervention that is effective, just the only one demonstrated to be effective.

– Again, this does not mean that ABA is the only intervention that is effective, just the only one demonstrated to be effective.

New York’s 1999Department of Health Report

– Cela ne signifie pas que l'ACA est la seule intervention qui est efficace, juste le seul démontré pour être efficace.

– Это не означает, что ABA является единственным вмешательством, которое является эффективным, так же только один доказали свою эффективность.

– Detta betyder inte att ABA är det enda som är effektiv, bara den enda visat sig vara effektiva.



New York’s Department of Health ReportBut what does it mean??– This finding is important as it demonstrates that

applied behavior analysis enjoyed a respectable body of scientific support

– ABA was scientifically investigated by a wide array of researchers.

– ABA has been known to be effective for a number of years now (in contrast to newer approaches that have no such track record of documented success)

NY Department of Education Report– The Autism Program Quality Indicators is a self

review and quality improvement guide for schools and programs servicing autism spectrum disorders.

– The Autism Program Quality Indicators (APQI) are a compilation of the best practices in educating students with autism, which were developed to serve as a means of guiding quality improvement activities.

– Information about the 2001 New York State document can be found at http://www.p12.nysed.gov/specialed/autism/apqi.htm.

Maine Department of Health & Human Services/Dept of Education (2009)• Six Levels of Evidence:

– Established Evidence: Multiple strong or adequately rated group and/or single-subject studies.

– Promising Evidence: Shown effective in more than two group design studies or three single-subject studies.

– Preliminary Evidence: The intervention has been shown effective in at least one group or single-subject design study.

– Studied and No Evidence of Effect: Numerous studies show no positive effect on the desired outcomes.

– Insufficient Evidence: Conclusions cannot be drawn due to a lack of quality research and/or mixed outcomes across several studies.

– Evidence of Harm: Studies indicate that the intervention involves significant harm or risk of harm, including injury and death.

Maine Department of Health & Human Services/Dept of Education (2009)• Published a report of the state of the evidence for autism intervention

in October 2009. • See http://www.maine.gov/dhhs/ocfs/cbhs/ebpac/asd-report.doc• This report was precipitated, in part, by the 276% increase in the

number of students receiving services for autism spectrum disorders in Maine from 2000 to 2008.

• Prior to delineating their findings, I want to share the final 2 sentences of the Executive Summary on page 11:

“The first step toward evidence-based practice is creating awareness of what the best practice research says. It is no longer enough to use what we believe works we must consider what we know works in order to close the gap between science and practice, utilize limited resources wisely, and best serve Maine children with ASD.”

Maine Department of Health & Human Services/Dept of Education (2009)• Established Treatments for Autism

– ABA• Challenging Behavior • Communication • Early Intensive Behavioral Intervention (EIBI)

– PECS– Pharmacological Interventions

• Halperidol (Haldol): aggression • Methylphenidate (Ritalin): hyperactivity • Risperidone (Risperidol): irritability, social

withdrawal, hyperactivity, stereotypy

Maine Department of Health & Human Services/Dept of Education (2009)• Promising Treatments for Autism

– ABA for adaptive living skills– CBT for anxiety– Voice Output Communication Aid (VOCA)

http://www.p12.nysed.gov/specialed/autism/apqi.htm

http://www.maine.gov/dhhs/ocfs/cbhs/ebpac/asd-report.doc



Maine Department of Health & Human Services/Dept of Education (2009)• Preliminary Evidence

– ABA for academics and vocational skills– CBT for anger management– Sign Language for communication– Vitamin C for sensorimotor symptoms

Maine Department of Health & Human Services/Dept of Education (2009)• Preliminary Evidence

– Pharmacological Approaches• Atomoxetine (Strattera): for attention deficit and

hyperactivity • Clomipramine (Anafranil): for stereotypy, ritualistic

behavior, social behavior • Clonidine (Catapres): for hyperactivity, irritability,

stereotypy– Touch Therapy/Massage– Hyperbaric Oxygen Treatment

Maine Department of Health & Human Services/Dept of Education (2009)• Studied and no evidence of effect

– Dimethylglycine (DMG)– Secretin

Maine Department of Health & Human Services/Dept of Education (2009)• Insufficient Evidence

– Facilitated Communication

– Gluten-Casein Free Diets – Omega-3 Fatty Acid

Supplements – Vitamin B6/Magnesium

Supplements – DIR/Floortime – RDI – Social Skills Training – Social Stories™ – Sensory Integration

Therapy

– Pharmacological Approaches

• Guanfacine (Tenex) • Intravenous

Immunoglobin • Melatonin • Naltrexone (Revia) • SSRIs• Valproic Acid

(Depakote)– Auditory Integration

Training– TEACCH

Maine Department of Health & Human Services/Dept of Education (2009)

• Evidence of Harm– Chelation (using Edetate Disodium)

Maine Department of Health & Human Services/Dept of Education (2009)

• “The first step toward evidence-based practice is creating awareness of what the best practice research says. It is no longer enough to use what we believe works we must consider what we know works in order to close the gap between science and practice, utilize limited resources wisely, and best serve Maine children with ASD.”



National Standards Project (2009)• A comprehensive report and analysis on the state of

the empirical literature was published by the National Autism Center.

• Review of the published literature regarding autism intervention.

• See http://www.nationalautismcenter.org/pdf/NAC%20Standards%20Report.pdf

Categories of theNational Standards Project (2009)

• Established Treatments were determined to be known to be effective for individuals on the autism spectrum.

• Emerging Treatments had some evidence of effectiveness, but not enough for the National Autism Center to be confident that they are truly effective treatments

• Un-established Treatments had no sound evidence of effectiveness

• In-effective or Harmful treatments had several published, peer-reviewed studies in which either no beneficial treatment effects or adverse treatment effects were found.

National Standards Project (2009)• Established Treatments

– Known to be effective for individuals on the autism spectrum. – Majority of these interventions were developed in the

behavioral literature (e.g., applied behavior analysis, behavioral psychology, and positive behavior support).

• Emerging Treatments – Some evidence of effectiveness, but not enough for us to be

confident that they are truly effective (e.g., Cognitive Behavior Therapy, AAC Devices, Developmental-Relationship based therapy, Social Skills Training, PECS)

• Un-established Treatments– No sound evidence of effectiveness (Auditory Integration

Training, Facilitated Communication, GFCF Diets)– Possibility these treatments are ineffective or harmful.

Evidence-based Practice and Autism in the Schools: A Guide to Providing Appropriate Interventions to Students with Autism Spectrum Disorders

• Also published by the National Autism Center and can be found at https://www.nationalautismcenter.org/resources/

• The 2nd chapter is devoted to description of 11 established treatments; however, there is tremendous overlap between the established treatments.

• It is important to note that all of these have a basis in applied behavior analysis.

• There are other useful chapters in this resource.

Evidence-based Practice and Autism in the Schools: A Guide to Providing Appropriate Interventions to Students with Autism Spectrum Disorders

• Antecedent package • Behavioral package • Comprehensive behavioral treatment for young children• Joint attention intervention • Modeling • Naturalistic teaching strategies• Peer training package • Pivotal response treatment • Schedules • Self-management • Story-based intervention package

National Standards Project (2021)– Will provide updated information about which

interventions have been shown to be effective• Across lifespan• Will include prior literature in Phases I and II• Will include new research (that meets criteria)

published through 2018

http://www.nationalautismcenter.org/pdf/NAC%20Standards%20Report.pdf

https://www.nationalautismcenter.org/resources/



• Evidence-based practice for individuals with ASD -efficacy must be established through peer-reviewed research in scientific journals using:– At least two randomized or quasi-experimental design

studies. Two high quality experimental or quasi-experimental group design studies,

– At least 5 high quality single-subject design studies. (by three different investigators or research groups), or

– Combination of evidence.

Please see: https://autismpdc.fpg.unc.edu/evidence-based-practices

National Professional Development Center

• Parent Implemented Intervention The NSP did not consider parent-implemented intervention as a category of evidence-based practice. However, 24 of the studies reviewed by the NSP under other intervention categories involve parents implementing the intervention.

These were identified as emerging by the NSP• Social Skills Training Groups (Social Skills Package in NSP)

Speech Generating Devices (Augmentative and Alternative Communication Device in NSP)

• Computer Aided Instruction (Technology-based Treatment in NSP• Picture Exchange Communication • Extinction (Reductive Package in NSP)

National Professional Development Center

Key take home messages• Although there are 27 identified EBPs, we still don’t know

exactly what works best for whom, and when (much more work to do to assess the generality of these findings!)

• The most important thing to consider when designing and implementing an evidence-based intervention for a child with autism is the progress the child makes when the intervention is implemented

• Clinical judgement and data-based decision-making at the individual level are critical

Key take home messages• And…it doesn't matter how effective our interventions are

– if our clients don’t like them and won’t use them! • We must continue to identify client values and integrate

values with the best available research evidence and our clinical decision-making

• The question remains - How can we better “sell” science to consumers, professionals, and the media?

How Can We Better “Sell” Science To The Media??• Be an effective communicator

– Avoid jargon– Explain things using plain English– Make the science accessible to the masses– Keep your take-away message at the forefront– Keep in mind what you want your audience to do, to

feel, to know• Practice your plain English explanations, so you are ready

when the opportunity presents itself!

How Can We Better “Sell” Science To The Media??• Show the humanity underlying ABA• Remember to highlight the heartfelt story within the

science– We love data, but we also love helping families and

changing people’s lives for the better!– People may not want to hear about our data, but they

do want to hear our stories– Sell the story, and include information about the

science as you go

https://autismpdc.fpg.unc.edu/evidence-based-practices



How Can We Better “Sell” Science To The Media??

• Be available to journalists– If there is an event believed to be covered by the

media, be there!– Recognize the tight deadlines of the writer– If a journalist contacts you asking to set up a time to

talk about what you do, get back to him/her promptly! – Have business cards and some simple informational

material ready to distribute– Write thank you notes and follow up after a meeting

Implications of EBP• Multiple benefits exist when individuals with autism receive

the MOST EFFECTIVE intervention in the short term:– Quality of life for the child– Quality of life for the family – More meaningful inclusive experiences in school– Broader array of opportunities within community

Implications of EBP• Multiple benefits exist when individuals with autism receive

the MOST EFFECTIVE intervention and in the long term:– Reduced financial expenditures and allocation of special education

resources (e.g., Jacobson, Mulick, Green, 1998)– Improved prognosis– Broader range of employment and higher education opportunities

– And reinforcing behaviors in treatment providers (accountability, adherence to science)

Implications of not accessing EBP• Individuals with autism subjected to:

– Stagnated methods – Ineffective methods– Providers unaware of scientific advances– Providers unwilling to incorporate scientific advances

– “One size fits all” programming

…..will unlikely realize their fullest potential

EBP: An evolving conceptThis is constantly evolving:•Research is published in peer reviewed journals each month.•Technology is refined and enhanced.•Best practices/Guidelines are developed

Bottom line: What is considered an “appropriate” education or treatment must similarly evolve over time.Unfortunately: Teachers and providers are often not in communities/systems that support access to this evolution.

The basis of our work:• Identification of Dependent Variable(s)• Identification of Independent Variable(s)• Systematic Manipulation of Independent Variables• Determining Functional Relationship between

Independent and Dependent Variables



Definition of Applied Behavior Analysis• “The science in which procedures derived from

principles of behavior are systematically applied to improve significant behavior to a meaningful degree and to demonstrate experimentally that the procedures employed were responsible for the improvement in behavior.”

- J. Cooper, T. Heron, and W. Heward, 1987

Some Challenges for Behavior Analysts• Bias against single case research studies

• Abundant misconceptions about our work

• Abundant misconceptions about behavior analysts

• Consumer perception that poor implementation is norm not the exception

Some Challenges for Behavior Analysts• Attacks on the “messenger”

• Some members of the neurodiversity community actively blog that ABA is harmful/abusive

• Some members of the neurodiversity community actively blog that behavior analysts are unethical…and even disparage parents

• Tremendous challenges outside of US (e.g., How ‘Fake News’ Affects Autism Policy Keenan and Dillenburger)

Some Challenges for Behavior Analysts• Behavior analysts will need to acknowledge scientific

advances of other disciplines.

• Some BAs must acknowledge that other disciplines are even capable of scientific research.

• Research that includes ABA + something else versus a community comparison often gives all the credit to that “something else”

• In many contexts and reviews, “scientific validation” is unfortunately a dichotomous variable.

Some Opportunities for Behavior Analysts

• To convey the strength of the commitment to science, behavior analysts can advocate for the consideration of other factors such as– Number of published researchers– Number of published researchers with 5+ publications– Number of laboratories– Number of universities– Number of conference attendees– Number of posters, etc. etc.


• Behavior analysts will need to (carefully) caution others not to overgeneralize the findings of new studies;– Those authored by tx proponents that will gain financially from the

publication– Those studies not yet replicated– Those studies with putting forth conclusions that are not warranted

by design/data.– And even those “studies” not yet published

• On the other hand, behavior analysts need to draw attention to Convergent findings (i.e., Results found using different behavior analytic approaches all point to the same conclusion).




• Behavior analysts will need to better advocate for the inclusion of single-case research designs when evaluating treatments (decades of research by scores of labs targeting a wide array of treatments often disregarded)

• We can show our colleagues that we have a literature base and a rich repertoire to address concurrent problems such as: wandering/elopement, lack of exercise, sleep problems, pica, food selectivity, anxiety, catatonia, etc.

Some Opportunities for Behavior Analysts• Behavior analysts may need to clarify what peer-review

actually means (i.e., quality on a continuum)

An Example: PloS ONE

A review of their stated Editorial and Peer Review Process (2) indicates that they employ a unique review process, with 69% of 2,216 manuscripts accepted in a recent 3-month period.

The journal charges authors $1,350 for publication of their article.


• Strive for data-based and collaboratively generated guidelines about which individuals, for which topographies of responses, and under what conditions, would likely benefit from which procedure, and when

• Behavior analysts will need to consider how to incorporate other advances into treatment packages to create synergistic gains, when and how to collaborate……….and/or when and how to make appropriate referrals. Can you still be “conceptually systematic?


• Behavior analysts may want to clarify what “promising” and “emerging” could/should mean (if I may chime in, an “accelerating trend” would be a better descriptor if applied accurately).

• Show the humanity of ABA• Acknowledge that poor implementation does indeed exist,

that is problematic, but it not the norm.

Some Opportunities for Behavior Analysts• Behavior analysts will need to use shaping procedures to

improve quality/rigor of research (and perhaps refrain from displaying too thin of a schedule of reinforcement)

• Use the totality of the principles of learning to strengthen and weaken behaviors that are consistent/inconsistent with science-based treatment.

………………and there are many!

ABA Practices that have Implications for Measuring and Evaluating

non-ABA Interventions

• Identify targets for change• Operationalize targets (specific, observable, measurable)• Obtain adequate baseline data• Rely on repeated measurements by collecting data on

ongoing basis • Collect Inter-Observer agreement data (IOA)• Change treatments one at a time to maximize accuracy of

decisions evaluating those changes





• Employ a variety of single case research designs:– To test hypotheses– To identify functional relationships between IV manipulation

and DV changes– To assess effect

• Commonly employed single case designs:– AB Design– ABA Reversal Design– ABAB Repeated Reversal Design– Multiple Baseline Design– Simultaneous or Alternating Treatments Design– Changing Criterion Design



• Visual Inspection of Data to Note:– Latency for change to be observed– Heterogeneity of data– Trends and levels– Overlapping data across conditions– Similarity in data display across conditions

• See some classic texts:– Horner, Carr, Halle, McGee, Odom, & Wolery (2005)– Kazdin (1992)– Kratochwill & Levin (1992)– Parsonson & Baer (1978)

ABA Practices that have Implications for the Implementation of

non-ABA Interventions• Operationalize define the intervention• Distinction between form versus function• Evaluate treatment fidelity• Maintain, alter, or discontinue intervention based upon

objective data• Explicit efforts to promote generalization and maintenance

(and assessment of such when measuring outcomes) • Family training• Social validity and promotion of quality of life• Consider threats to internal validity

Concerns that Have Implicationsfor Evaluation of Other Interventions

• Threats to Internal Validity (See Cook & Campbell, 1979)– History – Maturation– Testing– Instrumentation– Statistical regression– Selection biases– Attrition– Combinations of selection and other threats– Relationship Condition, Temporal Antecedence Condition, Lack of

Alternative Explanations

• External Validity Concerns

Aspirational Targets: Commitment to Science

• A respect and acknowledgment of the science that already exists

• Efforts and resources directed to support needed future scientific activity

• A shared expectation that providers (of all disciplines) will rely on science and the scientific method, collect data, and use data routinely to guide decision-making

Aspirational Targets:Responsibility

• Abundant transparency and accountability (to all consumers, even tax payers)

• Informed consent is fully respected• Providers and decision makers who treat other people’s

children like they would want their own treated if they had autism



Aspirational Targets:Parents’ Experiences

• Parents not having to fight so hard when advocating for their children or to access evidence-based treatment

• We don’t write off our adult population.• When the insurance reimbursement systems are clear and

accessible for consumers.

Provider Behavior:Targets for Deceleration

• Sweeping statements lacking support but appealing to consumers

• Exaggerating benefit and minimizing cost and effort• Accepting anecdotal evidence • Hiding behind the importance of “individualization” to

justify doing whatever one wants• Acting defensively when parents ask questions about:

– Available research– Competencies and experiences – Recommendations– Benefits

Provider Behavior:Targets for Acquisition

• Recognizing and respecting the relevance of existing research…or lack there of

• Providing realistic description of cost, effort involved in implementation

• Using the measurement and analysis tools inherent in ABA

• Putting data where one’s mouths are when making claims about treatment selection and progress made

• Engaging in data-based, decision making (e.g., objective measurement, prompt and effective assessment, analysis, and troubleshooting)

Tx Proponent Behavior:Targets for Deceleration

• Relying only on anecdotal evidence to advance tx• Ignoring existing science• Misusing research• Misapplying research citations• Exaggerating the status/extent of one’s research efforts• Exploiting the terms “promising” and “emerging” to mean

more than how it was intended

TX Proponent Behavior:Targets for Deceleration

• Working exclusively for financial gain• Shooting the messenger (e.g., maligning/dismissing those

who report…… “There is no published research supporting Treatment X” when such statements are actually entirely true!)

Tx Proponent Behavior:Targets for Acquisition

• Showing honesty and accuracy (when describing underlying research support)

• Demonstrating specificity (e.g., explicitly stated procedures, explicitly stated targets, ongoing assessment and commitment to within subject design, using idiographic data)

• Discussing limitations to the treatment• Understanding external validity• Providing funding to support external/objective research• Participating in the submission of clinical research in

autism treatment to peer-reviewed journals



Consumer Behavior:Targets for Deceleration

• Trying everything• Being susceptible to others arguing that they try anything

and everything• Awaiting the “next big thing”• “Quantity over Quality” decision-making• Second guessing choices based on blogs that bastardize

ABA• Allowing other people to choose targets• Accepting slow acquisition/ stagnation• Conflating bad implementation with a “bad” treatment

Consumer Behavior:Targets for Acquisition

• Recognizing that science matters (medicine, technology)• Being able to rise above all the “noise” and to remain

focused on science-based treatment.• Being able to counter the influence of peer and familiar

pressure• Assuming the role of a savvy consumer (e.g., Caveat

Emptor and Caveat Lector)

Consumer Behavior:Targets for Acquisition

• Asking a lot of questions – Why exactly is my child a good candidate for this intervention?– How are you going to measure that? – Is there research to support what you are promoting?– Where may I find it? – ……How come there is not any research as you have been

marketing this treatment for over 10 years?

At School:Targets for Deceleration

• Cookie-cutter treatment delivery and IEPs• Unplanned Inclusion, inclusion through osmosis• Those teachers and other providers who remain stagnant• Those teachers and other providers who do what they

want, what they like, what is easiest, what is cheapest, and what they already know

• Significant portions of the school day allocated to interventions lacking a research basis or data to support efficacy

• Administrative decisions that undercut quality (training, supervision, oversight)

In the School Setting:Targets for Acquisition

• Generalizable and enduring outcomes– And recognition when they are not!

• IEPs that are well conceptualized and well coordinated• IEP Goals that are actually measureable• Comprehensive and ongoing efforts to maximize learner

motivation• Comprehensive analysis of verbal operants• Comprehensive, functionally-relevant behavior plans• Systems that support quality implementation• Investment in training• Meaningful, planned, and well supported inclusion

Standing for Science September 2019€¦ · Standing for Science September 2019 David Celiberti,...

Documents

Transcript of Standing for Science September 2019€¦ · Standing for Science September 2019 David Celiberti,...