Post on 29-Jul-2020
12/12/2013
1
Evidence-Based Practice:
Definitions and Practical
Application
Patrick Coppens, Ph.D., CCC-SLP SUNY Plattsburgh
GSHA
Atlanta
February 8, 2014
Disclosures
Relevant financial relationship:
I am receiving an honorarium from GSHA for
this presentation
Relevant nonfinancial relationship:
none to disclose
SLP is a scientific field
Pseudoscience is “a body of belief and practices but seldom a field of active enquiry; it is tradition bound and dogmatic rather than forward looking and exploratory”
(Bunge, 1984, p. 41).
Science Pseudoscience
Objective (testable) Subjective (untestable)
True scientific method May sound “scientistic”.
No evidence. Belief-based.
Evolves with knowledge Does not change. Based
on traditions, anecdotes.
Science & Pseudoscience
SLPs are clinical scientists
Gather information about client
Observe and measure behaviors
Apply therapy
Draw clinical conclusions based on measurements
Write up results
EBP provides a strategy to ensure that all clinical decisions are of the highest quality and represent the best possible service to the client
Why EBP?
It’s the ethical thing to do!
Clinical decisions based on sound evidence.
Minimizes intuition and other unsupported claims = “data-driven care.”
Best care for best outcome.
Reduce disparities and variation in care
Recognizes that not all evidence is created equal!
Limits the value of “expert opinion.”
Explicitly includes the client’s values, preferences, etc.
Why EBP?
Everybody wins when EBP is applied! ◦ clinicians are ethical, accountable
◦ clients are well-served
◦ insurance companies get a good service that works for their rehabilitation $
12/12/2013
2
What ASHA says…. (ASHA position statement, 2005)
“It is the position of the American Speech-
Language-Hearing Association that audiologists
and speech-language pathologists incorporate
the principles of evidence-based practice in
clinical decision making to provide high quality
clinical care.”
“In making clinical practice evidence-based,
audiologists and speech-language pathologists—
◦ acquire and maintain the knowledge and skills that are
necessary to provide high quality professional
services, including knowledge and skills related to
evidence-based practice.”
What ASHA says…. (ASHA Code of Ethics, 2010)
Principle of Ethics I – Rule B
“Individuals shall use every resource … to
ensure that high-quality service is provided”
Principle of Ethics II
“Individuals shall honor their responsibility to
achieve and maintain the highest level of
professional competence and performance”
Barriers to EBP use
Reported
Problem
Solutions
Access ?
Time ?
Lack of evidence
or
Insufficient evidence ?
Contradictory
evidence ?
Limited training in
EBP and research. Congratulations!
That’s why you are here!
Lack of information
literacy skills. Congratulations!
That’s why you are here!
Barriers to EBP use:
one caveat….
Reported
Problem Solution?
Time • Not only the responsibility of the SLPs.
• The PARIHS framework (Promoting
Action on Research Implementation in
Health Services) recognizes
“Organizational Culture and Climate” as
partly responsible for the good
implementation of EBP. (Kitson et al., 1998)
• Successful implementation =
Evidence + Context + Facilitation
• Advocacy is the solution here (at the
individual and ASHA levels)
EBP: Skills to hone…
Scientific thinking
◦ Always doubt observed
relationships: a brain is easy to fool!!!
◦ Be a skeptic (including for your own work).
◦ Always think of alternative explanations.
Learn to say “why?”
◦ Some clinicians readily trust information
reported by authority figure or friends.
Armed with your scientific and critical thinking skills, it is
now time to tackle EBP…
12/12/2013
3
Evidence-Based Practice Evidence-Based Practice
Clinical Decision
Practice Based
Evidence
Best external scientific evidence
Patient preferences and values
http://www.asha.org/Members/ebp/web-tutorial
Dollaghan (2007); Lof (2011)
Topics to be discussed…
EBP components
1. Patient values, preferences, circumstances
2. Best external evidence:
A. Asking the right question
B. Finding the information
C. Evaluating the evidence
i. Strength of rationale
ii. Strength of design
iii. Strength of methods
3. Practice-based evidence
A. Asking the right question
B. Evaluating the evidence
Topics to be discussed…
EBP components
1. Patient values, preferences, circumstances
2. Best external evidence: A. Asking the right question
B. Finding the information
C. Evaluating the evidence
i. Strength of rationale
ii. Strength of design
iii. Strength of methods
3. Practice-based evidence A. Asking the right question
B. Evaluating the evidence
Clinical Decision
Patient preferences and values
1. Patient Values,
Preferences, Circumstances
We know how to do this: make it functional.
EBP (Dollaghan, 2007):
1. Choice of goals: find agreed upon objectives, but
may require counseling.
2. Choice of approach: all must be based on EBP, but
client preferences and/or
circumstances may tip the
balance.
Possible ethical dilemma:
client requests a discredited
approach.
Do
llag
ha
n (
20
07
).
12/12/2013
4
Patient Values, Preferences,
Circumstances
Conclusions:
What should we do?
1. Listen to the client/family
2. Understand needs but also limitations
(financial, transportation, support, etc.)
3. Develop common goals but without
compromising your prognosis. Counsel if
needed.
4. Use form to compare 2 possible Tx
approaches.
Barriers to EBP use
Reported
Problem
Solutions
Access ?
Time ?
Lack of evidence
or
Insufficient evidence ?
Contradictory
evidence ?
Limited training in
EBP and research. Congratulations!
That’s why you are here!
Lack of information
literacy skills. Congratulations!
That’s why you are here!
Topics to be discussed…
EBP components 1. Patient values, preferences, circumstances
2.Best external evidence: A. Asking the right question
B. Finding the information
C. Evaluating the evidence
i. Strength of rationale
ii. Strength of design
iii. Strength of methods
3. Practice-based evidence
A. Asking the right question
B. Evaluating the evidence
Clinical Decision
Best external scientific evidence
2. Best External Evidence
A. Asking the right question
B. Finding the information
C. Evaluating the evidence
A. Asking the right question
The PICO question:
Population/Patient
Intervention
Comparison
Outcome
A. Asking the right question
“Which is the best treatment for aphasia?”
In aphasic adults (P) does Semantic Feature Analysis Tx (I) lead to significantly improved naming (O) as compared to no treatment (C)?
Including all 4 characteristics will: make the information gathered more relevant for
the particular client
facilitate the search process.
Trade-off: level of specificity will increase relevancy but
make literature search more difficult.
12/12/2013
5
A. Asking the right question
E.g. (Gillam & Gillam, 2008):
Which type of intervention, computer based (I),
group pullout (C), or individual (C), provided to
preschool children with speech and language
impairments (P) results in the greatest
improvement on measures of phonemic
awareness (O)?
practice a PICO question:
B. Finding the information
For example, look at: ASHA EBP compendium :
http://www.asha.org/members/ebp/compendium/ ASHA evidence maps: http://www.ncepmaps.org ANCDS websites:
http://www.ancds.org/index.php/practice-guidelines-9 http://aphasiatx.arizona.edu/
B. Finding the information For example, look at: Public databases: http://scholar.google.com/ http://www.tripdatabase.com/ http://www.speechbite.com/ http://highwire.stanford.edu/ TBI resources: http://www.psycbite.com
Cochrane collaboration: http://www.cochrane.org Contact your local university. Contact the author.
C. Evaluating the evidence
Look at:
i. Strength of rationale
ii. Strength of design
iii. Strength of methods
Judge the importance
of the results
Importance of critical
and scientific thinking
There are good resources
available
There are forms (or create your own)
Do
llag
ha
n (
20
07).
Le
mo
nce
llo &
Fa
nn
ing
(2
011
).
12/12/2013
6
Gill
am
& G
illam
(2
00
8).
C. Evaluating the evidence
i. Strength of rationale
Is the review of the literature
thorough? Have the authors
ignored some important element?
Is there a reasonable research
question based on the lit review?
Is the question clinically relevant for
your purpose?
C. Evaluating the evidence
ii. Strength of design
ASHA levels of evidence
Level Description
Ia Well- designed meta-analysis of >1 randomized
controlled trial
Ib Well-designed randomized controlled study
IIa Well-designed controlled study without
randomization
IIb Well-designed quasi-experimental study
III Well-designed non-experimental studies (e.g.,
correlations, case studies)
IV Expert committee report, consensus conference,
clinical experience of respected authorities
C. Evaluating the evidence
ii. Strength of design
12/12/2013
7
C. Evaluating the evidence
iii. Strength of methods
• Essentially 3 broad avenues of
inquiry:
• Statistical issues: do the stats fit the design?
• Internal validity issues: are there alternate
explanations for the observed results?
• External validity issues: are the results
generalizable to other individuals?
Statistical issues
E.g.:
• alpha = 0.05
• Correlation and causation
• Between-subject vs. within-subject
Internal validity threats: E.g.:
An external variable intervenes during the experiment.
Maturation or spontaneous recovery effect. Precision of measurement: validity and reliability of tests and measures,
calibration of instruments. Inter- and intra-rater reliability.
Unequal groups. Floor & ceiling effects.
0
5
10
15
20
25
30
1 2
External validity issues: E.g.:
Is the sample representative? You can only
generalize to the same subjects.
You can’t generalize to other settings.
Multiple Treatment Interference: if there are
multiple steps or sequential treatments, the
generalization can only occur to people who
receive the same sequence of steps.
Examples for practice:
Find the possible confounding variables:
An investigator measures language comprehension in
10 male and 10 female elderly subjects without
dementia in the presence of 4 different levels of
ambient noise.
An investigator asked severe stutterers to have a
conversation with a close friend and a conversation
with a stranger in the clinical setting to investigate
the effect of conversation partner on stuttering
frequency.
Best External Evidence
Conclusions:
What can we do?
A. Asking the right question
Practice PICO.
B. Finding the information
Get familiar with the websites and databases.
Rely on guidelines, systematic
reviews, meta-analyses.
Use local university contacts.
12/12/2013
8
Best External Evidence
Conclusions:
What can we do?
C. Evaluating the evidence Use critical thinking.
Use scientific thinking.
Develop an easy-to-use form.
Practice evaluating articles.
Update statistical knowledge, get familiar with internal and external validity threats (e.g., general research method books).
And most importantly, do not do this for all clients at once! (Robey, 2011)
Best External Evidence
Robey (2011): A medley:
◦ Clinicians “came to EBP as competent and
experienced clinicians (and) were engaged in
ongoing professional-development learning
activities”
◦ “the process of EBP begins with clinicians’ …
choosing a certain aspect of practice for
enhancement”
◦ “target only one clinical decision for
improvement … and then move to another”
◦ “…must enforce realistic limits on their time”
Barriers to EBP use
Reported
Problem
Solutions
Access •Use databases. Use ASHA. •Ask your local university.
Time • Lots of review articles exist, USE THEM.
• Use a simple evaluation form.
• Tackle 1 topic at a time.
Lack of evidence
or
Insufficient evidence
•Ask a different question for your search
• Seek closest possible applicable evidence. • generate your own evidence (see below)
Contradictory
evidence • Which is strongest?
Limited training in EBP
and research.
Congratulations!
That’s why you are here!
Lack of information
literacy skills.
Congratulations!
That’s why you are here!
Topics to be discussed…
EBP components 1. Patient values, preferences, circumstances
2. Best external evidence:
A. Asking the right question
B. Finding the information
C. Evaluating the evidence
i. Strength of rationale
ii. Strength of design
iii. Strength of methods
3.Practice-based evidence A. Asking the right question
B. Evaluating the evidence
Clinical Decision
Practice Based
Evidence
3. Practice Based Evidence
Complements external evidence: effectiveness (clinical setting) instead of efficacy (controlled environment).
This must be more than subjective experience: “Practice-Based Evidence.”
The same critical and scientific thinking must be applied to clinical work. Controls are still necessary to draw reasonable conclusions.
If there is no evidence, provide it! But you need a supported rationale.
A. Asking the right question
The same PICO principle applies to daily
clinical application:
In a chronic patient with Broca’s aphasia
(P) does Semantic Feature Analysis Tx
(I) lead to significantly improved
naming (O) as compared to traditional
stimulation approach (C)?
In this case, you are attempting to answer the
question yourself
12/12/2013
9
Do
llag
ha
n (
20
07).
B. Evaluating the evidence
The problem is to defend against
confounding variables.
how confident am I that the therapy
caused the observed improvement as opposed to
another competing variable (maturation)?
2 areas to watch:
Measurement
Design
B. Evaluating the evidence:
Measurement
Establish a stable pre-Tx baseline.
Make sure your measurements are valid ◦ define your scoring protocol carefully
◦ use other scorers or multiple scorers (inter-rater
reliability)
B. Evaluating the evidence:
Design
The traditional pre/post design (or ABA)
has problems: it is difficult to conclude on
the success of the therapy.
B. Evaluating the evidence:
Design
B. Evaluating the evidence:
Design What we like to see:
12/12/2013
10
B. Evaluating the evidence:
Design B. Evaluating the evidence:
Design
B. Evaluating the evidence:
Design
B. Evaluating the evidence:
Design
B. Evaluating the evidence:
Design B. Evaluating the evidence:
Design
12/12/2013
11
B. Evaluating the evidence:
Design Practice Based Evidence
Conclusions: What can we do?
A. Use a client-specific PICO question. B. Apply the same critical and scientific
thinking to your clinical work: ◦ Watch quality of measurements
◦ Watch design set up
◦ Look for confounding variables.
C. Have a supported rationale for trying something new.
D. If there is no evidence in the lit., report yours!
Barriers to EBP use
Reported
Problem
Solutions
Access • Use databases. • Ask your local university.
Time • Lots of EBP articles exist, USE THEM.
• Tackle 1 topic at a time (Robey 2011).
Lack of evidence
or
Insufficient evidence
• Generate your own evidence: Always
have a sound rationale, try it, and report it! • Ask a different question for your search
• Seek the closest possible applicable evidence.
Contradictory
evidence
• Which is strongest? (see Evaluating the Evidence)
Limited training in
EBP and research.
• Congratulations! That’s why you are here.
Lack of information
literacy skills.
• Congratulations! That’s why you are here.
patrick.coppens@plattsburgh.edu
References American Speech-Language-Hearing Association.
(2005). Evidence-based practice in communication disorders [Position Statement]. Available from www.asha.org/policy.
Dollaghan, C. A. (2007). The handbook for evidence-based practice in communication disorders and sciences. Baltimore, MD: Paul Brookes.
Gillam, S., & Gillam, R. (2008). Teaching graduate students to make evidence-based decisions. Topics in Language Disorders, 28(3), 212-228.
Goldacre, B. (2008). Bad science. New York, NY: Faber & Faber.
References Kitson, A., Harvey, G., & McCormack, B. (1998).
Enabling the implementation of evidence based practice: a conceptual framework. Quality in Health Care, 7, 149-158.
Lemoncello, R., & Fanning, J. L. (2011, November). Practice-based evidence. Seminar presented at the ASHA meeting. San Diego, CA.
Lof, G. L. (2011). Science-based practice and the speech-language pathologist. International Journal of Speech-Language Pathology, 13(3), 189-196.
Lum, C. (2002). Scientific thinking in speech and language therapy. Mahwah, NJ: Lawrence Erlbaum.
Robey, R. (2011). Treatment effectiveness and evidence-based practice. In L. L. Lapointe (Ed.), Aphasia and related neurogenic language disorders (pp. 197-210). New York, NY: Thieme.